# sum function

Discussion in 'Python' started by mike20007@gmail.com, Oct 4, 2012.

1. ### Guest

Hi All,

I am new to python and am getting the data from hbase.
I am trying to do sum on the column as below

scanner = client.scannerOpenWithStop("tab", "10", "1000", ["cf:col1"])
total = 0.0
r = client.scannerGet(scanner)
while r:
for k in (r[0].columns):
total += float(r[0].columns[k].value)
r = client.scannerGet(scanner)

print total

Do you know of better (faster) way to do sum?

Thanks

, Oct 4, 2012

2. ### Ian KellyGuest

On Thu, Oct 4, 2012 at 2:52 PM, <> wrote:
> scanner = client.scannerOpenWithStop("tab", "10", "1000", ["cf:col1"])
> total = 0.0
> r = client.scannerGet(scanner)
> while r:
> for k in (r[0].columns):
> total += float(r[0].columns[k].value)
> r = client.scannerGet(scanner)
>
> print total
>
> Do you know of better (faster) way to do sum?

scanner = client.scannerOpenWithStop("tab", "10", "1000", ["cf:col1"])
next_r = itertools.partial(client.scannerGet, scanner)
total = sum(float(col.value) for r in iter(next_r, None) for col in
r.itervalues())

Ian Kelly, Oct 4, 2012

3. ### Ian KellyGuest

On Thu, Oct 4, 2012 at 3:04 PM, Ian Kelly <> wrote:
> scanner = client.scannerOpenWithStop("tab", "10", "1000", ["cf:col1"])
> next_r = itertools.partial(client.scannerGet, scanner)
> total = sum(float(col.value) for r in iter(next_r, None) for col in
> r.itervalues())

That should be "functools" above, not "itertools".

Ian Kelly, Oct 4, 2012
4. ### MikeGuest

I get below error

NameError: name 'functools' is not defined

Thanks

Mike, Oct 4, 2012
5. ### MikeGuest

I get below error

NameError: name 'functools' is not defined

Thanks

Mike, Oct 4, 2012
6. ### MikeGuest

Thanks Ian for the quick reply.

I get the below error.

NameError: name 'itertools' is not defined

Thanks

Mike, Oct 4, 2012
7. ### MikeGuest

Thanks Ian for the quick reply.

I get the below error.

NameError: name 'itertools' is not defined

Thanks

Mike, Oct 4, 2012
8. ### Chris AngelicoGuest

On Fri, Oct 5, 2012 at 7:29 AM, Mike <> wrote:
> I get below error
>
> NameError: name 'functools' is not defined
>
> Thanks

functools is a module:

import functools

ChrisA

Chris Angelico, Oct 4, 2012
9. ### Dave AngelGuest

On 10/04/2012 05:29 PM, Mike wrote:
> I get below error
>
> NameError: name 'functools' is not defined
>

functools is a module in the standard library. You need to import it.

import functools

--

DaveA

Dave Angel, Oct 4, 2012
10. ### MikeGuest

On Thursday, October 4, 2012 5:40:26 PM UTC-4, Dave Angel wrote:
> On 10/04/2012 05:29 PM, Mike wrote:
>
> > I get below error

>
> >

>
> > NameError: name 'functools' is not defined

>
> >

>
>
>
> functools is a module in the standard library. You need to import it.
>
>
>
> import functools
>
>
>
>
>
>
>
> --
>
>
>
> DaveA

I imported functools. Now I get the below error please.

Traceback (most recent call last):
File "test.py", line 16, in <module>
total = sum(float(col.value) for r in iter(next_r, None) for col in r.itervalues())
File "test.py", line 16, in <genexpr>
total = sum(float(col.value) for r in iter(next_r, None) for col in r.itervalues())
AttributeError: 'list' object has no attribute 'itervalues'

Thanks

Mike, Oct 5, 2012
11. ### MikeGuest

On Thursday, October 4, 2012 5:40:26 PM UTC-4, Dave Angel wrote:
> On 10/04/2012 05:29 PM, Mike wrote:
>
> > I get below error

>
> >

>
> > NameError: name 'functools' is not defined

>
> >

>
>
>
> functools is a module in the standard library. You need to import it.
>
>
>
> import functools
>
>
>
>
>
>
>
> --
>
>
>
> DaveA

I imported functools. Now I get the below error please.

Traceback (most recent call last):
File "test.py", line 16, in <module>
total = sum(float(col.value) for r in iter(next_r, None) for col in r.itervalues())
File "test.py", line 16, in <genexpr>
total = sum(float(col.value) for r in iter(next_r, None) for col in r.itervalues())
AttributeError: 'list' object has no attribute 'itervalues'

Thanks

Mike, Oct 5, 2012
12. ### MikeGuest

I agree with you, Ian. Thanks for all the help. Now I get the below error.

File "test.py", line 17, in <module>
total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
File "test.py", line 17, in <genexpr>
total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())

Thanks

Mike, Oct 5, 2012
13. ### MikeGuest

I agree with you, Ian. Thanks for all the help. Now I get the below error.

File "test.py", line 17, in <module>
total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
File "test.py", line 17, in <genexpr>
total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())

Thanks

Mike, Oct 5, 2012
14. ### Ramchandra ApteGuest

On Friday, 5 October 2012 07:31:24 UTC+5:30, Mike wrote:
> I agree with you, Ian. Thanks for all the help. Now I get the below error.
>
>
>
> File "test.py", line 17, in <module>
>
> total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
>
> File "test.py", line 17, in <genexpr>
>
> total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
>
>
>
> Thanks

You have missed the last line of the traceback (error)

Ramchandra Apte, Oct 5, 2012
15. ### Ramchandra ApteGuest

On Friday, 5 October 2012 07:31:24 UTC+5:30, Mike wrote:
> I agree with you, Ian. Thanks for all the help. Now I get the below error.
>
>
>
> File "test.py", line 17, in <module>
>
> total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
>
> File "test.py", line 17, in <genexpr>
>
> total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
>
>
>
> Thanks

You have missed the last line of the traceback (error)

Ramchandra Apte, Oct 5, 2012
16. ### MikeGuest

On Thursday, October 4, 2012 4:52:50 PM UTC-4, Mike wrote:
> Hi All,
>
>
>
> I am new to python and am getting the data from hbase.
>
> I am trying to do sum on the column as below
>
>
>
>
>
> scanner = client.scannerOpenWithStop("tab", "10", "1000", ["cf:col1"])
>
> total = 0.0
>
> r = client.scannerGet(scanner)
>
> while r:
>
> for k in (r[0].columns):
>
> total += float(r[0].columns[k].value)
>
> r = client.scannerGet(scanner)
>
>
>
> print total
>
>
>
> Do you know of better (faster) way to do sum?
>
>
>
>
>
>
> Thanks

Sorry about that. Here you go

Traceback (most recent call last):
File "test.py", line 17, in <module>
total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
File "test.py", line 17, in <genexpr>
total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
IndexError: list index out of range

Mike, Oct 5, 2012
17. ### Ramchandra ApteGuest

On Friday, 5 October 2012 19:09:15 UTC+5:30, Mike wrote:
> On Thursday, October 4, 2012 4:52:50 PM UTC-4, Mike wrote:
>
> > Hi All,

>
> >

>
> >

>
> >

>
> > I am new to python and am getting the data from hbase.

>
> >

>
> > I am trying to do sum on the column as below

>
> >

>
> >

>
> >

>
> >

>
> >

>
> > scanner = client.scannerOpenWithStop("tab", "10", "1000", ["cf:col1"])

>
> >

>
> > total = 0.0

>
> >

>
> > r = client.scannerGet(scanner)

>
> >

>
> > while r:

>
> >

>
> > for k in (r[0].columns):

>
> >

>
> > total += float(r[0].columns[k].value)

>
> >

>
> > r = client.scannerGet(scanner)

>
> >

>
> >

>
> >

>
> > print total

>
> >

>
> >

>
> >

>
> > Do you know of better (faster) way to do sum?

>
> >

>
> >

>
> >

>

>
> >

>
> >

>
> >

>
> > Thanks

>
>
>
> Sorry about that. Here you go
>
>
>
> Traceback (most recent call last):
>
> File "test.py", line 17, in <module>
>
> total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
>
> File "test.py", line 17, in <genexpr>
>
> total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
>
> IndexError: list index out of range

the variable "r" is an empty list

Ramchandra Apte, Oct 5, 2012
18. ### MikeGuest

On Friday, October 5, 2012 9:41:44 AM UTC-4, Ramchandra Apte wrote:
> On Friday, 5 October 2012 19:09:15 UTC+5:30, Mike wrote:
>
> > On Thursday, October 4, 2012 4:52:50 PM UTC-4, Mike wrote:

>
> >

>
> > > Hi All,

>
> >

>
> > >

>
> >

>
> > >

>
> >

>
> > >

>
> >

>
> > > I am new to python and am getting the data from hbase.

>
> >

>
> > >

>
> >

>
> > > I am trying to do sum on the column as below

>
> >

>
> > >

>
> >

>
> > >

>
> >

>
> > >

>
> >

>
> > >

>
> >

>
> > >

>
> >

>
> > > scanner = client.scannerOpenWithStop("tab", "10", "1000", ["cf:col1"])

>
> >

>
> > >

>
> >

>
> > > total = 0.0

>
> >

>
> > >

>
> >

>
> > > r = client.scannerGet(scanner)

>
> >

>
> > >

>
> >

>
> > > while r:

>
> >

>
> > >

>
> >

>
> > > for k in (r[0].columns):

>
> >

>
> > >

>
> >

>
> > > total += float(r[0].columns[k].value)

>
> >

>
> > >

>
> >

>
> > > r = client.scannerGet(scanner)

>
> >

>
> > >

>
> >

>
> > >

>
> >

>
> > >

>
> >

>
> > > print total

>
> >

>
> > >

>
> >

>
> > >

>
> >

>
> > >

>
> >

>
> > > Do you know of better (faster) way to do sum?

>
> >

>
> > >

>
> >

>
> > >

>
> >

>
> > >

>
> >

>
> > > Any thoughts please?

>
> >

>
> > >

>
> >

>
> > >

>
> >

>
> > >

>
> >

>
> > > Thanks

>
> >

>
> >

>
> >

>
> > Sorry about that. Here you go

>
> >

>
> >

>
> >

>
> > Traceback (most recent call last):

>
> >

>
> > File "test.py", line 17, in <module>

>
> >

>
> > total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())

>
> >

>
> > File "test.py", line 17, in <genexpr>

>
> >

>
> > total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())

>
> >

>
> > IndexError: list index out of range

>
>
>
> the variable "r" is an empty list

Here is the actual code.

scanner = client.scannerOpenWithStop("tab", "10", "1000", ["cf:col1"])
next_r = functools.partial(client.scannerGet, scanner)
total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())

Scanner does have rows.

Thanks

Mike, Oct 5, 2012
19. ### Terry ReedyGuest

On 10/5/2012 9:47 AM, Mike wrote:
> On Friday, October 5, 2012 9:41:44 AM UTC-4, Ramchandra Apte wrote:
>> On Friday, 5 October 2012 19:09:15 UTC+5:30, Mike wrote:
>>
>>> On Thursday, October 4, 2012 4:52:50 PM UTC-4, Mike wrote:

>>
>>>

>>
>>>> Hi All,

>>
>>>

>>
>>>>

>>
>>>

>>
>>>>

>>
>>>

>>
>>>>

>>
>>>

>>
>>>> I am new to python and am getting the data from hbase.

If you want as many people as possible to read your posts, stop using a
mail-agent and site that spits in the face of readers by doubling blank
lines each iteration. Alternatives have been discussed previously.

--
Terry Jan Reedy

Terry Reedy, Oct 5, 2012
20. ### Ian KellyGuest

On Fri, Oct 5, 2012 at 7:39 AM, Mike <> wrote:
> Sorry about that. Here you go
>
> Traceback (most recent call last):
> File "test.py", line 17, in <module>
> total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
> File "test.py", line 17, in <genexpr>
> total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
> IndexError: list index out of range

Maybe the sentinel value is not None as I assumed, and it's
overrunning the end of the data? What does
client.scannerGet return when there is no more data?

Ian Kelly, Oct 5, 2012