sum function


M

mike20007

Hi All,

I am new to python and am getting the data from hbase.
I am trying to do sum on the column as below


scanner = client.scannerOpenWithStop("tab", "10", "1000", ["cf:col1"])
total = 0.0
r = client.scannerGet(scanner)
while r:
for k in (r[0].columns):
total += float(r[0].columns[k].value)
r = client.scannerGet(scanner)

print total

Do you know of better (faster) way to do sum?

Any thoughts please?

Thanks
 
Ad

Advertisements

I

Ian Kelly

scanner = client.scannerOpenWithStop("tab", "10", "1000", ["cf:col1"])
total = 0.0
r = client.scannerGet(scanner)
while r:
for k in (r[0].columns):
total += float(r[0].columns[k].value)
r = client.scannerGet(scanner)

print total

Do you know of better (faster) way to do sum?

scanner = client.scannerOpenWithStop("tab", "10", "1000", ["cf:col1"])
next_r = itertools.partial(client.scannerGet, scanner)
total = sum(float(col.value) for r in iter(next_r, None) for col in
r.itervalues())
 
I

Ian Kelly

scanner = client.scannerOpenWithStop("tab", "10", "1000", ["cf:col1"])
next_r = itertools.partial(client.scannerGet, scanner)
total = sum(float(col.value) for r in iter(next_r, None) for col in
r.itervalues())

That should be "functools" above, not "itertools". :p
 
M

Mike

Thanks Ian for the quick reply.

I get the below error.

NameError: name 'itertools' is not defined

Thanks
 
Ad

Advertisements

M

Mike

Thanks Ian for the quick reply.

I get the below error.

NameError: name 'itertools' is not defined

Thanks
 
D

Dave Angel

I get below error

NameError: name 'functools' is not defined

functools is a module in the standard library. You need to import it.

import functools
 
M

Mike

functools is a module in the standard library. You need to import it.



import functools







--



DaveA

I imported functools. Now I get the below error please.


Traceback (most recent call last):
File "test.py", line 16, in <module>
total = sum(float(col.value) for r in iter(next_r, None) for col in r.itervalues())
File "test.py", line 16, in <genexpr>
total = sum(float(col.value) for r in iter(next_r, None) for col in r.itervalues())
AttributeError: 'list' object has no attribute 'itervalues'

Thanks
 
M

Mike

functools is a module in the standard library. You need to import it.



import functools







--



DaveA

I imported functools. Now I get the below error please.


Traceback (most recent call last):
File "test.py", line 16, in <module>
total = sum(float(col.value) for r in iter(next_r, None) for col in r.itervalues())
File "test.py", line 16, in <genexpr>
total = sum(float(col.value) for r in iter(next_r, None) for col in r.itervalues())
AttributeError: 'list' object has no attribute 'itervalues'

Thanks
 
Ad

Advertisements

M

Mike

I agree with you, Ian. Thanks for all the help. Now I get the below error.

File "test.py", line 17, in <module>
total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
File "test.py", line 17, in <genexpr>
total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())

Thanks
 
M

Mike

I agree with you, Ian. Thanks for all the help. Now I get the below error.

File "test.py", line 17, in <module>
total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
File "test.py", line 17, in <genexpr>
total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())

Thanks
 
R

Ramchandra Apte

I agree with you, Ian. Thanks for all the help. Now I get the below error.



File "test.py", line 17, in <module>

total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())

File "test.py", line 17, in <genexpr>

total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())



Thanks

You have missed the last line of the traceback (error)
 
R

Ramchandra Apte

I agree with you, Ian. Thanks for all the help. Now I get the below error.



File "test.py", line 17, in <module>

total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())

File "test.py", line 17, in <genexpr>

total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())



Thanks

You have missed the last line of the traceback (error)
 
M

Mike

Hi All,



I am new to python and am getting the data from hbase.

I am trying to do sum on the column as below





scanner = client.scannerOpenWithStop("tab", "10", "1000", ["cf:col1"])

total = 0.0

r = client.scannerGet(scanner)

while r:

for k in (r[0].columns):

total += float(r[0].columns[k].value)

r = client.scannerGet(scanner)



print total



Do you know of better (faster) way to do sum?



Any thoughts please?



Thanks

Sorry about that. Here you go

Traceback (most recent call last):
File "test.py", line 17, in <module>
total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
File "test.py", line 17, in <genexpr>
total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
IndexError: list index out of range
 
Ad

Advertisements

R

Ramchandra Apte

Hi All,
I am new to python and am getting the data from hbase.
I am trying to do sum on the column as below
scanner = client.scannerOpenWithStop("tab", "10", "1000", ["cf:col1"])
total = 0.0
r = client.scannerGet(scanner)
while r:
for k in (r[0].columns):
total += float(r[0].columns[k].value)
r = client.scannerGet(scanner)
print total
Do you know of better (faster) way to do sum?
Any thoughts please?
Thanks



Sorry about that. Here you go



Traceback (most recent call last):

File "test.py", line 17, in <module>

total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())

File "test.py", line 17, in <genexpr>

total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())

IndexError: list index out of range

the variable "r" is an empty list
 
M

Mike

I am new to python and am getting the data from hbase.
I am trying to do sum on the column as below
scanner = client.scannerOpenWithStop("tab", "10", "1000", ["cf:col1"])
total = 0.0
r = client.scannerGet(scanner)
for k in (r[0].columns):
total += float(r[0].columns[k].value)
r = client.scannerGet(scanner)
print total
Do you know of better (faster) way to do sum?
Any thoughts please?
Sorry about that. Here you go
Traceback (most recent call last):
File "test.py", line 17, in <module>
total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
File "test.py", line 17, in <genexpr>
total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
IndexError: list index out of range



the variable "r" is an empty list

Here is the actual code.

scanner = client.scannerOpenWithStop("tab", "10", "1000", ["cf:col1"])
next_r = functools.partial(client.scannerGet, scanner)
total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())


Scanner does have rows.

Are we missing something please?

Thanks
 
T

Terry Reedy

If you want as many people as possible to read your posts, stop using a
mail-agent and site that spits in the face of readers by doubling blank
lines each iteration. Alternatives have been discussed previously.
 
Ad

Advertisements

I

Ian Kelly

Sorry about that. Here you go

Traceback (most recent call last):
File "test.py", line 17, in <module>
total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
File "test.py", line 17, in <genexpr>
total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
IndexError: list index out of range

Maybe the sentinel value is not None as I assumed, and it's
overrunning the end of the data? What does
client.scannerGet return when there is no more data?
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Top