sum function

Discussion in 'Python' started by mike20007@gmail.com, Oct 4, 2012.

  1. Guest

    Hi All,

    I am new to python and am getting the data from hbase.
    I am trying to do sum on the column as below


    scanner = client.scannerOpenWithStop("tab", "10", "1000", ["cf:col1"])
    total = 0.0
    r = client.scannerGet(scanner)
    while r:
    for k in (r[0].columns):
    total += float(r[0].columns[k].value)
    r = client.scannerGet(scanner)

    print total

    Do you know of better (faster) way to do sum?

    Any thoughts please?

    Thanks
     
    , Oct 4, 2012
    #1
    1. Advertising

  2. Ian Kelly Guest

    On Thu, Oct 4, 2012 at 2:52 PM, <> wrote:
    > scanner = client.scannerOpenWithStop("tab", "10", "1000", ["cf:col1"])
    > total = 0.0
    > r = client.scannerGet(scanner)
    > while r:
    > for k in (r[0].columns):
    > total += float(r[0].columns[k].value)
    > r = client.scannerGet(scanner)
    >
    > print total
    >
    > Do you know of better (faster) way to do sum?


    scanner = client.scannerOpenWithStop("tab", "10", "1000", ["cf:col1"])
    next_r = itertools.partial(client.scannerGet, scanner)
    total = sum(float(col.value) for r in iter(next_r, None) for col in
    r.itervalues())
     
    Ian Kelly, Oct 4, 2012
    #2
    1. Advertising

  3. Ian Kelly Guest

    On Thu, Oct 4, 2012 at 3:04 PM, Ian Kelly <> wrote:
    > scanner = client.scannerOpenWithStop("tab", "10", "1000", ["cf:col1"])
    > next_r = itertools.partial(client.scannerGet, scanner)
    > total = sum(float(col.value) for r in iter(next_r, None) for col in
    > r.itervalues())


    That should be "functools" above, not "itertools". :p
     
    Ian Kelly, Oct 4, 2012
    #3
  4. Mike Guest

    I get below error

    NameError: name 'functools' is not defined

    Thanks
     
    Mike, Oct 4, 2012
    #4
  5. Mike Guest

    I get below error

    NameError: name 'functools' is not defined

    Thanks
     
    Mike, Oct 4, 2012
    #5
  6. Mike Guest

    Thanks Ian for the quick reply.

    I get the below error.

    NameError: name 'itertools' is not defined

    Thanks
     
    Mike, Oct 4, 2012
    #6
  7. Mike Guest

    Thanks Ian for the quick reply.

    I get the below error.

    NameError: name 'itertools' is not defined

    Thanks
     
    Mike, Oct 4, 2012
    #7
  8. On Fri, Oct 5, 2012 at 7:29 AM, Mike <> wrote:
    > I get below error
    >
    > NameError: name 'functools' is not defined
    >
    > Thanks


    functools is a module:

    import functools

    ChrisA
     
    Chris Angelico, Oct 4, 2012
    #8
  9. Dave Angel Guest

    On 10/04/2012 05:29 PM, Mike wrote:
    > I get below error
    >
    > NameError: name 'functools' is not defined
    >


    functools is a module in the standard library. You need to import it.

    import functools



    --

    DaveA
     
    Dave Angel, Oct 4, 2012
    #9
  10. Mike Guest

    On Thursday, October 4, 2012 5:40:26 PM UTC-4, Dave Angel wrote:
    > On 10/04/2012 05:29 PM, Mike wrote:
    >
    > > I get below error

    >
    > >

    >
    > > NameError: name 'functools' is not defined

    >
    > >

    >
    >
    >
    > functools is a module in the standard library. You need to import it.
    >
    >
    >
    > import functools
    >
    >
    >
    >
    >
    >
    >
    > --
    >
    >
    >
    > DaveA


    I imported functools. Now I get the below error please.


    Traceback (most recent call last):
    File "test.py", line 16, in <module>
    total = sum(float(col.value) for r in iter(next_r, None) for col in r.itervalues())
    File "test.py", line 16, in <genexpr>
    total = sum(float(col.value) for r in iter(next_r, None) for col in r.itervalues())
    AttributeError: 'list' object has no attribute 'itervalues'

    Thanks
     
    Mike, Oct 5, 2012
    #10
  11. Mike Guest

    On Thursday, October 4, 2012 5:40:26 PM UTC-4, Dave Angel wrote:
    > On 10/04/2012 05:29 PM, Mike wrote:
    >
    > > I get below error

    >
    > >

    >
    > > NameError: name 'functools' is not defined

    >
    > >

    >
    >
    >
    > functools is a module in the standard library. You need to import it.
    >
    >
    >
    > import functools
    >
    >
    >
    >
    >
    >
    >
    > --
    >
    >
    >
    > DaveA


    I imported functools. Now I get the below error please.


    Traceback (most recent call last):
    File "test.py", line 16, in <module>
    total = sum(float(col.value) for r in iter(next_r, None) for col in r.itervalues())
    File "test.py", line 16, in <genexpr>
    total = sum(float(col.value) for r in iter(next_r, None) for col in r.itervalues())
    AttributeError: 'list' object has no attribute 'itervalues'

    Thanks
     
    Mike, Oct 5, 2012
    #11
  12. Mike Guest

    I agree with you, Ian. Thanks for all the help. Now I get the below error.

    File "test.py", line 17, in <module>
    total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
    File "test.py", line 17, in <genexpr>
    total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())

    Thanks
     
    Mike, Oct 5, 2012
    #12
  13. Mike Guest

    I agree with you, Ian. Thanks for all the help. Now I get the below error.

    File "test.py", line 17, in <module>
    total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
    File "test.py", line 17, in <genexpr>
    total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())

    Thanks
     
    Mike, Oct 5, 2012
    #13
  14. On Friday, 5 October 2012 07:31:24 UTC+5:30, Mike wrote:
    > I agree with you, Ian. Thanks for all the help. Now I get the below error.
    >
    >
    >
    > File "test.py", line 17, in <module>
    >
    > total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
    >
    > File "test.py", line 17, in <genexpr>
    >
    > total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
    >
    >
    >
    > Thanks


    You have missed the last line of the traceback (error)
     
    Ramchandra Apte, Oct 5, 2012
    #14
  15. On Friday, 5 October 2012 07:31:24 UTC+5:30, Mike wrote:
    > I agree with you, Ian. Thanks for all the help. Now I get the below error.
    >
    >
    >
    > File "test.py", line 17, in <module>
    >
    > total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
    >
    > File "test.py", line 17, in <genexpr>
    >
    > total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
    >
    >
    >
    > Thanks


    You have missed the last line of the traceback (error)
     
    Ramchandra Apte, Oct 5, 2012
    #15
  16. Mike Guest

    On Thursday, October 4, 2012 4:52:50 PM UTC-4, Mike wrote:
    > Hi All,
    >
    >
    >
    > I am new to python and am getting the data from hbase.
    >
    > I am trying to do sum on the column as below
    >
    >
    >
    >
    >
    > scanner = client.scannerOpenWithStop("tab", "10", "1000", ["cf:col1"])
    >
    > total = 0.0
    >
    > r = client.scannerGet(scanner)
    >
    > while r:
    >
    > for k in (r[0].columns):
    >
    > total += float(r[0].columns[k].value)
    >
    > r = client.scannerGet(scanner)
    >
    >
    >
    > print total
    >
    >
    >
    > Do you know of better (faster) way to do sum?
    >
    >
    >
    > Any thoughts please?
    >
    >
    >
    > Thanks


    Sorry about that. Here you go

    Traceback (most recent call last):
    File "test.py", line 17, in <module>
    total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
    File "test.py", line 17, in <genexpr>
    total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
    IndexError: list index out of range
     
    Mike, Oct 5, 2012
    #16
  17. On Friday, 5 October 2012 19:09:15 UTC+5:30, Mike wrote:
    > On Thursday, October 4, 2012 4:52:50 PM UTC-4, Mike wrote:
    >
    > > Hi All,

    >
    > >

    >
    > >

    >
    > >

    >
    > > I am new to python and am getting the data from hbase.

    >
    > >

    >
    > > I am trying to do sum on the column as below

    >
    > >

    >
    > >

    >
    > >

    >
    > >

    >
    > >

    >
    > > scanner = client.scannerOpenWithStop("tab", "10", "1000", ["cf:col1"])

    >
    > >

    >
    > > total = 0.0

    >
    > >

    >
    > > r = client.scannerGet(scanner)

    >
    > >

    >
    > > while r:

    >
    > >

    >
    > > for k in (r[0].columns):

    >
    > >

    >
    > > total += float(r[0].columns[k].value)

    >
    > >

    >
    > > r = client.scannerGet(scanner)

    >
    > >

    >
    > >

    >
    > >

    >
    > > print total

    >
    > >

    >
    > >

    >
    > >

    >
    > > Do you know of better (faster) way to do sum?

    >
    > >

    >
    > >

    >
    > >

    >
    > > Any thoughts please?

    >
    > >

    >
    > >

    >
    > >

    >
    > > Thanks

    >
    >
    >
    > Sorry about that. Here you go
    >
    >
    >
    > Traceback (most recent call last):
    >
    > File "test.py", line 17, in <module>
    >
    > total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
    >
    > File "test.py", line 17, in <genexpr>
    >
    > total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
    >
    > IndexError: list index out of range


    the variable "r" is an empty list
     
    Ramchandra Apte, Oct 5, 2012
    #17
  18. Mike Guest

    On Friday, October 5, 2012 9:41:44 AM UTC-4, Ramchandra Apte wrote:
    > On Friday, 5 October 2012 19:09:15 UTC+5:30, Mike wrote:
    >
    > > On Thursday, October 4, 2012 4:52:50 PM UTC-4, Mike wrote:

    >
    > >

    >
    > > > Hi All,

    >
    > >

    >
    > > >

    >
    > >

    >
    > > >

    >
    > >

    >
    > > >

    >
    > >

    >
    > > > I am new to python and am getting the data from hbase.

    >
    > >

    >
    > > >

    >
    > >

    >
    > > > I am trying to do sum on the column as below

    >
    > >

    >
    > > >

    >
    > >

    >
    > > >

    >
    > >

    >
    > > >

    >
    > >

    >
    > > >

    >
    > >

    >
    > > >

    >
    > >

    >
    > > > scanner = client.scannerOpenWithStop("tab", "10", "1000", ["cf:col1"])

    >
    > >

    >
    > > >

    >
    > >

    >
    > > > total = 0.0

    >
    > >

    >
    > > >

    >
    > >

    >
    > > > r = client.scannerGet(scanner)

    >
    > >

    >
    > > >

    >
    > >

    >
    > > > while r:

    >
    > >

    >
    > > >

    >
    > >

    >
    > > > for k in (r[0].columns):

    >
    > >

    >
    > > >

    >
    > >

    >
    > > > total += float(r[0].columns[k].value)

    >
    > >

    >
    > > >

    >
    > >

    >
    > > > r = client.scannerGet(scanner)

    >
    > >

    >
    > > >

    >
    > >

    >
    > > >

    >
    > >

    >
    > > >

    >
    > >

    >
    > > > print total

    >
    > >

    >
    > > >

    >
    > >

    >
    > > >

    >
    > >

    >
    > > >

    >
    > >

    >
    > > > Do you know of better (faster) way to do sum?

    >
    > >

    >
    > > >

    >
    > >

    >
    > > >

    >
    > >

    >
    > > >

    >
    > >

    >
    > > > Any thoughts please?

    >
    > >

    >
    > > >

    >
    > >

    >
    > > >

    >
    > >

    >
    > > >

    >
    > >

    >
    > > > Thanks

    >
    > >

    >
    > >

    >
    > >

    >
    > > Sorry about that. Here you go

    >
    > >

    >
    > >

    >
    > >

    >
    > > Traceback (most recent call last):

    >
    > >

    >
    > > File "test.py", line 17, in <module>

    >
    > >

    >
    > > total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())

    >
    > >

    >
    > > File "test.py", line 17, in <genexpr>

    >
    > >

    >
    > > total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())

    >
    > >

    >
    > > IndexError: list index out of range

    >
    >
    >
    > the variable "r" is an empty list


    Here is the actual code.

    scanner = client.scannerOpenWithStop("tab", "10", "1000", ["cf:col1"])
    next_r = functools.partial(client.scannerGet, scanner)
    total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())


    Scanner does have rows.

    Are we missing something please?

    Thanks
     
    Mike, Oct 5, 2012
    #18
  19. Terry Reedy Guest

    On 10/5/2012 9:47 AM, Mike wrote:
    > On Friday, October 5, 2012 9:41:44 AM UTC-4, Ramchandra Apte wrote:
    >> On Friday, 5 October 2012 19:09:15 UTC+5:30, Mike wrote:
    >>
    >>> On Thursday, October 4, 2012 4:52:50 PM UTC-4, Mike wrote:

    >>
    >>>

    >>
    >>>> Hi All,

    >>
    >>>

    >>
    >>>>

    >>
    >>>

    >>
    >>>>

    >>
    >>>

    >>
    >>>>

    >>
    >>>

    >>
    >>>> I am new to python and am getting the data from hbase.


    If you want as many people as possible to read your posts, stop using a
    mail-agent and site that spits in the face of readers by doubling blank
    lines each iteration. Alternatives have been discussed previously.

    --
    Terry Jan Reedy
     
    Terry Reedy, Oct 5, 2012
    #19
  20. Ian Kelly Guest

    On Fri, Oct 5, 2012 at 7:39 AM, Mike <> wrote:
    > Sorry about that. Here you go
    >
    > Traceback (most recent call last):
    > File "test.py", line 17, in <module>
    > total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
    > File "test.py", line 17, in <genexpr>
    > total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
    > IndexError: list index out of range


    Maybe the sentinel value is not None as I assumed, and it's
    overrunning the end of the data? What does
    client.scannerGet return when there is no more data?
     
    Ian Kelly, Oct 5, 2012
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Roger Moore
    Replies:
    0
    Views:
    750
    Roger Moore
    Jul 7, 2003
  2. Davisro
    Replies:
    4
    Views:
    792
    Bruno Sirianni
    Apr 10, 2004
  3. cieron
    Replies:
    2
    Views:
    715
    Dimitre Novatchev
    May 16, 2005
  4. Weng Tianxiang
    Replies:
    2
    Views:
    465
  5. Russel Walker
    Replies:
    9
    Views:
    115
    Russel Walker
    Jul 7, 2013
Loading...

Share This Page