Is there some methods like 'datareader' in MySQLdb for handling massdata

Discussion in 'Python' started by Jeremy.Chen, Jan 6, 2009.

  1. Jeremy.Chen

    Jeremy.Chen Guest

    hi all:
    when I handled mass data table in mysql with python's MySQLdb lib,
    something frustrated me. I could't find any function like datareader,
    which yield one row by recording rows' anchor ,after each time I
    invoked 'read()' in a loop.fetchall or fetchmany just fetched all data
    in once, which take so much memory when meeting one mass data
    table.
    Is anybody know that function or other lib similar to MySQLdb ?
    thanks!
     
    Jeremy.Chen, Jan 6, 2009
    #1
    1. Advertisements

  2. Re: Is there some methods like 'datareader' in MySQLdb for handlingmass data

    Jeremy.Chen a écrit :
    > hi all:
    > when I handled mass data table in mysql with python's MySQLdb lib,
    > something frustrated me. I could't find any function like datareader,
    > which yield one row by recording rows' anchor ,after each time I
    > invoked 'read()' in a loop.


    I think you want cursor.fetchone(). But read below...

    > fetchall or fetchmany just fetched all data
    > in once, which take so much memory when meeting one mass data
    > table.


    The cursor object is it's own iterator. So you just have to do:

    cursor.execute(your_query)
    for row in cursor:
    process(row)

    HTH
     
    Bruno Desthuilliers, Jan 6, 2009
    #2
    1. Advertisements

  3. Jeremy.Chen

    Jeremy.Chen Guest

    Re: Is there some methods like 'datareader' in MySQLdb for handlingmass data

    On Jan 6, 7:29 pm, Bruno Desthuilliers <bruno.
    > wrote:
    > Jeremy.Chen a écrit :
    >
    > > hi all:
    > >   when I handled mass data table in mysql with python's MySQLdb lib,
    > > something frustrated me. I could't find any function like datareader,
    > > which yield one row by recording rows' anchor ,after each time I
    > > invoked 'read()' in a loop.

    >
    > I think you want cursor.fetchone(). But read below...
    >
    > > fetchall or fetchmany just fetched all data
    > > in once, which take so much memory when meeting one mass data
    > > table.

    >
    > The cursor object is it's own iterator. So you just have to do:
    >
    > cursor.execute(your_query)
    > for row in cursor:
    >     process(row)
    >
    > HTH


    That's what I want.
    I see,cursor also can be a iterator.
     
    Jeremy.Chen, Jan 6, 2009
    #3
    1. Advertisements

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. James Brown
    Replies:
    1
    Views:
    5,620
    bruce barker
    Jun 26, 2003
  2. Martin Raychev
    Replies:
    2
    Views:
    3,513
    bruce barker
    Apr 22, 2004
  3. baur79
    Replies:
    6
    Views:
    650
    baur79
    Feb 2, 2007
  4. Patrick Kowalzick
    Replies:
    5
    Views:
    743
    Patrick Kowalzick
    Mar 14, 2006
  5. dolphin
    Replies:
    3
    Views:
    493
    rossum
    Mar 9, 2007
  6. Sam Kong
    Replies:
    1
    Views:
    186
    Clifford Heath
    Aug 16, 2006
  7. Michael W. Ryder
    Replies:
    10
    Views:
    288
    Mark Wilden
    May 21, 2008
  8. Kenneth McDonald
    Replies:
    5
    Views:
    652
    Kenneth McDonald
    Sep 26, 2008
Loading...