Is there some methods like 'datareader' in MySQLdb for handling massdata

Discussion in 'Python' started by Jeremy.Chen, Jan 6, 2009.

  1. Jeremy.Chen

    Jeremy.Chen Guest

    hi all:
    when I handled mass data table in mysql with python's MySQLdb lib,
    something frustrated me. I could't find any function like datareader,
    which yield one row by recording rows' anchor ,after each time I
    invoked 'read()' in a loop.fetchall or fetchmany just fetched all data
    in once, which take so much memory when meeting one mass data
    table.
    Is anybody know that function or other lib similar to MySQLdb ?
    thanks!
     
    Jeremy.Chen, Jan 6, 2009
    #1
    1. Advertising

  2. Re: Is there some methods like 'datareader' in MySQLdb for handlingmass data

    Jeremy.Chen a écrit :
    > hi all:
    > when I handled mass data table in mysql with python's MySQLdb lib,
    > something frustrated me. I could't find any function like datareader,
    > which yield one row by recording rows' anchor ,after each time I
    > invoked 'read()' in a loop.


    I think you want cursor.fetchone(). But read below...

    > fetchall or fetchmany just fetched all data
    > in once, which take so much memory when meeting one mass data
    > table.


    The cursor object is it's own iterator. So you just have to do:

    cursor.execute(your_query)
    for row in cursor:
    process(row)

    HTH
     
    Bruno Desthuilliers, Jan 6, 2009
    #2
    1. Advertising

  3. Jeremy.Chen

    Jeremy.Chen Guest

    Re: Is there some methods like 'datareader' in MySQLdb for handlingmass data

    On Jan 6, 7:29 pm, Bruno Desthuilliers <bruno.
    > wrote:
    > Jeremy.Chen a écrit :
    >
    > > hi all:
    > >   when I handled mass data table in mysql with python's MySQLdb lib,
    > > something frustrated me. I could't find any function like datareader,
    > > which yield one row by recording rows' anchor ,after each time I
    > > invoked 'read()' in a loop.

    >
    > I think you want cursor.fetchone(). But read below...
    >
    > > fetchall or fetchmany just fetched all data
    > > in once, which take so much memory when meeting one mass data
    > > table.

    >
    > The cursor object is it's own iterator. So you just have to do:
    >
    > cursor.execute(your_query)
    > for row in cursor:
    >     process(row)
    >
    > HTH


    That's what I want.
    I see,cursor also can be a iterator.
     
    Jeremy.Chen, Jan 6, 2009
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. James Brown
    Replies:
    1
    Views:
    5,437
    bruce barker
    Jun 26, 2003
  2. Arjen

    Datareader, there is no data...

    Arjen, Jul 30, 2003, in forum: ASP .Net
    Replies:
    2
    Views:
    830
  3. Martin Raychev
    Replies:
    2
    Views:
    3,358
    bruce barker
    Apr 22, 2004
  4. Sam Kong
    Replies:
    1
    Views:
    133
    Clifford Heath
    Aug 16, 2006
  5. Kenneth McDonald
    Replies:
    5
    Views:
    328
    Kenneth McDonald
    Sep 26, 2008
Loading...

Share This Page