Question about parallel processing for bulk inserts into DB

Discussion in 'Python' started by harshu010, May 28, 2008.

  1. harshu010

    harshu010

    Joined:
    May 23, 2008
    Messages:
    10
    Hi,

    I am trying to insert multiple records in a number of few 100 thousands.
    This is crashing python. Can i implement parallel processing. I have
    read about it , but don't know how to implement.

    cursor.execute(("insert into tbl_query_detail(hash_value,file_name,package_name,QUERY_NUMBER,\
    author_name,FULL_QUERY,BASE_QUERY_INDICATOR) values \
    (A.NEXTVAL,:file_name,:package_name,:QUERY_NUMBER,:author_name,:FULL_QUERY,:BASE_QUERY_INDICATOR )"),(filename,Result_list[0],Result_list[ResultListIndex],'abc',Result_list[ResultListIndex + 1], Result_list[ResultListIndex +2]))


    My code is actually extracting queries from the code and inserting into
    database . That makes almost 1,00,000 queries to be
    inserted into database.
    THis is a one time activity. SO i am not bothered about performance. But more worried about to see that thepython doesnt get hung in between and insert into db.
    I have been using DCOracle2 on unix where it was working fine. I
    have shifted to windows and using cx_Oracle as the DB-API.


    Which is the best strategy to be followed please suggest.
     
    harshu010, May 28, 2008
    #1
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. michi
    Replies:
    0
    Views:
    336
    michi
    Oct 27, 2004
  2. Derick van Niekerk
    Replies:
    8
    Views:
    479
    Carsten Haese
    Jan 27, 2006
  3. Fendi Baba
    Replies:
    1
    Views:
    508
    Alvin Bruney [MVP]
    Feb 11, 2007
  4. Soren
    Replies:
    4
    Views:
    1,305
    c d saunter
    Feb 14, 2008
  5. MrHelpMe
    Replies:
    1
    Views:
    128
    ThatsIT.net.au
    Apr 4, 2007
Loading...

Share This Page