Re: submit jobs on multi-core

Discussion in 'Python' started by Laszlo Nagy, Sep 11, 2012.

  1. Laszlo Nagy

    Laszlo Nagy Guest

    On 2012-09-11 06:16, Dhananjay wrote:
    > Dear all,
    >
    > I have a python script in which I have a list of files to input one by
    > one and for each file I get a number as an output.
    > I used for loop to submit the file to script.
    > My script uses one file at a time and returns the output.
    >
    > My computers has 8 cores.
    > Is there any way that I could submit 8 jobs at a time and get all the
    > output faster ?
    > In other words, how can I modify my script so that I could submit 8
    > jobs together on 8 different processors ?
    >
    > I am bit new to this stuff, please suggest me some directions.

    You should first look at the multiprocessing module. It is part of the
    standard library.

    http://docs.python.org/library/multiprocessing.html
    Laszlo Nagy, Sep 11, 2012
    #1
    1. Advertising

  2. Laszlo Nagy

    zig-zag Guest

    On 09/11/2012 07:53 AM, Laszlo Nagy wrote:
    > On 2012-09-11 06:16, Dhananjay wrote:
    >> Dear all,
    >>
    >> I have a python script in which I have a list of files to input one by
    >> one and for each file I get a number as an output.
    >> I used for loop to submit the file to script.
    >> My script uses one file at a time and returns the output.
    >>
    >> My computers has 8 cores.
    >> Is there any way that I could submit 8 jobs at a time and get all the
    >> output faster ?
    >> In other words, how can I modify my script so that I could submit 8
    >> jobs together on 8 different processors ?
    >>
    >> I am bit new to this stuff, please suggest me some directions.

    > You should first look at the multiprocessing module. It is part of the
    > standard library.
    >
    > http://docs.python.org/library/multiprocessing.html
    >
    >

    Did anybody tried this on a cluster?
    zig-zag, Sep 11, 2012
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. hardikh2000

    JOBs............JOBs............JOBs

    hardikh2000, Aug 16, 2005, in forum: Python
    Replies:
    0
    Views:
    397
    hardikh2000
    Aug 16, 2005
  2. nicky boy
    Replies:
    0
    Views:
    670
    nicky boy
    Feb 7, 2009
  3. nicky boy
    Replies:
    0
    Views:
    711
    nicky boy
    Feb 7, 2009
  4. Oscar Benjamin

    Re: submit jobs on multi-core

    Oscar Benjamin, Sep 11, 2012, in forum: Python
    Replies:
    0
    Views:
    200
    Oscar Benjamin
    Sep 11, 2012
  5. Jason Friedman

    Re: submit jobs on multi-core

    Jason Friedman, Sep 13, 2012, in forum: Python
    Replies:
    1
    Views:
    214
    Matej Cepl
    Sep 13, 2012
Loading...

Share This Page