managing multiple subprocesses

Discussion in 'Python' started by Marcos, Feb 3, 2005.

  1. Marcos

    Marcos Guest

    Hi guys,
    I realise this question has been answered in one form or another many
    times before but I can't quite find the solution I need. I am trying to
    run multiple subprocesses from a python script and then wait until all
    subprocesses have completed before continuing. The subprocesses run on
    other machines ie this is a coarse grained parallel computation task.

    So the commands I am using are:
    for x in range(0, limit):
    os.system("xgrid .... ./someprogram %d &" % x)

    and then to check whether they are all finished I do a very crude:

    joblist = commands.getoutput("xgrid .... -job list")
    while joblist != "{jobArray = (); }":
    time.sleep(1)
    joblist = commands.getoutput("xgrid .... -job list")

    Xgrid is a system for parallel computation on Mac OS X. It is quite
    good, check it out at www.apple.com/acg/xgrid. But anyway the method I
    am using above, while it works, it quite often hangs. I'd like to just
    run all the subprocesses at once and then check within the script,
    rather than an auxilliary, to see if everything has returned. The
    output of the commands is not important as that gets redirected to
    files outside the script. I have tried all sorts of popens / excevs /
    os.systems / commands etc etc. I realise the subprocess module may have
    what I need but I can't get python 2.4 on the Mac so I need a 2.3 based
    solution. Any help is much appreciated. Cheers.
    Marcos, Feb 3, 2005
    #1
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Jane Austine
    Replies:
    4
    Views:
    442
    Colin Brown
    Nov 13, 2003
  2. Fuzzyman
    Replies:
    0
    Views:
    338
    Fuzzyman
    Dec 22, 2003
  3. Marcos

    managing multiple subprocesses

    Marcos, Feb 3, 2005, in forum: Python
    Replies:
    4
    Views:
    596
    Fredrik Lundh
    Feb 7, 2005
  4. cypher543

    Managing a queue of subprocesses?

    cypher543, Dec 30, 2006, in forum: Python
    Replies:
    3
    Views:
    280
    Tom Plunket
    Jan 1, 2007
  5. JB
    Replies:
    1
    Views:
    198
    Jean-Michel Pichavant
    Jan 27, 2011
Loading...

Share This Page