Main-process doesn't wait for child-processes

Discussion in 'Python' started by Markus Franz, Apr 22, 2004.

  1. Markus Franz

    Markus Franz Guest

    Hi.

    I created a little script:

    for currenturl in sys.argv[1:]:
    pid = os.fork()
    if pid == 0:
    signal.alarm(10)
    do_something() # placeholder for the download and print
    routine
    break

    This script is started by ./script.py http://www.website1.com
    http://www.website2.com
    As you see it creates child-processes for loading all websites specified on
    the command line. (So all websites are loaded in parallel.) Each
    child-process simply load a websites and print the contents to STDOUT (above
    replaced by do_something).
    (With signal.alarm(10) I make sure that each child-process will not run
    longer than 10 seconds.)


    But now I have a difficult problem: This script is executed by a
    comand-line-function (shell_exec) in PHP, the response of the Python-script
    is used inside the PHP application. In the python script the main process
    seem to stop immediately after starting the script, only the child-processes
    seem to run. Because of this my PHP application will not wait untill the
    child-processes have finised and so I can't use the contents of the
    websites. (I think the solution should be that the main-process waits untill
    the child-processes terminate.)

    Now my question is: How can I force the main-process to wait untill every
    child-process is finished?
    I tried:

    for currenturl in sys.argv[1:]:
    pid = os.fork()
    if pid == 0:
    signal.alarm(10)
    do_something() # placeholder for the download and print
    routine
    break
    else:
    os.wait()

    The version above behaves like a script that loads all given websites in
    sequence...
    I also tried:

    for currenturl in sys.argv[1:]:
    pid = os.fork()
    if pid == 0:
    signal.alarm(10)
    do_something() # placeholder for the download and print
    routine
    break
    time.sleep(4)

    That didn't help me, too.

    The following is a PHP application that may help you to understand the bad
    behavoir of my application (you will guess my problem if you execute both
    the PHP and the Python script):

    <?php
    echo shell_exec('./script.py http://www.website1.com
    http://www.website2.com');
    ?>


    Thank you.


    Best regards

    Markus Franz
     
    Markus Franz, Apr 22, 2004
    #1
    1. Advertising

  2. Markus Franz

    Jeff Epler Guest

    Keep track of the processes you create, then wait for them each to exit.
    Untested code:

    def do(task):
    pass

    processes = []
    try:
    for task in tasks:
    pid = os.fork()
    if pid == 0:
    do(task)
    os._exit(0)
    else:
    processes.append(pid)
    finally:
    while processes:
    pid = waitpid(0,0) # Wait for any child, block
    # print "reaped", pid
    processes.remove(pid)

    Instead of looping until 'processes' is empty, you could wait until
    waitpid raises os.error with errno==ECHILD ("No child processes").

    Jeff
     
    Jeff Epler, Apr 22, 2004
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Jeff Rodriguez
    Replies:
    23
    Views:
    1,130
    David Schwartz
    Dec 9, 2003
  2. Replies:
    22
    Views:
    809
    Mark McIntyre
    Mar 30, 2005
  3. empriser
    Replies:
    1
    Views:
    641
    Ian Collins
    Mar 6, 2007
  4. Uwe Kubosch
    Replies:
    5
    Views:
    223
    Gary Wright
    Feb 2, 2008
  5. Z W
    Replies:
    0
    Views:
    174
Loading...

Share This Page