Different behavior with multiprocessing

Discussion in 'Python' started by Tomas Kotal, Mar 4, 2013.

  1. Tomas Kotal

    Tomas Kotal Guest

    Hi all.

    Recently I discovered a strange behavior with multiprocessing library and call to function os.system (a different behavior under Linux and Windows to be more specific). I have this simple testing script:

    ############################################
    import sys
    import os
    from multiprocessing import Process

    def do_fork(cmd):
    ret = os.system(cmd)
    # print result of call
    print ret
    os._exit(ret)

    if __name__ == "__main__":
    cmds = [ "dir", "xy" ]

    procs = []
    for cmd in cmds:
    proc = Process(target=do_fork, args=(cmd, ))
    proc.start()
    procs.append( proc )

    for proc in procs:
    proc.join()
    print "exitcode: %d" % proc.exitcode

    print "ok"
    ############################################

    This script just starts 2 processes. Each process executes one command in system shell and exits setting it's exit code same as exit code of a system call. There are 2 commands: "dir" (which works fine on Windows and Linux) and "xy" (which is supposed to fail under both systems). The strange thing is that when I run this script under Windows I get this output:

    0
    1
    exitcode: 0
    exitcode: 1
    ok

    The first 0 and 1 are results of os.system call which are printed from child processes. Rest of lines are printed from main process. This is expected output.

    But when I run same script on Linux, what I get is this:

    0
    32512
    exitcode: 0
    exitcode: 0
    ok

    Although the second command fails and returns exit code 32512, the exit code from process in parent process gives me 0. I tried to change the script to use fork() instead of Process but the result was same.

    Can anybody explain me what's the problem here?

    (I use Python 2.6 on both Windows and Linux machines)
    Tomas Kotal, Mar 4, 2013
    #1
    1. Advertising

  2. Chris Angelico, Mar 4, 2013
    #2
    1. Advertising

  3. Tomas Kotal

    Tomas Kotal Guest

    > Under Unix, the return value from os.system() encodes more than one
    >
    > piece of information:
    >
    >
    >
    > http://docs.python.org/2/library/os.html#os.system
    >
    > http://docs.python.org/2/library/os.html#os.wait
    >
    >
    >
    > 32512 is 127*256, meaning that the shell exited with return code 127
    >
    > when given an unrecognized command.
    >
    >
    >
    > ChrisA


    Well the point is not the difference between return values of os.system under Windows and Linux.

    The point is that I set the exist code of child process with os._exit() which works fine on Windows but on Linux the process.exitcode returns always 0.

    What is very strange is, that if I set return code to some other value, like:

    ....
    os._exit( len(cmd) )
    ....

    Then I got exit codes 3 and 2 under Linux, which is correct. But when I do something like this:

    ....
    ret = os.system(cmd)
    print ret # this print real value, like 32512
    os._exit(ret) # exitcode of this child process will be 0 on Linux
    ....

    It doesn't work, which seems very strange to me.
    Tomas Kotal, Mar 4, 2013
    #3
  4. Tomas Kotal

    Tomas Kotal Guest

    > Under Unix, the return value from os.system() encodes more than one
    >
    > piece of information:
    >
    >
    >
    > http://docs.python.org/2/library/os.html#os.system
    >
    > http://docs.python.org/2/library/os.html#os.wait
    >
    >
    >
    > 32512 is 127*256, meaning that the shell exited with return code 127
    >
    > when given an unrecognized command.
    >
    >
    >
    > ChrisA


    Well the point is not the difference between return values of os.system under Windows and Linux.

    The point is that I set the exist code of child process with os._exit() which works fine on Windows but on Linux the process.exitcode returns always 0.

    What is very strange is, that if I set return code to some other value, like:

    ....
    os._exit( len(cmd) )
    ....

    Then I got exit codes 3 and 2 under Linux, which is correct. But when I do something like this:

    ....
    ret = os.system(cmd)
    print ret # this print real value, like 32512
    os._exit(ret) # exitcode of this child process will be 0 on Linux
    ....

    It doesn't work, which seems very strange to me.
    Tomas Kotal, Mar 4, 2013
    #4
  5. Tomas Kotal

    Tomas Kotal Guest

    Dne pondělí, 4. března 2013 15:31:41 UTC+1 Tomas Kotal napsal(a):
    > Hi all.
    >
    >
    >
    > Recently I discovered a strange behavior with multiprocessing library andcall to function os.system (a different behavior under Linux and Windows to be more specific). I have this simple testing script:
    >
    >
    >
    > ############################################
    >
    > import sys
    >
    > import os
    >
    > from multiprocessing import Process
    >
    >
    >
    > def do_fork(cmd):
    >
    > ret = os.system(cmd)
    >
    > # print result of call
    >
    > print ret
    >
    > os._exit(ret)
    >
    >
    >
    > if __name__ == "__main__":
    >
    > cmds = [ "dir", "xy" ]
    >
    >
    >
    > procs = []
    >
    > for cmd in cmds:
    >
    > proc = Process(target=do_fork, args=(cmd, ))
    >
    > proc.start()
    >
    > procs.append( proc )
    >
    >
    >
    > for proc in procs:
    >
    > proc.join()
    >
    > print "exitcode: %d" % proc.exitcode
    >
    >
    >
    > print "ok"
    >
    > ############################################
    >
    >
    >
    > This script just starts 2 processes. Each process executes one command insystem shell and exits setting it's exit code same as exit code of a system call. There are 2 commands: "dir" (which works fine on Windows and Linux)and "xy" (which is supposed to fail under both systems). The strange thingis that when I run this script under Windows I get this output:
    >
    >
    >
    > 0
    >
    > 1
    >
    > exitcode: 0
    >
    > exitcode: 1
    >
    > ok
    >
    >
    >
    > The first 0 and 1 are results of os.system call which are printed from child processes. Rest of lines are printed from main process. This is expected output.
    >
    >
    >
    > But when I run same script on Linux, what I get is this:
    >
    >
    >
    > 0
    >
    > 32512
    >
    > exitcode: 0
    >
    > exitcode: 0
    >
    > ok
    >
    >
    >
    > Although the second command fails and returns exit code 32512, the exit code from process in parent process gives me 0. I tried to change the scriptto use fork() instead of Process but the result was same.
    >
    >
    >
    > Can anybody explain me what's the problem here?
    >
    >
    >
    > (I use Python 2.6 on both Windows and Linux machines)


    Seems like I found the problem: os._exit probably takes as parametr unsigned char, so it uses as error code whatever value it gets modulo 256:

    os._exit(1) # process.exitcode == 1
    os._exit(255) # process.exitcode == 255
    os._exit(256) # process.exitcode == 0
    os._exit(257) # process.exitcode == 1
    os._exit(32512) # process.exitcode == 0

    So on Linux it's necesary to call something like this:
    os._exit( os.system(cmd) >> 8 )

    Because the first byte of return value on Linux is number of signal which kills the process and the second one is actual exit code.

    http://docs.python.org/2/library/os.html#os.wait
    Tomas Kotal, Mar 4, 2013
    #5
  6. On Tue, Mar 5, 2013 at 2:35 AM, Tomas Kotal <> wrote:
    > Seems like I found the problem: os._exit probably takes as parametr unsigned char, so it uses as error code whatever value it gets modulo 256:
    >
    > os._exit(1) # process.exitcode == 1
    > os._exit(255) # process.exitcode == 255
    > os._exit(256) # process.exitcode == 0
    > os._exit(257) # process.exitcode == 1
    > os._exit(32512) # process.exitcode == 0
    >
    > So on Linux it's necesary to call something like this:
    > os._exit( os.system(cmd) >> 8 )
    >
    > Because the first byte of return value on Linux is number of signal which kills the process and the second one is actual exit code.


    Yep. I had a reply part-written but you beat me to it! That is indeed
    what you need if you want to chain return values.

    However, why are you using os._exit? Check out the note here:

    http://docs.python.org/2/library/os.html#os._exit

    ChrisA
    Chris Angelico, Mar 4, 2013
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Raghu Rudra

    Different behavior locally vs remotely

    Raghu Rudra, Jul 21, 2003, in forum: ASP .Net
    Replies:
    0
    Views:
    316
    Raghu Rudra
    Jul 21, 2003
  2. Johnny Ruin
    Replies:
    5
    Views:
    493
  3. Mantorok Redgormor
    Replies:
    70
    Views:
    1,734
    Dan Pop
    Feb 17, 2004
  4. Scott Pigman
    Replies:
    6
    Views:
    322
    Scott Pigman
    Oct 4, 2010
  5. John L. Stephens

    Multiprocessing.Process Daemonic Behavior

    John L. Stephens, Mar 16, 2011, in forum: Python
    Replies:
    0
    Views:
    272
    John L. Stephens
    Mar 16, 2011
Loading...

Share This Page