Is the subprocess module thread-safe?

Discussion in 'Python' started by John Nagle, Oct 27, 2007.

  1. John Nagle

    John Nagle Guest

    Can the subprocess module be used to launch subprocesses from threads?
    Or does the subprocess module affect the entire process context, like "fork"?
    Or is this OS dependent?

    Also, if you launch processes from the subprocess module module and
    don't wait them to complete, do zombies accumulate under UNIX/Linux?

    John Nagle
     
    John Nagle, Oct 27, 2007
    #1
    1. Advertising

  2. > Can the subprocess module be used to launch subprocesses from threads?

    The answer to this question ("can") is a clear yes. Notice that this
    question is different from the question in the subject.

    > Or does the subprocess module affect the entire process context, like
    > "fork"?


    On POSIX systems, it uses fork(2) - what else could it do? I'm not sure
    what effects to the entire process context you are referring to. In
    a POSIX system, you can also fork from a thread just fine.

    > Or is this OS dependent?


    The precise semantics are OS dependent, yes. However, you can use it
    in the presence of threads on most systems that support threads (for
    a reasonable definition of "supports threads", which may exclude
    HP-UX and FreeBSD :)

    It's technically not thread-safe if other threads manipulate file
    descriptors; there is always a chance of race conditions when some
    thread opens a file handle and then tries to set the CLOEXEC flag
    on it - if some other thread creates a subprocess in-between, there
    is a chance that the child process "sees" the file handle even though
    it "should" have the CLOEXEC flag set (but didn't at the point when
    the process forked).

    > Also, if you launch processes from the subprocess module module and
    > don't wait them to complete, do zombies accumulate under UNIX/Linux?


    No. It depends on the precise code, but if you drop the reference to the
    Popen object, or call one of the functions that don't return one in the
    first place, the module itself will wait for the child processes
    "regularly".

    Regards,
    Martin
     
    =?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=, Oct 28, 2007
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Jane Austine

    Is re module thread-safe?

    Jane Austine, Jul 11, 2003, in forum: Python
    Replies:
    1
    Views:
    297
    Skip Montanaro
    Jul 11, 2003
  2. nicolas_riesch

    how to write thread-safe module ? and pytz

    nicolas_riesch, Aug 10, 2005, in forum: Python
    Replies:
    5
    Views:
    430
    nicolas_riesch
    Aug 15, 2005
  3. Gabriel Rossetti
    Replies:
    0
    Views:
    1,400
    Gabriel Rossetti
    Aug 29, 2008
  4. Danny Wong (dannwong)

    subprocess.Popen and thread module

    Danny Wong (dannwong), Aug 10, 2011, in forum: Python
    Replies:
    1
    Views:
    237
    Thomas Rachel
    Aug 10, 2011
  5. John Nagle
    Replies:
    5
    Views:
    514
    John Nagle
    Mar 12, 2012
Loading...

Share This Page