subprocess module and long-lived subprocesses

Discussion in 'Python' started by skip@pobox.com, Jan 20, 2012.

  1. Guest

    I'm converting some os.popen calls to use subprocess.Popen. I had
    previously been ignoring stdout and stderr when using os.popen. The primary
    motivation to switch to subprocess.Popen now is that I now want to check
    stderr, so would have to make code changes to use os.popen[34] anyway.
    Might as well go whole hog and switch to the new API.

    The library documentation doesn't talk a lot about long-lived subprocesses
    other than the possibility of deadlock when using Popen.wait(). Ideally, I
    would write to the subprocess's stdin, check for output on stdout and
    stderr, then lather, rinse, repeat. Is it safe to assume that if the stdout
    and/or stderr pipes have nothing for me the reads on those file objects (I'm
    using PIPE for all three std* files) will return immediately with an empty
    string for output? They won't block, will they? Will a broken pipe IOError
    get raised as for os.popen() or do I have to call Popen.poll() even in error
    situations?

    Thanks,

    --
    Skip Montanaro - - http://www.smontanaro.net/
     
    , Jan 20, 2012
    #1
    1. Advertising

  2. Nobody Guest

    On Fri, 20 Jan 2012 08:42:16 -0600, skip wrote:

    > The library documentation doesn't talk a lot about long-lived subprocesses
    > other than the possibility of deadlock when using Popen.wait(). Ideally, I
    > would write to the subprocess's stdin, check for output on stdout and
    > stderr, then lather, rinse, repeat. Is it safe to assume that if the stdout
    > and/or stderr pipes have nothing for me the reads on those file objects (I'm
    > using PIPE for all three std* files) will return immediately with an empty
    > string for output? They won't block, will they?


    They will. You need to use either threads, select() or non-blocking I/O in
    order to avoid deadlock. See the definitions of subprocess._communicate()
    (there's one version for Windows which uses threads and another for Unix
    using select()).

    > Will a broken pipe IOError get raised as for os.popen()


    IOError(EPIPE) will be raised if you write to the stdin pipe when there
    are no readers.

    > or do I have to call Popen.poll() even in error situations?


    Once you're finished with the process, you should close .stdin then
    consume all output from .stdout and .stderr until both report EOF, then
    call .wait(). That should cover any possible child behaviour (e.g. if
    the child explicitly close()s its stdin, getting EPIPE doesn't mean that
    you can forget about the process or that .wait() won't deadlock).
     
    Nobody, Jan 21, 2012
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    0
    Views:
    600
  2. Mathieu Dutour

    long long and long

    Mathieu Dutour, Jul 17, 2007, in forum: C Programming
    Replies:
    4
    Views:
    517
    santosh
    Jul 24, 2007
  3. Andreas Leitgeb

    Short-lived Objects - good or bad?

    Andreas Leitgeb, Apr 10, 2008, in forum: Java
    Replies:
    17
    Views:
    660
  4. Luke
    Replies:
    2
    Views:
    206
  5. mdos
    Replies:
    1
    Views:
    111
    Jeff Bender
    Jan 29, 2007
Loading...

Share This Page