Re: Subprocess does not return for longer-running process

Discussion in 'Python' started by Dennis Lee Bieber, Sep 22, 2010.

  1. On Tue, 21 Sep 2010 17:33:25 -0700, Chris Rebert <>
    declaimed the following in gmane.comp.python.general:

    > As your Traceback clearly indicates, the Popen() call has already
    > completed; it's *the os.waitpid() call* that's blocking, but that's
    > entirely to be expected given its defined behavior. If you don't want
    > to wait around for the process to die, then don't call waitpid() on it
    > in the first place!
    >

    And a likely reason the waitpid() if blocking is that the
    sub-process is blocking trying to write to pipe that is full.

    --
    Wulfraed Dennis Lee Bieber AF6VN
    HTTP://wlfraed.home.netcom.com/
     
    Dennis Lee Bieber, Sep 22, 2010
    #1
    1. Advertising

  2. Dennis Lee Bieber

    Nobody Guest

    On Tue, 21 Sep 2010 23:54:04 -0700, Dennis Lee Bieber wrote:

    >> As your Traceback clearly indicates, the Popen() call has already
    >> completed; it's *the os.waitpid() call* that's blocking, but that's
    >> entirely to be expected given its defined behavior. If you don't want
    >> to wait around for the process to die, then don't call waitpid() on it
    >> in the first place!
    >>

    > And a likely reason the waitpid() if blocking is that the
    > sub-process is blocking trying to write to pipe that is full.


    Which is why you should always use the .communicate() method if you
    redirect two or more of std{in,out,err} to a pipe (this is why the Unix
    popen() function only lets you redirect one of the standard descriptors).

    And I can't think of any reason why you should use os.waitpid() or
    similar; use the .wait() method.
     
    Nobody, Sep 22, 2010
    #2
    1. Advertising

  3. In message <>, Nobody wrote:

    > And I can't think of any reason why you should use os.waitpid() or
    > similar; use the .wait() method.


    I have used WNOHANG to poll for completion of a subprocess while providing
    progress updates to the user.
     
    Lawrence D'Oliveiro, Sep 23, 2010
    #3
  4. Dennis Lee Bieber

    Nobody Guest

    On Thu, 23 Sep 2010 12:25:53 +1200, Lawrence D'Oliveiro wrote:

    >> And I can't think of any reason why you should use os.waitpid() or
    >> similar; use the .wait() method.

    >
    > I have used WNOHANG to poll for completion of a subprocess while providing
    > progress updates to the user.


    This can be done via the .poll() method.
     
    Nobody, Sep 23, 2010
    #4
  5. In message <>, Nobody wrote:

    > On Thu, 23 Sep 2010 12:25:53 +1200, Lawrence D'Oliveiro wrote:
    >
    >>> And I can't think of any reason why you should use os.waitpid() or
    >>> similar; use the .wait() method.

    >>
    >> I have used WNOHANG to poll for completion of a subprocess while
    >> providing progress updates to the user.

    >
    > This can be done via the .poll() method.


    And what do you think the poll method uses?
     
    Lawrence D'Oliveiro, Sep 24, 2010
    #5
  6. Dennis Lee Bieber

    Nobody Guest

    On Fri, 24 Sep 2010 15:18:47 +1200, Lawrence D'Oliveiro wrote:

    >>>> And I can't think of any reason why you should use os.waitpid() or
    >>>> similar; use the .wait() method.
    >>>
    >>> I have used WNOHANG to poll for completion of a subprocess while
    >>> providing progress updates to the user.

    >>
    >> This can be done via the .poll() method.

    >
    > And what do you think the poll method uses?


    1. Not relevant; use the defined interface.

    2. It currently uses waitpid() on Unix, WaitForSingleObject() on Windows,
    maybe other functions if subprocess gets ported to other platforms or
    extended. If you use the .poll() method, you don't need to worry about any
    of this.

    More importantly the .poll() method sets the .returncode attribute. Why is
    this important? Because if the .returncode attribute is None,
    Popen.__del__ will add the object to the _active list. Not only does this
    prevent the object from being finalised, it will continue to poll for
    termination via _cleanup().

    But if you have manually reaped the child, its PID becomes available for
    re-use. This can result in a subsequent _cleanup() reaping some other
    child process. In the 2.6 implementation, manually setting the .returncode
    attribute will solve this. In future implementations ... who knows?

    Again: use the .wait() or .poll() methods. These are portable, can be
    assumed to do the right thing for subclasses of subprocess.Popen(), and
    will perform any necessary maintenance of "internal" state.

    If those methods don't suffice, you can't reliably use subprocess. E.g. if
    you need to wait until any child terminates, using os.wait() will cause
    the child to be reaped, which will confuse Popen().

    POSIX (and Linux since 2.6.9) has waitid(), which allows you to wait for a
    child *without* reaping it (via the WNOWAIT flag), but Python doesn't
    include an interface for this.
     
    Nobody, Sep 24, 2010
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. ~~~~ .NET Ed ~~~~
    Replies:
    6
    Views:
    1,195
    George Hester
    Dec 24, 2004
  2. Greenhorn
    Replies:
    15
    Views:
    852
    Keith Thompson
    Mar 6, 2005
  3. hiral
    Replies:
    2
    Views:
    614
    Jean-Michel Pichavant
    May 5, 2010
  4. Jason Friedman
    Replies:
    0
    Views:
    234
    Jason Friedman
    Sep 21, 2010
  5. the_one
    Replies:
    1
    Views:
    397
    the_one
    Mar 5, 2012
Loading...

Share This Page