Re: How can I know when a sub-process is effectively initialized?

Discussion in 'Python' started by Jean-Paul Calderone, Feb 26, 2009.

  1. On Thu, 26 Feb 2009 12:27:26 -0800 (PST), Giampaolo Rodola' <> wrote:
    >Hi,
    >I'm working on a Python module called psutil [1] for reading process
    >information in a cross-platform way.
    >I'm having a problem with psutil test suite.
    >Almost all the test cases we implemented so far look like this:
    >
    >def test_foo(self):
    > test_process = subprocess.Popen(sys.executable, stdout=DEVNULL,
    >stderr=DEVNULL)
    > time.sleep(0.1) # XXX: provisional, give some time the sub
    >process to initialize
    > p = psutil.Process(test_process.pid)
    > # start test here
    > ...
    >
    >As you can see we put a time.sleep(0.1) call after subprocess.Popen()
    >as a provisional workaround to let the sub process initialize
    >properly.
    >We're searching for some kind of way to know when the child process is
    >properly initialized so that we can actually start testing cases
    >without worries.
    >Does someone has an idea how could we do that?


    Hi Giampaolo,

    "Properly initialized" is application specific. Generally, what you want
    to do is have the child process tell you when it's ready. A common way
    to do this is for the child to write some bytes to a file descriptor the
    parent is monitoring when it is ready. The parent then just waits for
    those bytes. It looks like you entirely control the child in this case,
    so that should be possible here.

    Alternatively, maybe by "properly initialized" you mean something very
    specific about how subprocess.Popen creates processes. If so, can you
    elaborate a bit?

    Jean-Paul
     
    Jean-Paul Calderone, Feb 26, 2009
    #1
    1. Advertising

  2. On 26 Feb, 21:59, Jean-Paul Calderone <> wrote:
    > On Thu, 26 Feb 2009 12:27:26 -0800 (PST), Giampaolo Rodola' <gne...@gmail..com> wrote:
    > >Hi,
    > >I'm working on a Python module called psutil [1] for reading process
    > >information in a cross-platform way.
    > >I'm having a problem with psutil test suite.
    > >Almost all the test cases we implemented so far look like this:

    >
    > >def test_foo(self):
    > >    test_process = subprocess.Popen(sys.executable, stdout=DEVNULL,
    > >stderr=DEVNULL)
    > >    time.sleep(0.1)  # XXX: provisional, give some time the sub
    > >process to initialize
    > >    p = psutil.Process(test_process.pid)
    > >    # start test here
    > >    ...

    >
    > >As you can see we put a time.sleep(0.1) call after subprocess.Popen()
    > >as a provisional workaround to let the sub process initialize
    > >properly.
    > >We're searching for some kind of way to know when the child process is
    > >properly initialized so that we can actually start testing cases
    > >without worries.
    > >Does someone has an idea how could we do that?

    >
    > Hi Giampaolo,
    >
    > "Properly initialized" is application specific.  Generally, what you want
    > to do is have the child process tell you when it's ready.  A common way
    > to do this is for the child to write some bytes to a file descriptor the
    > parent is monitoring when it is ready.  The parent then just waits for
    > those bytes.  It looks like you entirely control the child in this case,
    > so that should be possible here.
    >
    > Alternatively, maybe by "properly initialized" you mean something very
    > specific about how subprocess.Popen creates processes.  If so, can you
    > elaborate a bit?
    >
    > Jean-Paul


    By "properly initialized" I mean when I can actually start fetching
    information from the process (e.g. name, path, parent pid, etc...).
    Anyway, I came out with this solution:


    def wait_for_pid(pid, timeout=1):
    """Wait for pid to show up in the process list then return.
    Used in the test suite to give time the sub process to initialize.
    """
    raise_at = time.time() + timeout
    while 1:
    if pid in psutil.get_pid_list():
    return
    time.sleep(0.0001)
    if time.time() >= raise_at:
    raise RuntimeError("Timed out")


    def test_foo():
    proc = subprocess.Popen(PYTHON, stdout=DEVNULL, stderr=DEVNULL)
    wait_for_pid(proc.pid)
    ...


    It seems to work fine on posix but we still got problems on Windows
    where it seems that waiting for pid to show up in the system process
    list is not enough for considering the process effectively
    initialized.
    Anyway, I think it's a problem in our code.
    Thanks for your help, anyway.


    --- Giampaolo
    http://code.google.com/p/pyftpdlib
     
    Giampaolo Rodola', Feb 27, 2009
    #2
    1. Advertising

  3. Jean-Paul Calderone

    Tim Golden Guest

    Giampaolo Rodola' wrote:
    > It seems to work fine on posix but we still got problems on Windows
    > where it seems that waiting for pid to show up in the system process
    > list is not enough for considering the process effectively
    > initialized.
    > Anyway, I think it's a problem in our code.
    > Thanks for your help, anyway.



    On Windows, I'd just wait on a Windows event. The child process
    sets it; the parent waits on it. If you need various of them,
    the parent can pass a suitable id on the commandline to the
    child. There are alternatives but this one is very easy to achieve.

    TJG
     
    Tim Golden, Feb 27, 2009
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. loga123
    Replies:
    2
    Views:
    330
    loga123
    May 20, 2006
  2. Ben
    Replies:
    2
    Views:
    893
  3. Giampaolo Rodola'
    Replies:
    0
    Views:
    233
    Giampaolo Rodola'
    Feb 26, 2009
  4. Lawrence D'Oliveiro

    Death To Sub-Sub-Sub-Directories!

    Lawrence D'Oliveiro, May 5, 2011, in forum: Java
    Replies:
    92
    Views:
    2,036
    Lawrence D'Oliveiro
    May 20, 2011
  5. Andries

    I know, I know, I don't know

    Andries, Apr 23, 2004, in forum: Perl Misc
    Replies:
    3
    Views:
    236
    Gregory Toomey
    Apr 23, 2004
Loading...

Share This Page