subprocess leaves child living

Discussion in 'Python' started by Thomas Dybdahl Ahle, Jun 5, 2007.

  1. Hi, When I do a small program like

    from subprocess import Popen
    popen = Popen(["ping", "google.com"])
    from time import sleep
    sleep(100)

    start it and kill it, the ping process lives on.
    Is there a way to ensure that the ping process is always killed when the
    python process is?
    I can't use atexit, as ping then isn't killed when python is killed "in
    the hard way"
    Thomas Dybdahl Ahle, Jun 5, 2007
    #1
    1. Advertising

  2. Thomas Dybdahl Ahle schrieb:
    > Hi, When I do a small program like
    >
    > from subprocess import Popen
    > popen = Popen(["ping", "google.com"])
    > from time import sleep
    > sleep(100)
    >
    > start it and kill it, the ping process lives on.
    > Is there a way to ensure that the ping process is always killed when the
    > python process is?
    > I can't use atexit, as ping then isn't killed when python is killed "in
    > the hard way"
    >

    Calling popen.close() perhaps ?
    You basically open a pipe, which spawns a shell and the command is then
    started in there.
    So, if your program quits, the spawned shell is still alive, only the
    pipe is dead.
    Stefan Sonnenberg-Carstens, Jun 5, 2007
    #2
    1. Advertising

  3. Den Tue, 05 Jun 2007 14:07:44 +0200 skrev Stefan Sonnenberg-Carstens:

    > Thomas Dybdahl Ahle schrieb:
    >> Hi, When I do a small program like
    >>
    >> from subprocess import Popen
    >> popen = Popen(["ping", "google.com"]) from time import sleep
    >> sleep(100)
    >>
    >> start it and kill it, the ping process lives on. Is there a way to
    >> ensure that the ping process is always killed when the python process
    >> is?
    >> I can't use atexit, as ping then isn't killed when python is killed "in
    >> the hard way"
    >>

    > Calling popen.close() perhaps ?
    > You basically open a pipe, which spawns a shell and the command is then
    > started in there.
    > So, if your program quits, the spawned shell is still alive, only the
    > pipe is dead.


    Problem is - I can't do that when I get killed.
    Isn't it possible to open processes in such a way like terminals? If I
    kill the terminal, everything open in it will die too.
    Thomas Dybdahl Ahle, Jun 5, 2007
    #3
  4. Thomas Dybdahl Ahle

    Rob Wolfe Guest

    Thomas Dybdahl Ahle wrote:

    > Problem is - I can't do that when I get killed.
    > Isn't it possible to open processes in such a way like terminals? If I
    > kill the terminal, everything open in it will die too.


    On POSIX platform you can use signals and ``os.kill`` function.
    Fo example:

    <code>
    import os, signal
    from subprocess import Popen
    from time import sleep

    def handler(signum, frame):
    print 'Signal handler called'
    raise KeyboardInterrupt

    signal.signal(signal.SIGTERM, handler)

    try:
    popen = Popen(["ping", "google.com"])
    try:
    sleep(100)
    except KeyboardInterrupt:
    pass
    finally:
    if popen.poll() is None:
    print "killing process: %d" % popen.pid
    os.kill(popen.pid, signal.SIGTERM)
    </code>

    --
    HTH,
    Rob
    Rob Wolfe, Jun 5, 2007
    #4
  5. Den Tue, 05 Jun 2007 07:06:15 -0700 skrev Rob Wolfe:

    > Thomas Dybdahl Ahle wrote:
    >
    >> Problem is - I can't do that when I get killed. Isn't it possible to
    >> open processes in such a way like terminals? If I kill the terminal,
    >> everything open in it will die too.

    >
    > On POSIX platform you can use signals and ``os.kill`` function. Fo
    > example:
    >
    > <code>
    > import os, signal
    > from subprocess import Popen
    > from time import sleep
    >
    > def handler(signum, frame):
    > print 'Signal handler called'
    > raise KeyboardInterrupt
    >
    > signal.signal(signal.SIGTERM, handler)
    >
    > try:
    > popen = Popen(["ping", "google.com"]) try:
    > sleep(100)
    > except KeyboardInterrupt:
    > pass
    > finally:
    > if popen.poll() is None:
    > print "killing process: %d" % popen.pid os.kill(popen.pid,
    > signal.SIGTERM)
    > </code>


    But you can't ever catch sigkill.
    Isn't there a way to make sure the os kills the childprocess when the
    parrent dies?
    Thomas Dybdahl Ahle, Jun 5, 2007
    #5
  6. Thomas Dybdahl Ahle

    Rob Wolfe Guest

    Thomas Dybdahl Ahle <> writes:

    > But you can't ever catch sigkill.


    There is no protection against sigkill.

    > Isn't there a way to make sure the os kills the childprocess when the
    > parrent dies?


    If the parent dies suddenly without any notification childprocesses
    become zombies and there is no way to avoid that.

    --
    HTH,
    Rob
    Rob Wolfe, Jun 5, 2007
    #6
  7. On Jun 5, 2007, at 3:01 PM, Rob Wolfe wrote:

    > Thomas Dybdahl Ahle <> writes:
    >
    >> But you can't ever catch sigkill.

    >
    > There is no protection against sigkill.
    >
    >> Isn't there a way to make sure the os kills the childprocess when the
    >> parrent dies?

    >
    > If the parent dies suddenly without any notification childprocesses
    > become zombies and there is no way to avoid that.


    Apologies for picking nits...

    But actually *that* is an orphan process. When a parent process dies
    and the child continues to run, the child becomes an orphan and is
    adopted by init. Orphan processes can be cleaned up on most Unices
    with 'init q' (or something very similar).

    Zombies on the other hand, are those processes that have completed
    execution but still have an entry in the process table. The cause of
    zombies AFAIK, is a parent that has failed to call wait(2). To clean
    up zombies, you can send a SIGCHLD signal to the parent process --
    probably with 'kill -17' (but use 'kill -l' to find out what it is on
    your system).

    hth,
    Michael
    ---
    "I would rather use Java than Perl. And I'd rather be eaten by a
    crocodile than use Java." — Trouser
    Michael Bentley, Jun 5, 2007
    #7
  8. Den Tue, 05 Jun 2007 22:01:44 +0200 skrev Rob Wolfe:

    > Thomas Dybdahl Ahle <> writes:
    >
    >> But you can't ever catch sigkill.

    >
    > There is no protection against sigkill.
    >
    >> Isn't there a way to make sure the os kills the childprocess when the
    >> parrent dies?

    >
    > If the parent dies suddenly without any notification childprocesses
    > become zombies and there is no way to avoid that.


    If I call "kill -9 pythonpid" on the python code you posted earlier, the
    terminal running the script will continue pinging forever.
    If it was a harder program like a chessengine or such, it would continue
    consuming 100% cpu time.
    Zombies would be fine to me.
    Thomas Dybdahl Ahle, Jun 5, 2007
    #8
  9. Den Tue, 05 Jun 2007 15:46:39 -0500 skrev Michael Bentley:

    > But actually *that* is an orphan process. When a parent process dies
    > and the child continues to run, the child becomes an orphan and is
    > adopted by init. Orphan processes can be cleaned up on most Unices with
    > 'init q' (or something very similar).


    Is it not possible to tell python that this process should not be adopted
    by init, but die with its parrent?
    Just like terminals seem to do it..
    Thomas Dybdahl Ahle, Jun 5, 2007
    #9
  10. On Jun 5, 2007, at 4:17 PM, Thomas Dybdahl Ahle wrote:

    > Den Tue, 05 Jun 2007 15:46:39 -0500 skrev Michael Bentley:
    >
    >> But actually *that* is an orphan process. When a parent process dies
    >> and the child continues to run, the child becomes an orphan and is
    >> adopted by init. Orphan processes can be cleaned up on most
    >> Unices with
    >> 'init q' (or something very similar).

    >
    > Is it not possible to tell python that this process should not be
    > adopted
    > by init, but die with its parrent?
    > Just like terminals seem to do it..


    Well, the way you posed the original question:

    > from subprocess import Popen
    > popen = Popen(["ping", "google.com"])
    > from time import sleep
    > sleep(100)


    is really what adoption by init is designed to handle. Here you've
    created a child but have not waited for its return value. Like a
    good adoptive parent, init will wait(2) for the child.

    I think if you really looked into it you'd find that the terminal had
    called wait(2) before it was killed. Similarly, if you start a long-
    running subprocess in python and wait for it to return -- killing the
    parent will slaughter the child as well.

    hth,
    Michael
    ---
    Let the wookie win.
    Michael Bentley, Jun 5, 2007
    #10
  11. On Jun 5, 2007, at 5:13 PM, Michael Bentley wrote:

    >
    > On Jun 5, 2007, at 4:17 PM, Thomas Dybdahl Ahle wrote:
    >
    >> Den Tue, 05 Jun 2007 15:46:39 -0500 skrev Michael Bentley:
    >>
    >>> But actually *that* is an orphan process. When a parent process
    >>> dies
    >>> and the child continues to run, the child becomes an orphan and is
    >>> adopted by init. Orphan processes can be cleaned up on most
    >>> Unices with
    >>> 'init q' (or something very similar).

    >>
    >> Is it not possible to tell python that this process should not be
    >> adopted
    >> by init, but die with its parrent?
    >> Just like terminals seem to do it..

    >
    > Well, the way you posed the original question:
    >
    >> from subprocess import Popen
    >> popen = Popen(["ping", "google.com"])
    >> from time import sleep
    >> sleep(100)

    >
    > is really what adoption by init is designed to handle. Here you've
    > created a child but have not waited for its return value. Like a
    > good adoptive parent, init will wait(2) for the child.
    >
    > I think if you really looked into it you'd find that the terminal had
    > called wait(2) before it was killed. Similarly, if you start a long-
    > running subprocess in python and wait for it to return -- killing the
    > parent will slaughter the child as well.


    I guess I should have verified my suspicions before speaking up -- I
    was worng -- even if you are waiting for a return code, the child
    will persist as a child of init.

    regards,
    Michael
    ---
    The Rules of Optimization are simple.
    Rule 1: Don't do it.
    Rule 2 (for experts only): Don't do it yet.
    -Michael A. Jackson
    Michael Bentley, Jun 5, 2007
    #11
  12. Thomas Dybdahl Ahle <> wrote:
    > But you can't ever catch sigkill.
    > Isn't there a way to make sure the os kills the childprocess when the
    > parrent dies?


    Not as far as I know.

    If you've got a pipe open to the child then killing the parent should
    deliver SIGPIPE to the child which may (or may not) kill it. At least
    it got some sort of notification.

    --
    Nick Craig-Wood <> -- http://www.craig-wood.com/nick
    Nick Craig-Wood, Jun 6, 2007
    #12
  13. Den Tue, 05 Jun 2007 17:41:47 -0500 skrev Michael Bentley:

    > On Jun 5, 2007, at 5:13 PM, Michael Bentley wrote:
    >
    >
    >> On Jun 5, 2007, at 4:17 PM, Thomas Dybdahl Ahle wrote:
    >>
    >>> Den Tue, 05 Jun 2007 15:46:39 -0500 skrev Michael Bentley:
    >>>
    >>>> But actually *that* is an orphan process. When a parent process dies
    >>>> and the child continues to run, the child becomes an orphan and is
    >>>> adopted by init. Orphan processes can be cleaned up on most Unices
    >>>> with
    >>>> 'init q' (or something very similar).
    >>>
    >>> Is it not possible to tell python that this process should not be
    >>> adopted
    >>> by init, but die with its parrent?
    >>> Just like terminals seem to do it..

    >>
    >> Well, the way you posed the original question:
    >>
    >>> from subprocess import Popen
    >>> popen = Popen(["ping", "google.com"]) from time import sleep
    >>> sleep(100)

    >>
    >> is really what adoption by init is designed to handle. Here you've
    >> created a child but have not waited for its return value. Like a good
    >> adoptive parent, init will wait(2) for the child.
    >>
    >> I think if you really looked into it you'd find that the terminal had
    >> called wait(2) before it was killed. Similarly, if you start a long-
    >> running subprocess in python and wait for it to return -- killing the
    >> parent will slaughter the child as well.

    >
    > I guess I should have verified my suspicions before speaking up -- I was
    > worng -- even if you are waiting for a return code, the child will
    > persist as a child of init.


    Bugger.
    I know that if you use forkpty, the child get slaughtered, but using the
    subprocess module just seems much easier than forking.
    Thomas Dybdahl Ahle, Jun 6, 2007
    #13
  14. On Jun 6, 2007, at 7:11 AM, Thomas Dybdahl Ahle wrote:

    > Den Tue, 05 Jun 2007 17:41:47 -0500 skrev Michael Bentley:
    >
    >> On Jun 5, 2007, at 5:13 PM, Michael Bentley wrote:
    >>
    >>
    >>> On Jun 5, 2007, at 4:17 PM, Thomas Dybdahl Ahle wrote:
    >>>
    >>>> Den Tue, 05 Jun 2007 15:46:39 -0500 skrev Michael Bentley:
    >>>>
    >>>>> But actually *that* is an orphan process. When a parent
    >>>>> process dies
    >>>>> and the child continues to run, the child becomes an orphan and is
    >>>>> adopted by init. Orphan processes can be cleaned up on most
    >>>>> Unices
    >>>>> with
    >>>>> 'init q' (or something very similar).
    >>>>
    >>>> Is it not possible to tell python that this process should not be
    >>>> adopted
    >>>> by init, but die with its parrent?
    >>>> Just like terminals seem to do it..
    >>>
    >>> Well, the way you posed the original question:
    >>>
    >>>> from subprocess import Popen
    >>>> popen = Popen(["ping", "google.com"]) from time import sleep
    >>>> sleep(100)
    >>>
    >>> is really what adoption by init is designed to handle. Here you've
    >>> created a child but have not waited for its return value. Like a
    >>> good
    >>> adoptive parent, init will wait(2) for the child.
    >>>
    >>> I think if you really looked into it you'd find that the terminal
    >>> had
    >>> called wait(2) before it was killed. Similarly, if you start a
    >>> long-
    >>> running subprocess in python and wait for it to return -- killing
    >>> the
    >>> parent will slaughter the child as well.

    >>
    >> I guess I should have verified my suspicions before speaking up --
    >> I was
    >> worng -- even if you are waiting for a return code, the child will
    >> persist as a child of init.

    >
    > Bugger.
    > I know that if you use forkpty, the child get slaughtered, but
    > using the
    > subprocess module just seems much easier than forking.


    Yeah, and if there is some way (there may be, but I don't know) to
    make your python script the head of a process group (or session --
    I'm not very clear on the distinction) -- children would be
    automatically slain. That's how a terminal session manages to get
    its children cleaned up I think...

    If it is important that children get terminated upon death of a
    parent, you probably do have to fork.

    g'luck!
    Michael
    ---
    "Neurons are far more valuable than disk space, screen lines, or CPU
    cycles." - Ben Finney
    Michael Bentley, Jun 6, 2007
    #14
  15. Thomas Dybdahl Ahle

    reed Guest

    On Jun 5, 7:58 am, Thomas Dybdahl Ahle <> wrote:
    > Hi, When I do a small program like
    >
    > from subprocess import Popen
    > popen = Popen(["ping", "google.com"])
    > from time import sleep
    > sleep(100)
    >
    > start it and kill it, the ping process lives on.
    > Is there a way to ensure that the ping process is always killed when the
    > python process is?
    > I can't use atexit, as ping then isn't killed when python is killed "in
    > the hard way"




    pid = popen.pid
    pidfile = open('/usr/local/var/somefile.pid', 'w')
    pidfile.write('pid')
    pidfile.close()


    then you can check if it is still running when your ?program? restarts
    and can kill it.

    maybe not the perfect answer, but it answers an imperfect question.
    reed, Jun 7, 2007
    #15
  16. Den Thu, 07 Jun 2007 07:00:53 +0000 skrev reed:

    > On Jun 5, 7:58 am, Thomas Dybdahl Ahle <> wrote:
    >> Hi, When I do a small program like
    >>
    >> from subprocess import Popen
    >> popen = Popen(["ping", "google.com"]) from time import sleep
    >> sleep(100)
    >>
    >> start it and kill it, the ping process lives on. Is there a way to
    >> ensure that the ping process is always killed when the python process
    >> is?
    >> I can't use atexit, as ping then isn't killed when python is killed "in
    >> the hard way"

    >
    >
    >
    > pid = popen.pid
    > pidfile = open('/usr/local/var/somefile.pid', 'w') pidfile.write('pid')
    > pidfile.close()


    > then you can check if it is still running when your ?program? restarts
    > and can kill it.


    If it restarts yeah.

    > maybe not the perfect answer, but it answers an imperfect question.


    Any details you need?
    Thomas Dybdahl Ahle, Jun 7, 2007
    #16
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Nicol Nghia
    Replies:
    0
    Views:
    340
    Nicol Nghia
    Dec 3, 2005
  2. Tim Tyler

    Re: Living in a RAM-rich World

    Tim Tyler, Jun 27, 2003, in forum: Java
    Replies:
    0
    Views:
    1,291
    Tim Tyler
    Jun 27, 2003
  3. Tim Tyler

    Re: Living in a RAM-rich World

    Tim Tyler, Jun 27, 2003, in forum: Java
    Replies:
    3
    Views:
    1,138
    Jon Skeet
    Jun 27, 2003
  4. Jeff Rodriguez
    Replies:
    23
    Views:
    1,070
    David Schwartz
    Dec 9, 2003
  5. hiral
    Replies:
    2
    Views:
    574
    Jean-Michel Pichavant
    May 5, 2010
Loading...

Share This Page