Run process with timeout

[Natan]

Hi.

I have a python script under linux where I poll many hundreds of
interfaces with mrtg every 5 minutes. Today I create some threads and
use os.system(command) to run the process, but some of them just hang.
I would like to terminate the process after 15 seconds if it doesn't
finish, but os.system() doesn't have any timeout parameter.

Can anyone help me on what can I use to do this?

Thank you

 

[Alex Martelli]

Hi.

I have a python script under linux where I poll many hundreds of
interfaces with mrtg every 5 minutes. Today I create some threads and
use os.system(command) to run the process, but some of them just hang.
I would like to terminate the process after 15 seconds if it doesn't
finish, but os.system() doesn't have any timeout parameter.

Can anyone help me on what can I use to do this?

Use the subprocess module. With a subprocess.Popen object, you can then
sleep a while, check (with .poll()) if it's finished, otherwise kill it
(use its .pid attribute).


Alex

 

[P]

Hi.

I have a python script under linux where I poll many hundreds of
interfaces with mrtg every 5 minutes. Today I create some threads and
use os.system(command) to run the process, but some of them just hang.
I would like to terminate the process after 15 seconds if it doesn't
finish, but os.system() doesn't have any timeout parameter.

Can anyone help me on what can I use to do this?

The new subprocess module has that functionality.
If your python version doesn't have that you
could try my unix specific version:
http://www.pixelbeat.org/libs/subProcess.py

Pádraig.

 

[Micah Elliott]

Use the subprocess module. With a subprocess.Popen object, you can
... kill it (use its .pid attribute).

The problem I've run into with this approach is the inability to kill
the pid's children. Most often I'm not so fortunate to be able to
depend on the process to not be doing its own forking. So here's a
simplified use case:

$ cat sleep10.sh
#! /bin/bash
sleep 10 # does not get killed
$
$ cat to3.py
#! /usr/bin/env python

from subprocess import Popen
from time import sleep
from os import kill
from signal import SIGTERM

p = Popen(['./sleep10.sh'])
sleep(3)
kill(p.pid, SIGTERM) # Oops, won't kill p.pid's children.
##kill(-p.pid, SIGTERM) # Won't work since not group leader,
# and I'd rather avoid fork/dup/exec.
$
$ ./to3.py
$ # to3.py finished but sleep 10 still running

If you try this you will see that sleep10.sh gets killed, but its
"sleep 10" subprocess does not, and runs for an additional 7 seconds.

I presently rely on an ugly script to do this properly. It uses low
level calls such as pipe, close, dup2, fork, exec, setpgrp, etc. I
won't post that here for brevity's sake (unless requested). For this
case it would fork/exec sleep10.sh, make it a group leader, and the
parent would kill its group.

Is there any way to enable Python's subprocess module to do (implicit?)
group setup to ease killing of all children? If not, is it a reasonable
RFE?

 

[Alex Martelli]

Is there any way to enable Python's subprocess module to do (implicit?)
group setup to ease killing of all children? If not, is it a reasonable
RFE?

Not as far as I know. It might be a reasonable request in suitable
dialects of Unix-like OSes, though. A setpgrp call (in the callback
which you can request Popen to perform, after it forks and before it
execs) might suffice... except that you can't rely on children process
not to setpgrp's themselves, can you?!


Alex

 

[Donn Cave]

Micah Elliott <[email protected]> wrote:

[... re problem killing children of shell script ...]

Not as far as I know. It might be a reasonable request in suitable
dialects of Unix-like OSes, though. A setpgrp call (in the callback
which you can request Popen to perform, after it forks and before it
execs) might suffice... except that you can't rely on children process
not to setpgrp's themselves, can you?!

I bet that wouldn't be a problem, though. subprocess.Popen
constructor takes a preexec_fn parameter that looks like it
might be a suitable place to try this. (Interesting that
it's a function parameter, not a method to be overridden by
a subclass.)

Donn Cave, (e-mail address removed)