Using signal.alarm to terminate a thread

[Adrian Casey]

I have a multi-threaded python application which uses pexpect to connect to
multiple systems concurrently. Each thread within my application is a
connection to a remote system. The problem is when one of the child
threads runs a command which generates an unlimited amount of output. The
classic example of this is the "yes" command. If you
execute "pexpect.run('yes')", your cpu will sit at 100% forever.

Here is a simple multi-threaded program using pexpect which demonstrates the
problem. The command 'yes' is run in a thread. The parent says that when
the alarm goes off, run the handler function. The thread sets the alarm to
trigger after 5 seconds.

#!/usr/bin/env python
import signal, os, pexpect, threading

def handler(signum, frame):
# This should get called after 5 seconds when the alarm fires.
os.system('killall yes')
print "The yes command has been killed!"

def runyes():
# Run the 'yes' command in a thread. Set an alarm
# to fire in 5 seconds.
signal.alarm(5)
print "Running yes command..."
# If you run 'sleep 10' instead, this works.
pexpect.run('yes')
# Re-set the alarm. (This is never reached. The 'yes'
# command runs forever and is not interrupted by the alarm)
signal.alarm(0)

signal.signal(signal.SIGALRM, handler)
t = threading.Thread(target=runyes)
t.start()
t.join()
# END of code

Note that if the 'yes' command is substituted for 'sleep 10', the code works
perfectly.

Why can't the 'yes' command be interrupted using the SIGALRM method when the
sleep command can? I realize that noone would really run 'yes' but what if
a user account has a buggy .bashrc and loops forever? Just one thread
locking up like this holds up all the others.

Any ideas or suggestions on how to handle such situations in a
multi-threaded way would be appreciated.

Cheers.
Adrian Casey.
Alice Springs Linux User Goup.
http://www.aslug.org.au

 

[Nick Craig-Wood]

I have a multi-threaded python application which uses pexpect to connect to
multiple systems concurrently. Each thread within my application is a
connection to a remote system. The problem is when one of the child
threads runs a command which generates an unlimited amount of output. The
classic example of this is the "yes" command. If you
execute "pexpect.run('yes')", your cpu will sit at 100% forever.

Here is a simple multi-threaded program using pexpect which demonstrates the
problem. The command 'yes' is run in a thread. The parent says that when
the alarm goes off, run the handler function. The thread sets the alarm to
trigger after 5 seconds.

1) Don't ever mix threads and signals - you are heading for trouble!

2) pexpect has a timeout parameter exactly for this case

import os, pexpect, threading

def runyes():
print "Running yes command..."
pexpect.run('yes', timeout=5)

t = threading.Thread(target=runyes)
t.start()
t.join()

 

[Adrian Casey]

1) Don't ever mix threads and signals - you are heading for trouble!

2) pexpect has a timeout parameter exactly for this case

import os, pexpect, threading

def runyes():
print "Running yes command..."
pexpect.run('yes', timeout=5)

t = threading.Thread(target=runyes)
t.start()
t.join()

The timeout parameter will not work in this case. If you run the sample
code above, it will run forever. The 'yes' command presents a class of
command which can not be easily be handled by pexpect. As far as I know,
mixing threads and signals is OK provided the parent creates the alarm.

Adrian.

 

[Nick Craig-Wood]

The timeout parameter will not work in this case. If you run the sample
code above, it will run forever.

The above runs just fine for me, stopping after 5 seconds. Did you
try it?

The 'yes' command presents a class of command which can not be
easily be handled by pexpect.

Worked for me under Debian/testing.

As far as I know, mixing threads and signals is OK provided the
parent creates the alarm.

There are so many pitfalls here that I advise you not to try. From
the linuxthreads FAQ

J.3: How shall I go about mixing signals and threads in my program?

The less you mix them, the better. Notice that all pthread_*
functions are not async-signal safe, meaning that you should not
call them from signal handlers. This recommendation is not to be
taken lightly: your program can deadlock if you call a pthread_*
function from a signal handler!

The only sensible things you can do from a signal handler is set a
global flag, or call sem_post on a semaphore, to record the delivery
of the signal. The remainder of the program can then either poll the
global flag, or use sem_wait() and sem_trywait() on the semaphore.

Another option is to do nothing in the signal handler, and dedicate
one thread (preferably the initial thread) to wait synchronously for
signals, using sigwait(), and send messages to the other threads
accordingly.

Note also that the signal can be delivered to any thread which
complicates things.

 

[Fredrik Lundh]

The only sensible things you can do from a signal handler is set a
global flag, or call sem_post on a semaphore, to record the delivery
of the signal. The remainder of the program can then either poll the
global flag, or use sem_wait() and sem_trywait() on the semaphore.

but that's exactly what Python's signal handlers do, right ?

(the interpreter uses a "pending call" queue to collect events, and
executes them from the interpreter main loop in a controlled fashion).

</F>

 

[Nick Craig-Wood]

but that's exactly what Python's signal handlers do, right ?

(the interpreter uses a "pending call" queue to collect events, and
executes them from the interpreter main loop in a controlled
fashion).

Yes you are absolutely right

From http://docs.python.org/lib/module-signal.html

Some care must be taken if both signals and threads are used in
the same program. The fundamental thing to remember in using
signals and threads simultaneously is: always perform signal()
operations in the main thread of execution. Any thread can perform
an alarm(), getsignal(), or pause(); only the main thread can set
a new signal handler, and the main thread will be the only one to
receive signals (this is enforced by the Python signal module,
even if the underlying thread implementation supports sending
signals to individual threads). This means that signals can't be
used as a means of inter-thread communication. Use locks instead.

 

[Adrian Casey]

The above runs just fine for me, stopping after 5 seconds. Did you
try it?


Worked for me under Debian/testing.


There are so many pitfalls here that I advise you not to try. From
the linuxthreads FAQ

J.3: How shall I go about mixing signals and threads in my program?

The less you mix them, the better. Notice that all pthread_*
functions are not async-signal safe, meaning that you should not
call them from signal handlers. This recommendation is not to be
taken lightly: your program can deadlock if you call a pthread_*
function from a signal handler!

The only sensible things you can do from a signal handler is set a
global flag, or call sem_post on a semaphore, to record the delivery
of the signal. The remainder of the program can then either poll the
global flag, or use sem_wait() and sem_trywait() on the semaphore.

Another option is to do nothing in the signal handler, and dedicate
one thread (preferably the initial thread) to wait synchronously for
signals, using sigwait(), and send messages to the other threads
accordingly.

Note also that the signal can be delivered to any thread which
complicates things.

I'm running Kubuntu 06-06 with python 2.4.3 and the above code runs forever
at 100% cpu utilization. I shall look into semaphores. However, that means
another thread whose sole purpose is to watch the semaphore.

Thanks for you help.

Cheers.
Adrian.

 

[Nick Craig-Wood]

I'm running Kubuntu 06-06 with python 2.4.3 and the above code runs forever
at 100% cpu utilization.

Interesting... I wonder if that is a fixed bug.

On Debian/etch with python-pexpect 2.1-1 I get

Python 2.4.4c0 (#2, Jul 30 2006, 15:43:58)
[GCC 4.1.2 20060715 (prerelease) (Debian 4.1.1-9)] on linux2
Type "help", "copyright", "credits" or "license" for more information. ... print "Running yes command..."
... pexpect.run('yes', timeout=5)
...

Wheras on Ubuntu/dapper with python-pexpect 0.999-5ubuntu2 I get

Python 2.4.3 (#2, Apr 27 2006, 14:43:58)
[GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2
Type "help", "copyright", "credits" or "license" for more information. ... print "Running yes command..."
... pexpect.run('yes', timeout=5)
... t.join()
[never returns]

I'd guess at differences between the pexpect versions. You could try
the pexpect from debian/testing easily enough I expect.