How to force a thread to stop

[Nick Craig-Wood]

No, Python has no threadicide method

Actually it does in the C API, but it isn't exported to python.
ctypes can fix that though.

and its absence is not an oversight. Threads often have important
business left to do, such as releasing locks on shared data; killing
them at arbitrary times tends to leave the system in an inconsistent
state.

Here is a demo of how to kill threads in python in a cross platform
way. It requires ctypes. Not sure I'd use the code in production but
it does work...

"""
How to kill a thread demo
"""

import threading
import time
import ctypes

class ThreadKilledError(Exception): pass
_PyThreadState_SetAsyncExc = ctypes.pythonapi.PyThreadState_SetAsyncExc
_c_ThreadKilledError = ctypes.py_object(ThreadKilledError)

def _do_stuff(t):
"""Busyish wait for t seconds. Just sleeping delays the exeptions in the example"""
start = time.time()
while time.time() - start < t:
time.sleep(0.01)

class KillableThread(threading.Thread):
"""
Show how to kill a thread
"""
def __init__(self, name="thread", *args, **kwargs):
threading.Thread.__init__(self, *args, **kwargs)
self.name = name
print "Starting %s" % self.name
def kill(self):
"""Kill this thread"""
print "Killing %s" % self.name
_PyThreadState_SetAsyncExc(self.id, _c_ThreadKilledError)
def run(self):
self.id = threading._get_ident()
while 1:
print "Thread %s running" % self.name
_do_stuff(1.0)

if __name__ == "__main__":
thread1 = KillableThread(name="thread1")
thread1.start()
_do_stuff(0.5)
thread2 = KillableThread(name="thread2")
thread2.start()
_do_stuff(2.0)
thread1.kill()
thread1.join()
_do_stuff(2.0)
thread2.kill()
thread2.join()
print "Done"

 

[Paul Boddie]

Instead of using os.system, maybe you want to use one of the popens or
the subprocess module. For each ssh, you'd spawn off a process that
does the ssh and communicates back to the control process through a
set of file descriptors (Unix pipe endpoints or whatever). The
control process could use either threads or polling/select to talk to
the pipes and keep track of what the subprocesses were doing.

For some insight into what you might need to do to monitor asynchronous
communications, take a look at the parallel/pprocess module, which I
wrote as a convenience for spawning processes using a thread
module-style API whilst providing explicit channels for interprocess
communication:

http://www.python.org/pypi/parallel

Better examples can presumably be found in any asynchronous
communications framework, I'm sure.

I don't think you need anything as complex as shared memory for this.
You're just writing a special purpose chat server.

Indeed. The questioner might want to look at the py.execnet software
that has been presented now at two consecutive EuroPython conferences
(at the very least):

http://indico.cern.ch/contributionDisplay.py?contribId=91&sessionId=41&confId=44

Whether this solves the questioner's problems remains to be seen, but
issues of handling SSH-based communications streams do seem to be
addressed.

Paul

 

[Dennis Lee Bieber]

If a thread never performs any I/O operations, signal handlers will still
get invokes on the arrival of a signal.

I thought I'd qualified that scenario -- from the point of view of
one OS I've programmed, where signals were only handled as part of the
I/O system (or, in more general, as part of any /blocking/ system call
from an application -- not even a <ctrl-c> would have an effect unless
the application invoked a blocking call, or explicitly polled the signal
bits of its task header)
--
Wulfraed Dennis Lee Bieber KD6MOG
(e-mail address removed) (e-mail address removed)
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: (e-mail address removed))
HTTP://www.bestiaria.com/

 

[sjdevnull]

Well, I guess I'm thinking of an event driven mechanism, kinda like
setting up signal handlers. I don't necessarily know how it works under
the hood, but I don't poll for a signal. I setup a handler, when the
signal comes, if it comes, the handler gets thrown into action. That's
what I'd be interesting in doing with threads.

Note that you see many of the same problems with signal handlers
(including only being able to call reentrant functions from them).

Most advanced Unix programming books say you should treat signal
handlers in a manner similar to what people are advocating for remote
thread stoppage in this thread: unless you're doing something trivial,
your signal handler should just set a global variable. Then your
process can check that variable in the main loop and take more complex
action if it's set.

 

[Paul Rubin]

Whether this solves the questioner's problems remains to be seen, but
issues of handling SSH-based communications streams do seem to be
addressed.

Actually I don't understand the need for SSH. This is traffic over a
LAN, right? Is all of the LAN traffic encrypted? That's unusual; SSH
is normally used to secure connections over the internet, but the
local network is usually trusted. Hopefully it's not wireless.

 

[Carl J. Van Arsdall]

Actually I don't understand the need for SSH. This is traffic over a
LAN, right? Is all of the LAN traffic encrypted? That's unusual; SSH
is normally used to secure connections over the internet, but the
local network is usually trusted. Hopefully it's not wireless.

The reason for ssh is legacy. I think the person who originally set
things up (it was an 8 node farm at the time) just used ssh to execute
commands on the remote machine. It was a quick and dirty approach I
believe, but at the time, it wasn't worth investing in anything better.
Just setup some keys for each node and use ssh (as opposed to rexec or
something else, I doubt they put much thought into it). Its not a need
as much as I was working with what was handed, and as in many projects,
it becomes difficult to change everything at once. So the goal was to
change what we had to and abstract the ssh calls away so that we could
do something better later.

-c

--

Carl J. Van Arsdall
(e-mail address removed)
Build and Release
MontaVista Software

 

[sjdevnull]

Actually I don't understand the need for SSH. This is traffic over a
LAN, right? Is all of the LAN traffic encrypted? That's unusual; SSH
is normally used to secure connections over the internet, but the
local network is usually trusted. Hopefully it's not wireless.

Most places I've worked do use a lot of encryption on the LAN. Not
everything on the LAN is encrypted (e.g outgoing http connections) but
a lot of things are. Trusting the whole network is a bad idea, since
it allows a compromise of one machine to turn into a compromise of the
whole LAN.

 

[bryanjugglercryptographer]

Actually I don't understand the need for SSH.

Who are you and what have you done with the real Paul Rubin?

This is traffic over a
LAN, right? Is all of the LAN traffic encrypted? That's unusual; SSH
is normally used to secure connections over the internet, but the
local network is usually trusted. Hopefully it's not wireless.

I think not running telnet and rsh daemons is a good policy anyway.

 

[H J van Rooyen]

| On Thu, 27 Jul 2006 08:48:37 -0400, Jean-Paul Calderone
| <[email protected]> declaimed the following in comp.lang.python:
|
| >
| > If a thread never performs any I/O operations, signal handlers will still
| > get invokes on the arrival of a signal.
| >
| I thought I'd qualified that scenario -- from the point of view of
| one OS I've programmed, where signals were only handled as part of the
| I/O system (or, in more general, as part of any /blocking/ system call
| from an application -- not even a <ctrl-c> would have an effect unless
| the application invoked a blocking call, or explicitly polled the signal
| bits of its task header)
| --

I have to support this - a processor is only doing one instruction at a time, if
its not a multi core device - and the only ways that the "operating system part"
of the system can get control back from the *user program part* of the system
are:

a) when the User makes an OS call (like blocking I/O, or any OS request)
b) when the user code is interrupted by some hardware thingy and the OS part
handles the interrupt.

So on a processor that does not have protected instructions - if an idiot writes
something to the following effect :

*instructions to disable interrupts*

followed by :

*instructions that go into an infinite loop AND that make no OS calls*

the whole bang shoot stops dead - Reset your machine...

Dennis - did your OS not have a ticker running?

- Hendrik

 

[Paul Rubin]

So on a processor that does not have protected instructions - if an
idiot writes something to the following effect :

*instructions to disable interrupts*

followed by :

*instructions that go into an infinite loop AND that make no OS calls*

the whole bang shoot stops dead - Reset your machine...

A common recovery mechanism in embedded systems is a watchdog timer,
which is a hardware device that must be poked by the software every
so often (e.g. by writing to some register). If too long an interval
goes by without a poke, the WDT hard-resets the cpu. Normally the
software would poke the WDT from its normal periodic timing routine.
A loop like you describe would stop the timing routine from running,
eventually resulting in a reset.

 

[Paul Boddie]

Actually I don't understand the need for SSH. This is traffic over a
LAN, right? Is all of the LAN traffic encrypted? That's unusual; SSH
is normally used to secure connections over the internet, but the
local network is usually trusted. Hopefully it's not wireless.

I don't run any wireless networks, but given the apparently poor state
of wireless network security (as far as the actual implemented
standards in commercially available products are concerned), I'd want
to be using as much encryption as possible if I did.

Anyway, the py.execnet thing is presumably designed to work over the
Internet and over local networks, with the benefit of SSH being that it
applies well to both domains. Whether it's a better solution for the
questioner's problem than established alternatives such as PVM (which
I've never had the need to look into, even though it seems
interesting), various distributed schedulers or anything else out
there, I can't really say.

Paul

 

[H J van Rooyen]

| > So on a processor that does not have protected instructions - if an
| > idiot writes something to the following effect :
| >
| > *instructions to disable interrupts*
| >
| > followed by :
| >
| > *instructions that go into an infinite loop AND that make no OS calls*
| >
| > the whole bang shoot stops dead - Reset your machine...
|
| A common recovery mechanism in embedded systems is a watchdog timer,
| which is a hardware device that must be poked by the software every
| so often (e.g. by writing to some register). If too long an interval
| goes by without a poke, the WDT hard-resets the cpu. Normally the
| software would poke the WDT from its normal periodic timing routine.
| A loop like you describe would stop the timing routine from running,
| eventually resulting in a reset.

*grin* - Yes of course - if the WDT was enabled - its something that I have not
seen on PC's yet...

- Hendrik

 

[Paul Rubin]

Anyway, the py.execnet thing is presumably designed to work over the
Internet and over local networks, with the benefit of SSH being that it
applies well to both domains. Whether it's a better solution for the
questioner's problem than established alternatives such as PVM (which
I've never had the need to look into, even though it seems
interesting), various distributed schedulers or anything else out
there, I can't really say.

You could use ssh's port forwarding features and just open a normal
TCP connection to a local port that the local ssh server listens to.
Then the ssh server forwards the traffic through an encrypted tunnel
to the other machine. Your application doesn't have to know anything
about ssh.

In fact there's a VPN function (using tun/tap) in recent versions of
ssh that should make it even simpler, but I hvean't tried it yet.

 

[Paul Rubin]

*grin* - Yes of course - if the WDT was enabled - its something that
I have not seen on PC's yet...

They are available for PC's, as plug-in cards, at least for the ISA
bus in the old days, and almost certainly for the PCI bus today.

 

[Dennis Lee Bieber]

Dennis - did your OS not have a ticker running?

That ancient machine, while round-robin, multi-priority,
pre-emptive, seemed still to only "deliver" signals on deliberate
blocking calls -- perhaps to prevent potential corruption if the signal
had been delivered(handled) in the middle of some multi-instruction
sequence. The OS level would "see" the <ctrl-c>, and set the signal bit
in the task header -- but without the blocking I/O (typically), the code
to activate a registered signal handler would not be invoked. Operation
was something like: submit I/O request, AND(signal bits, signal mask) --
invoke handler if non-zero, block for I/O return [or return directly for
asynchronous I/O request]

--
Wulfraed Dennis Lee Bieber KD6MOG
(e-mail address removed) (e-mail address removed)
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: (e-mail address removed))
HTTP://www.bestiaria.com/

 

[H J van Rooyen]

| > *grin* - Yes of course - if the WDT was enabled - its something that
| > I have not seen on PC's yet...
|
| They are available for PC's, as plug-in cards, at least for the ISA
| bus in the old days, and almost certainly for the PCI bus today.

That is cool, I was not aware of this - added to a long running server it will
help to make the system more stable - a hardware solution to hard to find bugs
in Software - (or even stuff like soft errors in hardware - speak to the
Avionics boys about Neutrons) do you know who sells them and what they are
called? -

Sorry if this is getting off topic on this thread... ( in a way it is on topic -
because a reset will stop a thread every time...)

- Hendrik

 

[H J van Rooyen]

| On Fri, 28 Jul 2006 08:27:18 +0200, "H J van Rooyen"
| <[email protected]> declaimed the following in comp.lang.python:
|
| >
| > Dennis - did your OS not have a ticker running?
| >
| That ancient machine, while round-robin, multi-priority,
| pre-emptive, seemed still to only "deliver" signals on deliberate
| blocking calls -- perhaps to prevent potential corruption if the signal
| had been delivered(handled) in the middle of some multi-instruction
| sequence. The OS level would "see" the <ctrl-c>, and set the signal bit
| in the task header -- but without the blocking I/O (typically), the code
| to activate a registered signal handler would not be invoked. Operation
| was something like: submit I/O request, AND(signal bits, signal mask) --
| invoke handler if non-zero, block for I/O return [or return directly for
| asynchronous I/O request]

- Hah! - so it *could* have responded - it just chose not to - so it was pre
emptive - but hey - what is different between modern OS's and what you are
describing? - it seems to me that there is just a lot of extra memory control,
as well as control over who is allowed to do what - in an effort to make things
more stable - and all this stuff just eats cycles and slows you down... (or
alternatively, makes the hardware more complex and expensive...)

But to get back to the OP's problem - basically the thread has to see some sort
of variable change, or receive a message (by examining something to see if there
is a message there) and then kill itself, or the OS must be told to stop giving
control back to the thread in question - which option will leave all the loose
ends in the thread loose...

So its either: "hey mr nice thread please stop" - or "hey Mr OS - kill that
thread ..." - now from the OS viewpoint - if the application implements some
threading itself - it may not even know that the thread exists - OS threads are
known variously as "Tasks" or "Processes" or "Running Programmes" - so using the
big guns on a thread may not be possible without killing the parent too...

So if you want to use the OS to kill the thread - it has to be a formal OS
thread - something started with a call to the OS, and not something that an
application implements by itself - and I am not familiar enough with Python
threading and dummy threading to pretend to know what is "under the hood" - but
I haven't seen an additional process appearing on my Linux box when I start a
thread - so its either something that Python does on its own without
"registering" the new thread with Linux - or I haven't looked closely enough...

So if somebody else can take over here, we might convince the OP that "hey mr
nice thread" is the way to go, even in the case that the "thread" in question is
an OS Process - after all - there has to be inter - task communication in any
case - so the cleanest solution is to build the kill in right from scratch...

Why do I think of COBOL:

read master_file_record at end go to end_routine....

HTH - Hendrik

 

[Paul Rubin]

That is cool, I was not aware of this - added to a long running server it will
help to make the system more stable - a hardware solution to hard to find bugs
in Software - (or even stuff like soft errors in hardware - speak to the
Avionics boys about Neutrons) do you know who sells them and what they are
called? -

I usually try froogle.com to find stuff like that.

 

[Damjan]

| A common recovery mechanism in embedded systems is a watchdog timer,

| which is a hardware device that must be poked by the software every
| so often (e.g. by writing to some register). If too long an interval
| goes by without a poke, the WDT hard-resets the cpu. Normally the
| software would poke the WDT from its normal periodic timing routine.
| A loop like you describe would stop the timing routine from running,
| eventually resulting in a reset.

*grin* - Yes of course - if the WDT was enabled - its something that I
have not seen on PC's yet...

The intel 810 chipset (and all after that) has a builtin watchdog timer -
unfortunetally on some motherboards it's disabled (I guess in the BIOS).

How do I know that?
Once I got Linux installed on a new machine.... and although the install
went without a problem, after the first boot the machine would reboot on
exactly 2 minutes.
After a bit of poking around I found that hotplug detected the WDT support
and loaded the driver for it (i8xx_tco), and it seems the WDT chip was set
to start ticking right away after the driver poked it.

 

[H J van Rooyen]

| > | A common recovery mechanism in embedded systems is a watchdog timer,
| > | which is a hardware device that must be poked by the software every
| > | so often (e.g. by writing to some register). If too long an interval
| > | goes by without a poke, the WDT hard-resets the cpu. Normally the
| > | software would poke the WDT from its normal periodic timing routine.
| > | A loop like you describe would stop the timing routine from running,
| > | eventually resulting in a reset.
| >
| > *grin* - Yes of course - if the WDT was enabled - its something that I
| > have not seen on PC's yet...
|
| The intel 810 chipset (and all after that) has a builtin watchdog timer -
| unfortunetally on some motherboards it's disabled (I guess in the BIOS).
|
| How do I know that?
| Once I got Linux installed on a new machine.... and although the install
| went without a problem, after the first boot the machine would reboot on
| exactly 2 minutes.
| After a bit of poking around I found that hotplug detected the WDT support
| and loaded the driver for it (i8xx_tco), and it seems the WDT chip was set
| to start ticking right away after the driver poked it.

Yikes! "some poking around" - with two minutes to do it in - must have scarred
you for life!

- Hendrik