thread, threading; how to kill a thread?

[Jerry Sievers]

Greetings Pythonists;

I have limited experience with threaded apps and plenty with old style
forked heavyweight multi-processing apps.

Using Python 2.3.3 on a Redhat 7.x machine.

Wondering if there is a simple way from a main python program to kill
a running thread? I see with the 'threading' module the way daemonic
threads behave when the main program finishes.

But suppose we have a simple thread running a function like;

def timedLoop():
while True:
time.sleep(10)
doSomething()

All I am trying to do is stop that thread immediatly from the main
program and it is unclear how to do this. I see that using
del(threadObj) on the thread object that's running the loop function
does nothing.

Didn't notice anything obvious in the docs like a 'kill' method or
similar. I don't necessarily want the main program to exit, just to
kill one or more threads.

Locks, conditions, semafores, signals?

ARG! For heavens sake, what am I missing?

Thanks

 

[Aaron Bingham]

I have limited experience with threaded apps and plenty with old style
forked heavyweight multi-processing apps.

Wondering if there is a simple way from a main python program to kill
a running thread?

There is no way to 'kill' a thread in Python. You will need to do
something like

def timedLoop():
while not done:
time.sleep(10)
doSomething()

Clearly, done should be implemented as an instance variable of the
thread object rather than a global, and it should probably be set by a
setDone() method that does any necessary locking. And yes, the thread
will not terminate immediately, but only the next time the loop test is
executed.

Aaron

 

[Peter Hansen]

Wondering if there is a simple way from a main python program to kill
a running thread?
No.

I see with the 'threading' module the way daemonic
threads behave when the main program finishes.

This is one of several approaches, any of which -- or none of
which -- might be suitable for you.

What drives your need for this behaviour? The answer
will determine the best approach.

All I am trying to do is stop that thread immediatly from the main

Define "immediately". In answering please consider issues such
as threads that are blocked in kernel calls, such as receiving
data on a socket, as well as data integrity issues (e.g. what happens
if by "immediately" you mean "at the end of the current bytecode
instruction", and if that implies that you may have system
data structures that are corrupted?).

Didn't notice anything obvious in the docs like a 'kill' method or
similar. I don't necessarily want the main program to exit, just to
kill one or more threads.

Searching the list/newsgroup archives with Google will reveal
past discussions and possibly some useful responses.

-Peter

 

[Kent Johnson]

There is no way to 'kill' a thread in Python. You will need to do
something like

def timedLoop():
while not done:
time.sleep(10)
doSomething()

See this recipe for an example using a threading.Event as the flag:
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/65448

Kent

 

[JCM]

Greetings Pythonists;

I have limited experience with threaded apps and plenty with old style
forked heavyweight multi-processing apps.

Using Python 2.3.3 on a Redhat 7.x machine.

Wondering if there is a simple way from a main python program to kill
a running thread?

Sort of. You can send a unix signal to the main thread (it causes an
exception which you can catch). But as others have and will surely
point out, it's usually better to have your thread occasionally check
whether it should die, so it doesn't leave anything in a bad state.

 

[Ian Bicking]

I might reiterate this request, but knowing the standard answer (you
can't) -- what if I *really* want to? In situations when a thread
becomes wedged, I don't know a way to get out of it. I don't even know
a way to end the process elegantly, instead I have to kill -9 the process.

Even if it messes things up and I have to restart the process at the
soonest possible time to clean up the problems, it would be nice to be
able to exit the process. As it is, the only robust way to deal with
the situation is to have a separate process that (at the original
process's request) will kill it, an assisted-suicide style. That's
awkward to handle.

This is an issue with long-running threaded servers (like Webware or
even Zope), where even in the face of bugs it's important to keep the
service up.

 

[Peter Hansen]

I might reiterate this request, but knowing the standard answer (you
can't) -- what if I *really* want to?

It looks to me sort of as if you are actually reiterating the
request above... but then going ahead and answering your own
question. Is that what happened here? Because if it isn't,
I'm not sure what other answer one could give....

 

[Ian Bicking]

It looks to me sort of as if you are actually reiterating the
request above... but then going ahead and answering your own
question. Is that what happened here? Because if it isn't,
I'm not sure what other answer one could give....

It's more, "yes, I know the answer, but I remain unsatisfied". I'm 100%
okay with a platform-specific way to accomplish this (especially if the
platform is Linux and BSD). I'm okay with horrid hacks, or memory
leaks, or other possible compromises. But I'm not really okay with the
standard answer. And so I'm hoping someone else felt the same way and
figured something out...?

 

[David Bolen]

(...)
Even if it messes things up and I have to restart the process at the
soonest possible time to clean up the problems, it would be nice to be
able to exit the process. (...)

I'm not positive, but os._exit() may at least let you get the process
exited regardless of any thread states.

-- David

 

[David Bolen]

It's more, "yes, I know the answer, but I remain unsatisfied". I'm
100% okay with a platform-specific way to accomplish this (especially
if the platform is Linux and BSD). I'm okay with horrid hacks, or
memory leaks, or other possible compromises. But I'm not really okay
with the standard answer. And so I'm hoping someone else felt the
same way and figured something out...?

I'm not familiar enough with pthreads to guess there, but under
Windows, you can technically accomplish what you want using
TerminateThread. Perhaps there is a similar flow that could be used
with pthreads.

My major concern though in any attempt to externally kill off a thread
would be that you somehow strand the interpreter with that dead thread
still owning the GIL. There's plenty of other sorts of system
resources that could also be stranded (at least until process exit),
but killing off the GIL owner would definitely mess up the
application's day.

As a sample, the following works under Windows:

- - - - - - - - - - - - - - - - - - - - - - - - -

import threading
import ctypes
import time

w32 = ctypes.windll.kernel32
THREAD_TERMINATE = 1 # Privilege level for termination

class DummyThread(threading.Thread):

def __init__(self):
threading.Thread.__init__(self)
self.setDaemon(1)

def run(self):
self.tid = w32.GetCurrentThreadId()

while 1:
print 'Running'
time.sleep(1)

def kill_thread(threadobj):

handle = w32.OpenThread(THREAD_TERMINATE, False, threadobj.tid)
result = w32.TerminateThread(handle, 0)
w32.CloseHandle(handle)

return result

if __name__ == "__main__":

print 'Starting thread...'
x = DummyThread()
x.start()

time.sleep(5)
print 'Terminating thread...'
kill_thread(x)

time.sleep(5)
print 'Exiting'

- - - - - - - - - - - - - - - - - - - - - - - - -

In DummyThread, a "self.get_ident()" appears to return the native
platform thread id (so the same as the GetCurrentThreadId() call), but
I didn't want to depend on that necessarily being the case.

Of course, while the above runs on my system, I do think it has a GIL
risk - I expect that ctypes is releasing the GIL just before calling
the native function, so there's a chance the other thread could resume
and obtain the GIL just prior to TerminateThread executing at the
Win32 layer. My odds are probably lower in this example since the
dummy thread is sleeping most of the time. So if I was going to use
this in practice I'd probably provide my own extension module that
wrapped TerminateThread but made sure *not* to release the GIL before
calling it.

Of course, there are other pieces of state that remained messed up,
particularly if you use the threading package as above - it still
thinks the thread is alive and would wait for it at process exit.
Setting it to a daemon works as above, but you could also get crufty
and just reach in and manually call the __stop and __delete methods of
the thread object (dealing with the name mangling).

Not sure I'd ever have the gumption to use this in practice unless my
very next step was to exit the process (in which case setting the
thread as daemon works just about as well), but it's technically
possible

-- David

 

[Ian Bicking]

I'm not positive, but os._exit() may at least let you get the process
exited regardless of any thread states.

Indeed, sir, you are right! Thanks.

I whipped up something to try to do the right thing (sys.exit), unless
things go badly, then do the wrong thing (os._exit):

http://svn.colorstudy.com/home/ianb/thread_die.py

In Webware we usually run the server so that if it exits with an error
code of 3, the server gets restarted. (In other setups you might always
restart the server.) Anyway, this way we could actually test if there
were wedged threads, and restart the whole server if so. At the same
time, most resources should be properly freed.

 

[Thomas Heller]

[...]

Of course, while the above runs on my system, I do think it has a GIL
risk - I expect that ctypes is releasing the GIL just before calling
the native function, so there's a chance the other thread could resume
and obtain the GIL just prior to TerminateThread executing at the
Win32 layer. My odds are probably lower in this example since the
dummy thread is sleeping most of the time. So if I was going to use
this in practice I'd probably provide my own extension module that
wrapped TerminateThread but made sure *not* to release the GIL before
calling it.

For the record: Yes, ctypes releases and reacquires the GIL around
normal function calls. But in recent releases this can even be
customized - there's a FUNCFLAG_PYTHONAPI flag which prevents this,
because it wouldn't be possible to call Python api functions otherwise.
See the sources for details (it also calls PyErr_Occurred() after the
function call, which is unneeded in your case but also doesn't hurt).

Thomas

 

[Mustafa Demirhan]

I had a very similar problem recently and I think finally found a
solution for that.

Let me first tell you that I needed this in a multi-threaded C++
program. Python is embedded into C++. I was able to kill the threads
using the TerminateThread API function of Windows; however this caused
GIL problems. The terminated thread doesn't release the GIL and all
other Python scripts running on different threads enter a deadlock.

So, I looked for a magical function that is called from a thread
(could be the main thread of the application) and this call will fail
another thread. In other words, lets say i have two threads: T1 and T2
- T1 wants to terminate T2. To achieve this, you should use the
PyThreadState_SetAsyncExc function.

In C++, my code is something like this (application specific parts of
the code it removed):

void SetAsyncExc (int nThreadId, PyThreadState * threadState)
{
try
{
PyEval_AcquireLock ();
PyThreadState_Swap (threadState);
PyObject * exc = PyString_FromString ("Exit interrupt!");
//PyObject * exc = PyErr_NewException ("TerminateThread", NULL,
NULL);

int count = PyThreadState_SetAsyncExc (nThreadId, exc);
if (count > 1) // we're in trouble!
PyThreadState_SetAsyncExc (nThreadId, NULL);

Py_DECREF (exc);

PyThreadState_Swap (NULL);
PyEval_ReleaseLock ();
}
catch (...)
{
#ifdef _DEBUG
AfxMessageBox (_T ("Exception thrown in CPyManager::SetAsyncExc "));
#endif
}
return true;
}

You should be able to convert this code into a Python code easily. I
will leave that up to you

nThreadId is the ID of the thread that you want to terminate (i.e.
thread id of T2). threadState is a pointer to the ThreadState
structure of thread that wants to terminate the target thread (i.e.
thread state of T1). Calling this function will cause an exception to
be thrown in T2 - Please note that i simply created a string as an
exception however instead of a string, a proper exception object
should be created. When the exception is thrown and caught in T2, T2
should simply exit.

I hope this helps.

Best wishes,
Mustafa Demirhan