thread, threading; how to kill a thread?

Discussion in 'Python' started by Jerry Sievers, Nov 17, 2004.

  1. Greetings Pythonists;

    I have limited experience with threaded apps and plenty with old style
    forked heavyweight multi-processing apps.

    Using Python 2.3.3 on a Redhat 7.x machine.

    Wondering if there is a simple way from a main python program to kill
    a running thread? I see with the 'threading' module the way daemonic
    threads behave when the main program finishes.

    But suppose we have a simple thread running a function like;

    def timedLoop():
    while True:
    time.sleep(10)
    doSomething()

    All I am trying to do is stop that thread immediatly from the main
    program and it is unclear how to do this. I see that using
    del(threadObj) on the thread object that's running the loop function
    does nothing.

    Didn't notice anything obvious in the docs like a 'kill' method or
    similar. I don't necessarily want the main program to exit, just to
    kill one or more threads.

    Locks, conditions, semafores, signals?

    ARG! For heavens sake, what am I missing?

    Thanks


    --
    -------------------------------------------------------------------------------
    Jerry Sievers 305 854-3001 (home) WWW ECommerce Consultant
    305 321-1144 (mobile http://www.JerrySievers.com/
     
    Jerry Sievers, Nov 17, 2004
    #1
    1. Advertising

  2. Jerry Sievers wrote:

    >I have limited experience with threaded apps and plenty with old style
    >forked heavyweight multi-processing apps.
    >
    >Wondering if there is a simple way from a main python program to kill
    >a running thread?
    >

    There is no way to 'kill' a thread in Python. You will need to do
    something like

    def timedLoop():
    while not done:
    time.sleep(10)
    doSomething()

    Clearly, done should be implemented as an instance variable of the
    thread object rather than a global, and it should probably be set by a
    setDone() method that does any necessary locking. And yes, the thread
    will not terminate immediately, but only the next time the loop test is
    executed.

    Aaron
     
    Aaron Bingham, Nov 17, 2004
    #2
    1. Advertising

  3. Jerry Sievers

    Peter Hansen Guest

    Jerry Sievers wrote:
    > Wondering if there is a simple way from a main python program to kill
    > a running thread?


    No.

    > I see with the 'threading' module the way daemonic
    > threads behave when the main program finishes.


    This is one of several approaches, any of which -- or none of
    which -- might be suitable for you.

    What drives your need for this behaviour? The answer
    will determine the best approach.

    > All I am trying to do is stop that thread immediatly from the main


    Define "immediately". In answering please consider issues such
    as threads that are blocked in kernel calls, such as receiving
    data on a socket, as well as data integrity issues (e.g. what happens
    if by "immediately" you mean "at the end of the current bytecode
    instruction", and if that implies that you may have system
    data structures that are corrupted?).

    > Didn't notice anything obvious in the docs like a 'kill' method or
    > similar. I don't necessarily want the main program to exit, just to
    > kill one or more threads.


    Searching the list/newsgroup archives with Google will reveal
    past discussions and possibly some useful responses.

    -Peter
     
    Peter Hansen, Nov 17, 2004
    #3
  4. Jerry Sievers

    Kent Johnson Guest

    Aaron Bingham wrote:
    > There is no way to 'kill' a thread in Python. You will need to do
    > something like
    >
    > def timedLoop():
    > while not done:
    > time.sleep(10)
    > doSomething()


    See this recipe for an example using a threading.Event as the flag:
    http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/65448

    Kent
     
    Kent Johnson, Nov 17, 2004
    #4
  5. Jerry Sievers

    JCM Guest

    Jerry Sievers <> wrote:
    > Greetings Pythonists;


    > I have limited experience with threaded apps and plenty with old style
    > forked heavyweight multi-processing apps.


    > Using Python 2.3.3 on a Redhat 7.x machine.


    > Wondering if there is a simple way from a main python program to kill
    > a running thread?


    Sort of. You can send a unix signal to the main thread (it causes an
    exception which you can catch). But as others have and will surely
    point out, it's usually better to have your thread occasionally check
    whether it should die, so it doesn't leave anything in a bad state.
     
    JCM, Nov 17, 2004
    #5
  6. Jerry Sievers

    Ian Bicking Guest

    I might reiterate this request, but knowing the standard answer (you
    can't) -- what if I *really* want to? In situations when a thread
    becomes wedged, I don't know a way to get out of it. I don't even know
    a way to end the process elegantly, instead I have to kill -9 the process.

    Even if it messes things up and I have to restart the process at the
    soonest possible time to clean up the problems, it would be nice to be
    able to exit the process. As it is, the only robust way to deal with
    the situation is to have a separate process that (at the original
    process's request) will kill it, an assisted-suicide style. That's
    awkward to handle.

    This is an issue with long-running threaded servers (like Webware or
    even Zope), where even in the face of bugs it's important to keep the
    service up.

    --
    Ian Bicking / / http://blog.ianbicking.org
     
    Ian Bicking, Nov 17, 2004
    #6
  7. Jerry Sievers

    Peter Hansen Guest

    Ian Bicking wrote:
    > I might reiterate this request, but knowing the standard answer (you
    > can't) -- what if I *really* want to?


    It looks to me sort of as if you are actually reiterating the
    request above... but then going ahead and answering your own
    question. Is that what happened here? Because if it isn't,
    I'm not sure what other answer one could give....

    > In situations when a thread
    > becomes wedged, I don't know a way to get out of it. I don't even know
    > a way to end the process elegantly, instead I have to kill -9 the process.
    >
    > Even if it messes things up and I have to restart the process at the
    > soonest possible time to clean up the problems, it would be nice to be
    > able to exit the process. As it is, the only robust way to deal with
    > the situation is to have a separate process that (at the original
    > process's request) will kill it, an assisted-suicide style. That's
    > awkward to handle.
    >
    > This is an issue with long-running threaded servers (like Webware or
    > even Zope), where even in the face of bugs it's important to keep the
    > service up.
     
    Peter Hansen, Nov 17, 2004
    #7
  8. Jerry Sievers

    Ian Bicking Guest

    Peter Hansen wrote:
    > Ian Bicking wrote:
    >
    >> I might reiterate this request, but knowing the standard answer (you
    >> can't) -- what if I *really* want to?

    >
    >
    > It looks to me sort of as if you are actually reiterating the
    > request above... but then going ahead and answering your own
    > question. Is that what happened here? Because if it isn't,
    > I'm not sure what other answer one could give....


    It's more, "yes, I know the answer, but I remain unsatisfied". I'm 100%
    okay with a platform-specific way to accomplish this (especially if the
    platform is Linux and BSD). I'm okay with horrid hacks, or memory
    leaks, or other possible compromises. But I'm not really okay with the
    standard answer. And so I'm hoping someone else felt the same way and
    figured something out...?

    --
    Ian Bicking / / http://blog.ianbicking.org
     
    Ian Bicking, Nov 17, 2004
    #8
  9. Jerry Sievers

    David Bolen Guest

    Ian Bicking <> writes:

    > (...)
    > Even if it messes things up and I have to restart the process at the
    > soonest possible time to clean up the problems, it would be nice to be
    > able to exit the process. (...)


    I'm not positive, but os._exit() may at least let you get the process
    exited regardless of any thread states.

    -- David
     
    David Bolen, Nov 17, 2004
    #9
  10. Jerry Sievers

    David Bolen Guest

    Ian Bicking <> writes:

    > It's more, "yes, I know the answer, but I remain unsatisfied". I'm
    > 100% okay with a platform-specific way to accomplish this (especially
    > if the platform is Linux and BSD). I'm okay with horrid hacks, or
    > memory leaks, or other possible compromises. But I'm not really okay
    > with the standard answer. And so I'm hoping someone else felt the
    > same way and figured something out...?


    I'm not familiar enough with pthreads to guess there, but under
    Windows, you can technically accomplish what you want using
    TerminateThread. Perhaps there is a similar flow that could be used
    with pthreads.

    My major concern though in any attempt to externally kill off a thread
    would be that you somehow strand the interpreter with that dead thread
    still owning the GIL. There's plenty of other sorts of system
    resources that could also be stranded (at least until process exit),
    but killing off the GIL owner would definitely mess up the
    application's day.

    As a sample, the following works under Windows:

    - - - - - - - - - - - - - - - - - - - - - - - - -

    import threading
    import ctypes
    import time

    w32 = ctypes.windll.kernel32
    THREAD_TERMINATE = 1 # Privilege level for termination

    class DummyThread(threading.Thread):

    def __init__(self):
    threading.Thread.__init__(self)
    self.setDaemon(1)

    def run(self):
    self.tid = w32.GetCurrentThreadId()

    while 1:
    print 'Running'
    time.sleep(1)

    def kill_thread(threadobj):

    handle = w32.OpenThread(THREAD_TERMINATE, False, threadobj.tid)
    result = w32.TerminateThread(handle, 0)
    w32.CloseHandle(handle)

    return result

    if __name__ == "__main__":

    print 'Starting thread...'
    x = DummyThread()
    x.start()

    time.sleep(5)
    print 'Terminating thread...'
    kill_thread(x)

    time.sleep(5)
    print 'Exiting'

    - - - - - - - - - - - - - - - - - - - - - - - - -

    In DummyThread, a "self.get_ident()" appears to return the native
    platform thread id (so the same as the GetCurrentThreadId() call), but
    I didn't want to depend on that necessarily being the case.

    Of course, while the above runs on my system, I do think it has a GIL
    risk - I expect that ctypes is releasing the GIL just before calling
    the native function, so there's a chance the other thread could resume
    and obtain the GIL just prior to TerminateThread executing at the
    Win32 layer. My odds are probably lower in this example since the
    dummy thread is sleeping most of the time. So if I was going to use
    this in practice I'd probably provide my own extension module that
    wrapped TerminateThread but made sure *not* to release the GIL before
    calling it.

    Of course, there are other pieces of state that remained messed up,
    particularly if you use the threading package as above - it still
    thinks the thread is alive and would wait for it at process exit.
    Setting it to a daemon works as above, but you could also get crufty
    and just reach in and manually call the __stop and __delete methods of
    the thread object (dealing with the name mangling).

    Not sure I'd ever have the gumption to use this in practice unless my
    very next step was to exit the process (in which case setting the
    thread as daemon works just about as well), but it's technically
    possible :)

    -- David
     
    David Bolen, Nov 17, 2004
    #10
  11. Jerry Sievers

    Ian Bicking Guest

    David Bolen wrote:
    > Ian Bicking <> writes:
    >
    >
    >>(...)
    >>Even if it messes things up and I have to restart the process at the
    >>soonest possible time to clean up the problems, it would be nice to be
    >>able to exit the process. (...)

    >
    >
    > I'm not positive, but os._exit() may at least let you get the process
    > exited regardless of any thread states.


    Indeed, sir, you are right! Thanks.

    I whipped up something to try to do the right thing (sys.exit), unless
    things go badly, then do the wrong thing (os._exit):

    http://svn.colorstudy.com/home/ianb/thread_die.py

    In Webware we usually run the server so that if it exits with an error
    code of 3, the server gets restarted. (In other setups you might always
    restart the server.) Anyway, this way we could actually test if there
    were wedged threads, and restart the whole server if so. At the same
    time, most resources should be properly freed.

    --
    Ian Bicking / / http://blog.ianbicking.org
     
    Ian Bicking, Nov 17, 2004
    #11
  12. David Bolen <> writes:

    [...]
    > Of course, while the above runs on my system, I do think it has a GIL
    > risk - I expect that ctypes is releasing the GIL just before calling
    > the native function, so there's a chance the other thread could resume
    > and obtain the GIL just prior to TerminateThread executing at the
    > Win32 layer. My odds are probably lower in this example since the
    > dummy thread is sleeping most of the time. So if I was going to use
    > this in practice I'd probably provide my own extension module that
    > wrapped TerminateThread but made sure *not* to release the GIL before
    > calling it.


    For the record: Yes, ctypes releases and reacquires the GIL around
    normal function calls. But in recent releases this can even be
    customized - there's a FUNCFLAG_PYTHONAPI flag which prevents this,
    because it wouldn't be possible to call Python api functions otherwise.
    See the sources for details (it also calls PyErr_Occurred() after the
    function call, which is unneeded in your case but also doesn't hurt).

    Thomas
     
    Thomas Heller, Nov 18, 2004
    #12
  13. I had a very similar problem recently and I think finally found a
    solution for that.

    Let me first tell you that I needed this in a multi-threaded C++
    program. Python is embedded into C++. I was able to kill the threads
    using the TerminateThread API function of Windows; however this caused
    GIL problems. The terminated thread doesn't release the GIL and all
    other Python scripts running on different threads enter a deadlock.

    So, I looked for a magical function that is called from a thread
    (could be the main thread of the application) and this call will fail
    another thread. In other words, lets say i have two threads: T1 and T2
    - T1 wants to terminate T2. To achieve this, you should use the
    PyThreadState_SetAsyncExc function.

    In C++, my code is something like this (application specific parts of
    the code it removed):

    void SetAsyncExc (int nThreadId, PyThreadState * threadState)
    {
    try
    {
    PyEval_AcquireLock ();
    PyThreadState_Swap (threadState);
    PyObject * exc = PyString_FromString ("Exit interrupt!");
    //PyObject * exc = PyErr_NewException ("TerminateThread", NULL,
    NULL);

    int count = PyThreadState_SetAsyncExc (nThreadId, exc);
    if (count > 1) // we're in trouble!
    PyThreadState_SetAsyncExc (nThreadId, NULL);

    Py_DECREF (exc);

    PyThreadState_Swap (NULL);
    PyEval_ReleaseLock ();
    }
    catch (...)
    {
    #ifdef _DEBUG
    AfxMessageBox (_T ("Exception thrown in CPyManager::SetAsyncExc "));
    #endif
    }
    return true;
    }

    You should be able to convert this code into a Python code easily. I
    will leave that up to you ;)

    nThreadId is the ID of the thread that you want to terminate (i.e.
    thread id of T2). threadState is a pointer to the ThreadState
    structure of thread that wants to terminate the target thread (i.e.
    thread state of T1). Calling this function will cause an exception to
    be thrown in T2 - Please note that i simply created a string as an
    exception however instead of a string, a proper exception object
    should be created. When the exception is thrown and caught in T2, T2
    should simply exit.

    I hope this helps.

    Best wishes,
    Mustafa Demirhan



    Jerry Sievers <> wrote in message news:<>...
    > Greetings Pythonists;
    >
    > I have limited experience with threaded apps and plenty with old style
    > forked heavyweight multi-processing apps.
    >
    > Using Python 2.3.3 on a Redhat 7.x machine.
    >
    > Wondering if there is a simple way from a main python program to kill
    > a running thread? I see with the 'threading' module the way daemonic
    > threads behave when the main program finishes.
    >
    > But suppose we have a simple thread running a function like;
    >
    > def timedLoop():
    > while True:
    > time.sleep(10)
    > doSomething()
    >
    > All I am trying to do is stop that thread immediatly from the main
    > program and it is unclear how to do this. I see that using
    > del(threadObj) on the thread object that's running the loop function
    > does nothing.
    >
    > Didn't notice anything obvious in the docs like a 'kill' method or
    > similar. I don't necessarily want the main program to exit, just to
    > kill one or more threads.
    >
    > Locks, conditions, semafores, signals?
    >
    > ARG! For heavens sake, what am I missing?
    >
    > Thanks
     
    Mustafa Demirhan, Nov 19, 2004
    #13
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    9
    Views:
    1,056
    Mark Space
    Dec 29, 2007
  2. dmitrey
    Replies:
    7
    Views:
    395
    Antoon Pardon
    Sep 22, 2008
  3. Steven Woody
    Replies:
    0
    Views:
    431
    Steven Woody
    Jan 9, 2009
  4. Steven Woody
    Replies:
    0
    Views:
    461
    Steven Woody
    Jan 9, 2009
  5. Charles Oliver Nutter
    Replies:
    43
    Views:
    632
    Roger Pack
    Mar 25, 2008
Loading...

Share This Page