Killing threads

Discussion in 'Python' started by ericwoodworth@gmail.com, Apr 5, 2009.

  1. Guest

    Hi,
    I'm new to python and even newer to threading and it seems as
    though I'm missing something fundamental about threads. Basically I
    have a program that looks like this:

    class ThreadOne(threading.Thread):
    while 1:
    do stuff

    class ThreadTwo(threading.Thread):
    while 1:
    do other stuff


    first = ThreadOne()
    second = ThreadTwo()

    while 1:
    do stuff


    The issue that I'm having is...I don't know how to kill this app in
    window. I hit ctrl-c but that only seems to kill one of the threads.
    The rest of the app just lingers. There's got to be a more graceful
    way but so far I haven't googled anything up.

    I'm using queues to talk between these threads so I could certainly
    put some kind of message on the queue that causes the threads to
    commit suicide but I'm thinking there's a more built in way to do what
    I want. I'm just not sure what it is.

    I'd appreciate it if somebody could point me in the right direction.
    Thanks.
     
    , Apr 5, 2009
    #1
    1. Advertising

  2. Aahz Guest

    In article <>,
    <> wrote:
    >
    >I'm using queues to talk between these threads so I could certainly
    >put some kind of message on the queue that causes the threads to
    >commit suicide but I'm thinking there's a more built in way to do what
    >I want. I'm just not sure what it is.


    There isn't, you have the right idea about using queues.
    --
    Aahz () <*> http://www.pythoncraft.com/

    "Debugging is twice as hard as writing the code in the first place.
    Therefore, if you write the code as cleverly as possible, you are, by
    definition, not smart enough to debug it." --Brian W. Kernighan
     
    Aahz, Apr 5, 2009
    #2
    1. Advertising

  3. Guest

    On Apr 5, 12:22 am, (Aahz) wrote:
    > In article <>,
    >
    >  <> wrote:
    >
    > >I'm using queues to talk between these threads so I could certainly
    > >put some kind of message on the queue that causes the threads to
    > >commit suicide but I'm thinking there's a more built in way to do what
    > >I want.  I'm just not sure what it is.

    >
    > There isn't, you have the right idea about using queues.
    > --
    > Aahz ()           <*>        http://www.pythoncraft.com/
    >
    > "Debugging is twice as hard as writing the code in the first place.
    > Therefore, if you write the code as cleverly as possible, you are, by
    > definition, not smart enough to debug it."  --Brian W. Kernighan


    Ok good to know. I was letting the search for a really cool solution
    stop me from rolling out what I think I already know how to do.
    Thanks.
     
    , Apr 5, 2009
    #3
  4. imageguy Guest

    On Apr 4, 10:43 pm, wrote:
    > Hi,
    >      I'm new to python and even newer to threading and it seems as
    > though I'm missing something fundamental about threads.  Basically I
    > have a program that looks like this:
    >
    > class ThreadOne(threading.Thread):
    >      while 1:
    >           do stuff
    >
    > class ThreadTwo(threading.Thread):
    >      while 1:
    >           do other stuff
    >
    > first = ThreadOne()
    > second = ThreadTwo()
    >
    > while 1:
    >     do stuff
    >
    > The issue that I'm having is...I don't know how to kill this app in
    > window.  I hit ctrl-c but that only seems to kill one of the threads.
    > The rest of the app just lingers.  There's got to be a more graceful
    > way but so far I haven't googled anything up.
    >
    > I'm using queues to talk between these threads so I could certainly
    > put some kind of message on the queue that causes the threads to
    > commit suicide but I'm thinking there's a more built in way to do what
    > I want.  I'm just not sure what it is.
    >
    > I'd appreciate it if somebody could point me in the right direction.
    > Thanks.


    I am not an expert either, however, I think the standard practice is
    to use and threading.Event to communicate with the threads.

    So the structure goes something like this;
    In the main thread set a threading.Event called 'keepgoing'.
    Pass this object into the thread at instantiation or as a parameter in
    the .start method
    Over-ride the .run method to handle the process
    Change the while loop's condition to check the .keepgoing.isSet()
    In the primary thread, when you want the threads to stop, simply .clear
    () the keepgoing event
    The while loop terminates and at the end of the .run method, the
    thread will terminate.

    In the primary program, you should then .join() the worker threads.
    - doing this will block the program until all of the thread have
    stopped.

    So your example would become;

    class ThreadOne(threading.Thread):

    def __init__(self, keepgoing):
    threading.Thread.__init__(self)
    self.keepgoing = keepgoing

    def run(self)
    while self.keepgoing.isSet():
    do stuff thread one

    class ThreadTwo(threading.Thread):

    def __init__(self, keepgoing):
    threading.Thread.__init__(self)
    self.keepgoing = keepgoing

    def run(self)
    while self.keepgoing.isSet():
    do stuff in thread two
    #
    #now start the main process
    #

    keepgoing = threading.Event()
    keepgoing.set()

    first = ThreadOne(keepgoing)
    second = ThreadTwo(keepgoing)

    # you could also pass in the variable with a thread.start() method,
    which will then call the thread.run()

    while 1:
    do stuff
    if stop_condition: #keyboard interrupt ?
    keepgoing.clear()

    for worker in threading.enumerate():
    if worker.isAlive():
    worker.join()

    FULL DISCLAIMER: I am NOT a threading expert and have limited
    experience, so if this doesn't work, you are on your own !
     
    imageguy, Apr 5, 2009
    #4
  5. Aahz Guest

    In article <>,
    imageguy <> wrote:
    >On Apr 4, 10:43=A0pm, wrote:
    >>
    >> The issue that I'm having is...I don't know how to kill this app in
    >> window.

    >
    >I am not an expert either, however, I think the standard practice is
    >to use and threading.Event to communicate with the threads.


    Nope, Eric had the right idea the first time: use Queue.Queue. And the
    reason is precisely because you're not a thread expert; using Queue is
    the easiest way to avoid deadlocks and other threading problems.

    For more info, see the slides from my thread tutorial:
    http://pythoncraft.com/OSCON2001/
    --
    Aahz () <*> http://www.pythoncraft.com/

    "Debugging is twice as hard as writing the code in the first place.
    Therefore, if you write the code as cleverly as possible, you are, by
    definition, not smart enough to debug it." --Brian W. Kernighan
     
    Aahz, Apr 5, 2009
    #5
  6. On Sun, 05 Apr 2009 12:54:45 +0200, Francesco Bochicchio
    <> declaimed the following in
    gmane.comp.python.general:


    > If yor threads are not set as 'deamons' using Thread.setDaemon method,
    > then your main program at its termination should call Thread.join for
    > each of the thread spawned, otherwise the whole process will not quit.
    >

    .join() alone won't do anything but wait for the thread itself to
    quit -- which means one still has to signal the threads to commit
    suicide.

    1) Use of a global scope "SHUTDOWN = true" which each thread tests at
    some point...

    2) Use of an instance SHUTDOWN, if subclassing from threading, to
    permit targeted suicide requests to specific threads, again the thread
    must test at some point...

    3) Send some special marker via a QUEUE (which may be shared by
    multiple worker threads -- so they must PUT the marker back on the queue
    before committing hari kari so the next worker discovers it; or thread
    specific queues for targeted shutdowns). Threads must be in a state to
    read from the queue.

    Main program, after sending the signal (1, 2, or 3, above) may then
    wait on .join() operations for the particular threads.

    If the thread has the capability to become blocked on some operation
    (say a socket read without timeout), none of these solutions will work.
    That just leaves setting the threads daemonic at the start -- which
    indicates the runtime may brutally kill them when the main program
    exits.


    --
    Wulfraed Dennis Lee Bieber KD6MOG

    HTTP://wlfraed.home.netcom.com/
    (Bestiaria Support Staff: )
    HTTP://www.bestiaria.com/
     
    Dennis Lee Bieber, Apr 5, 2009
    #6
  7. imageguy Guest


    > For more info, see the slides from my thread tutorial:http://pythoncraft.com/OSCON2001/
    > --
    > Aahz ()           <*>        http://www.pythoncraft.com/
    >

    Aahz, thanks for this reference and link to your presentation. At the
    risk of highjacking the OP's question, I am bit confused as to how
    using an Event would cause a deadlock. You presentation outlines
    Events, but doesn't elaborate on the specifics or usage.

    In threading.Event python 2.5 docs say;
    "This is one of the simplest mechanisms for communication between
    threads: one thread signals an event and other threads wait for it. "

    Again, I have limited experience, however, in my reading of the
    threading manual and review examples, Events were specifically design
    to be a thread safe way to communicate a 'state' to running threads ?
    In the OP's example 'do stuff' was open to wide interpretation,
    however, if within the thread's main 'while' loop the tread checks to
    see if the 'keepgoing' Event.isSet(), in what scenario would this
    create deadlock ?

    Thanks for your patience. I thought I had grasped the basics of using
    threads/threading, so hope I can learn more.

    g.
     
    imageguy, Apr 6, 2009
    #7
  8. On Sun, 5 Apr 2009 17:27:15 -0700 (PDT), imageguy
    <> declaimed the following in
    gmane.comp.python.general:

    > In threading.Event python 2.5 docs say;
    > "This is one of the simplest mechanisms for communication between
    > threads: one thread signals an event and other threads wait for it. "
    >
    > Again, I have limited experience, however, in my reading of the
    > threading manual and review examples, Events were specifically design
    > to be a thread safe way to communicate a 'state' to running threads ?
    > In the OP's example 'do stuff' was open to wide interpretation,
    > however, if within the thread's main 'while' loop the tread checks to
    > see if the 'keepgoing' Event.isSet(), in what scenario would this
    > create deadlock ?
    >

    If you are going to perform a CPU intensive polling loop, there is
    no sense in using the Event system in the first place... Just create a
    globally accessible flag and set it to true when you want to signal the
    threads (or false if you don't want to use the negation "while not
    flagged: do next processing step")

    Event is optimized for the case wherein threads can WAIT (block) on
    the Event object.
    --
    Wulfraed Dennis Lee Bieber KD6MOG

    HTTP://wlfraed.home.netcom.com/
    (Bestiaria Support Staff: )
    HTTP://www.bestiaria.com/
     
    Dennis Lee Bieber, Apr 6, 2009
    #8
  9. Guest

    On Apr 5, 11:07 pm, Dennis Lee Bieber <> wrote:
    > On Sun, 5 Apr 2009 17:27:15 -0700 (PDT), imageguy
    > <> declaimed the following in
    > gmane.comp.python.general:
    >
    > > In threading.Event python 2.5 docs say;
    > > "This is one of the simplest mechanisms for communication between
    > > threads: one thread signals an event and other threads wait for it. "

    >
    > > Again, I have limited experience, however, in my reading of the
    > > threading manual and review examples, Events were specifically design
    > > to be a thread safe way to communicate a 'state' to running threads ?
    > > In the OP's example 'do stuff' was open to wide interpretation,
    > > however, if within the thread's main 'while' loop the tread checks to
    > > see if the 'keepgoing' Event.isSet(), in what scenario would this
    > > create deadlock ?

    >
    >         If you are going to perform a CPU intensive polling loop, there is
    > no sense in using the Event system in the first place... Just create a
    > globally accessible flag and set it to true when you want to signal the
    > threads (or false if you don't want to use the negation "while not
    > flagged: do next processing step")
    >
    >         Event is optimized for the case wherein threads can WAIT (block) on
    > the Event object.
    > --
    >         Wulfraed        Dennis Lee Bieber               KD6MOG
    >                    
    >                 HTTP://wlfraed.home.netcom.com/
    >         (Bestiaria Support Staff:               )
    >                 HTTP://www.bestiaria.com/



    Well it turns out my problem was with queues not with threads. I had
    a self.die prop in my thread object that defaults to FALSE and that I
    set to true when i wanted the thread to die. then my loop would be
    while not die: It seemed pretty simple so I didn't know why it was
    failing. What I didn't know, because I'm quite new to python, is that
    queue.get was blocking. So my producer thread why dying immediately
    but my worker threads were all blocking on their queue.gets. So they
    were never falling off the loop. I changed it to queue.get_nowait()
    and added a queue.empty exception and everything worked as expected.

    So I thought I knew what was going on and that I was having a really
    esoteric problem when i was actually having a pretty boring problem I
    didn't recognize.

    Thanks everybody for the help!
     
    , Apr 6, 2009
    #9
  10. Guest

    On Apr 5, 9:48 pm, Dennis Lee Bieber <> wrote:
    > On Sun, 05 Apr 2009 12:54:45 +0200, Francesco Bochicchio
    > <> declaimed the following in
    > gmane.comp.python.general:
    >
    > > If yor threads are not set as 'deamons' using Thread.setDaemon method,
    > > then your main program at its termination should call Thread.join for
    > > each of the thread spawned, otherwise the whole process will not quit.

    >
    >         .join() alone won't do anything but wait for the thread itself to
    > quit -- which means one still has to signal the threads to commit
    > suicide.
    >


    Yes. Mine was an 'additional suggestion' to the ones that the OP
    already received.
    I guests that was not clear enough ...

    ... follows a nice explanation on methods to stop threads that I was
    too lazy to write ...

    >
    >         If the thread has the capability to become blocked on some operation
    > (say a socket read without timeout), none of these solutions will work.
    > That just leaves setting the threads daemonic at the start -- which
    > indicates the runtime may brutally kill them when the main program
    > exits.
    >


    You know, this bugger me a little. I know that killing threads is hard
    in any language
    (I'm facing now the issue in a C++ program I'm writing at work),
    expecially doing in
    a platform-independent way, but Java managed to do it. Now python is
    in many ways an
    higher level language than Java, but when it comes to threading I feel
    it lacks something.
    I know that often it is not too hard to avoid blocking reads, and you
    can always use subprocesses that
    with the new multiprocessing module are almost as easy as threads, but
    still ...

    Ciao
    -----
    FB
     
    , Apr 6, 2009
    #10
  11. Guest

    On 6 Apr, 05:25, wrote:
    > On Apr 5, 11:07 pm, Dennis Lee Bieber <> wrote:
    >
    >
    >
    >
    >
    > > On Sun, 5 Apr 2009 17:27:15 -0700 (PDT), imageguy
    > > <> declaimed the following in
    > > gmane.comp.python.general:

    >
    > > > In threading.Event python 2.5 docs say;
    > > > "This is one of the simplest mechanisms for communication between
    > > > threads: one thread signals an event and other threads wait for it. "

    >
    > > > Again, I have limited experience, however, in my reading of the
    > > > threading manual and review examples, Events were specifically design
    > > > to be a thread safe way to communicate a 'state' to running threads ?
    > > > In the OP's example 'do stuff' was open to wide interpretation,
    > > > however, if within the thread's main 'while' loop the tread checks to
    > > > see if the 'keepgoing' Event.isSet(), in what scenario would this
    > > > create deadlock ?

    >
    > >         If you are going to perform a CPU intensive polling loop, there is
    > > no sense in using the Event system in the first place... Just create a
    > > globally accessible flag and set it to true when you want to signal the
    > > threads (or false if you don't want to use the negation "while not
    > > flagged: do next processing step")

    >
    > >         Event is optimized for the case wherein threads can WAIT (block) on
    > > the Event object.
    > > --
    > >         Wulfraed        Dennis Lee Bieber               KD6MOG
    > >                    
    > >                 HTTP://wlfraed.home.netcom.com/
    > >         (Bestiaria Support Staff:               )
    > >                 HTTP://www.bestiaria.com/

    >
    > Well it turns out my problem was with queues not with threads.  I had
    > a self.die prop in my thread object that defaults to FALSE and that I
    > set to true when i wanted the thread to die.  then my loop would be
    > while not die:  It seemed pretty simple so I didn't know why it was
    > failing.  What I didn't know, because I'm quite new to python, is that
    > queue.get was blocking.  So my producer thread why dying immediately
    > but my worker threads were all blocking on their queue.gets.  So they
    > were never falling off the loop.  I changed it to queue.get_nowait()
    > and added a queue.empty exception and everything worked as expected.
    >
    > So I thought I knew what was going on and that I was having a really
    > esoteric problem when i was actually having a pretty boring problem I
    > didn't recognize.
    >
    > Thanks everybody for the help!>


    I've gone through that also, when I started with python threads :)
    Be aware that using get_nowait may lead to your thread using too much
    CPU in checking a queue often empty. I tend to use Queue.get with a
    timeout, smaller enough to keep the thread responsive but large enough
    not
    to waste CPU in too-frequent checks.

    Ciao
    -----
    FB
     
    , Apr 6, 2009
    #11
  12. Guest

    On Apr 6, 3:45 am, wrote:
    > On 6 Apr, 05:25, wrote:
    >
    >
    >
    > > On Apr 5, 11:07 pm, Dennis Lee Bieber <> wrote:

    >
    > > > On Sun, 5 Apr 2009 17:27:15 -0700 (PDT), imageguy
    > > > <> declaimed the following in
    > > > gmane.comp.python.general:

    >
    > > > > In threading.Event python 2.5 docs say;
    > > > > "This is one of the simplest mechanisms for communication between
    > > > > threads: one thread signals an event and other threads wait for it. "

    >
    > > > > Again, I have limited experience, however, in my reading of the
    > > > > threading manual and review examples, Events were specifically design
    > > > > to be a thread safe way to communicate a 'state' to running threads ?
    > > > > In the OP's example 'do stuff' was open to wide interpretation,
    > > > > however, if within the thread's main 'while' loop the tread checks to
    > > > > see if the 'keepgoing' Event.isSet(), in what scenario would this
    > > > > create deadlock ?

    >
    > > >         If you are going to perform a CPU intensive polling loop, there is
    > > > no sense in using the Event system in the first place... Just create a
    > > > globally accessible flag and set it to true when you want to signal the
    > > > threads (or false if you don't want to use the negation "while not
    > > > flagged: do next processing step")

    >
    > > >         Event is optimized for the case wherein threads can WAIT (block) on
    > > > the Event object.
    > > > --
    > > >         Wulfraed        Dennis Lee Bieber               KD6MOG
    > > >                    
    > > >                 HTTP://wlfraed.home.netcom.com/
    > > >         (Bestiaria Support Staff:               )
    > > >                 HTTP://www.bestiaria.com/

    >
    > > Well it turns out my problem was with queues not with threads.  I had
    > > a self.die prop in my thread object that defaults to FALSE and that I
    > > set to true when i wanted the thread to die.  then my loop would be
    > > while not die:  It seemed pretty simple so I didn't know why it was
    > > failing.  What I didn't know, because I'm quite new to python, is that
    > > queue.get was blocking.  So my producer thread why dying immediately
    > > but my worker threads were all blocking on their queue.gets.  So they
    > > were never falling off the loop.  I changed it to queue.get_nowait()
    > > and added a queue.empty exception and everything worked as expected.

    >
    > > So I thought I knew what was going on and that I was having a really
    > > esoteric problem when i was actually having a pretty boring problem I
    > > didn't recognize.

    >
    > > Thanks everybody for the help!>

    >
    > I've gone through that also, when I started with python threads :)
    > Be aware that using get_nowait may lead to your thread using too much
    > CPU in checking a queue often empty. I tend to use  Queue.get with a
    > timeout, smaller enough to keep the thread responsive but large enough
    > not
    > to waste CPU in too-frequent checks.
    >
    > Ciao
    > -----
    > FB


    Ok thanks - good to now. I'm trying to throttle it with a 1/2 sec
    sleep statement in the loop but I might just have the "main" loop toss
    some stuff on that queue as another solution. I'm still kicking it
    around
     
    , Apr 6, 2009
    #12
  13. Aahz Guest

    In article <>,
    <> wrote:
    >
    >I know that killing threads is hard in any language (I'm facing now
    >the issue in a C++ program I'm writing at work), expecially doing in a
    >platform-independent way, but Java managed to do it.


    That's not my understanding:

    http://www.roseindia.net/javatutorials/shutting_down_threads_cleanly.shtml
    --
    Aahz () <*> http://www.pythoncraft.com/

    "...string iteration isn't about treating strings as sequences of strings,
    it's about treating strings as sequences of characters. The fact that
    characters are also strings is the reason we have problems, but characters
    are strings for other good reasons." --Aahz
     
    Aahz, Apr 6, 2009
    #13
  14. Aahz Guest

    In article <>,
    imageguy <> wrote:
    >Aahz:
    >>
    >>For more info, see the slides from my thread tutorial:
    >>http://pythoncraft.com/OSCON2001/

    >
    >Aahz, thanks for this reference and link to your presentation. At the
    >risk of highjacking the OP's question, I am bit confused as to how
    >using an Event would cause a deadlock. You presentation outlines
    >Events, but doesn't elaborate on the specifics or usage.


    >In threading.Event python 2.5 docs say;
    >"This is one of the simplest mechanisms for communication between
    >threads: one thread signals an event and other threads wait for it. "
    >
    >Again, I have limited experience, however, in my reading of the
    >threading manual and review examples, Events were specifically design
    >to be a thread safe way to communicate a 'state' to running threads ?
    >In the OP's example 'do stuff' was open to wide interpretation,
    >however, if within the thread's main 'while' loop the tread checks to
    >see if the 'keepgoing' Event.isSet(), in what scenario would this
    >create deadlock ?


    Consider two threads trying to pass information to each other. You need
    two Event objects; unless you are very careful, they can both block on
    waiting for each other. It's easier if Queue is the only construct you
    use, it can do everything that the lower-level constructs can do, without
    needing to figure out the little gotchas -- and you'll need a Queue
    eventually, so just learn that one. It's all about simplifying.
    --
    Aahz () <*> http://www.pythoncraft.com/

    "...string iteration isn't about treating strings as sequences of strings,
    it's about treating strings as sequences of characters. The fact that
    characters are also strings is the reason we have problems, but characters
    are strings for other good reasons." --Aahz
     
    Aahz, Apr 6, 2009
    #14
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. cppaddict
    Replies:
    20
    Views:
    820
    Chris Uppal
    Aug 11, 2004
  2. Don Garrett

    Killing threads during development.

    Don Garrett, Jun 5, 2005, in forum: Python
    Replies:
    0
    Views:
    306
    Don Garrett
    Jun 5, 2005
  3. jimzat

    Killing subservient threads

    jimzat, Feb 20, 2009, in forum: Python
    Replies:
    6
    Views:
    326
  4. Chris Angelico
    Replies:
    2
    Views:
    277
    Antoon Pardon
    Sep 21, 2011
  5. x1
    Replies:
    15
    Views:
    265
Loading...

Share This Page