How to get a raised exception from other thread

Discussion in 'Python' started by dcrespo, Oct 14, 2005.

  1. dcrespo

    dcrespo Guest

    Hi all,

    How can I get a raised exception from other thread that is in an
    imported module?

    For example:

    ---------------
    programA.py
    ---------------

    import programB

    thread = programB.MakeThread()
    thread.start()

    ---------------
    programB.py
    ---------------
    import threading, time

    class SomeException(Exception):
    pass

    class MakeThread(threading.Thread):
    def __init__(self):
    threading.Thread.__init__(self)

    def run(self):
    i = 0
    while 1:
    print i
    i += 1
    time.sleep(1) #wait a second to continue
    if i>10:
    raise SomeException()


    Thanks

    Daniel
     
    dcrespo, Oct 14, 2005
    #1
    1. Advertising

  2. dcrespo

    Jeremy Moles Guest

    On non-Windows system there are a ton of ways to do it--this is almost a
    whole field unto itself. :) (D-BUS, fifos, sockets, shmfs, etc.) In
    Windows, I wouldn't have a clue.

    I guess this is a hard question to answer without a bit more
    information. :)

    On Fri, 2005-10-14 at 14:45 -0700, dcrespo wrote:
    > Hi all,
    >
    > How can I get a raised exception from other thread that is in an
    > imported module?
    >
    > For example:
    >
    > ---------------
    > programA.py
    > ---------------
    >
    > import programB
    >
    > thread = programB.MakeThread()
    > thread.start()
    >
    > ---------------
    > programB.py
    > ---------------
    > import threading, time
    >
    > class SomeException(Exception):
    > pass
    >
    > class MakeThread(threading.Thread):
    > def __init__(self):
    > threading.Thread.__init__(self)
    >
    > def run(self):
    > i = 0
    > while 1:
    > print i
    > i += 1
    > time.sleep(1) #wait a second to continue
    > if i>10:
    > raise SomeException()
    >
    >
    > Thanks
    >
    > Daniel
    >
     
    Jeremy Moles, Oct 14, 2005
    #2
    1. Advertising

  3. dcrespo

    dcrespo Guest

    Thanks for your answer, but I think that created thread in python
    should create a thread either on windows and linux.

    Can you give me Python example of how to do what I want to do? Thanks
     
    dcrespo, Oct 15, 2005
    #3
  4. dcrespo

    Peter Hansen Guest

    dcrespo wrote:
    > How can I get a raised exception from other thread that is in an
    > imported module?


    Define what "get" means for your purposes. It appears that you mean you
    want to catch the exception, but in the thread which launched the other
    thread in the first place. If that's true, please show where you would
    expect to catch this exception, given that when you start the thread,
    the main thread continues running and might even finish before the other
    thread finishes.

    > thread = programB.MakeThread()
    > thread.start()

    ....
    # later code which might have completed by the time the thread finishes
    ....

    Are you looking, for example, for some kind of
    thread.waitUntilCompletionAndReRaiseExceptions() method?

    -Peter
     
    Peter Hansen, Oct 15, 2005
    #4
  5. dcrespo

    Guest

    I also need an answer to this problem. I'm using windows. Throwing an
    exception in thread B from thread A using a callback function.

    The function runs, but the thread never actually catches the exception.
    The try/except block is in the run() method (over-riden from the
    Thread class)
     
    , Oct 16, 2005
    #5
  6. dcrespo

    Peter Hansen Guest

    wrote:
    > I also need an answer to this problem.


    What _is_ the problem? We're still waiting for a clear description from
    the OP as to what the problem really is. How would you describe what
    _your_ problem is?

    > I'm using windows. Throwing an
    > exception in thread B from thread A using a callback function.
    >
    > The function runs, but the thread never actually catches the exception.
    > The try/except block is in the run() method (over-riden from the
    > Thread class)


    Rather than force us all to guess what you are doing wrong, maybe it
    would be better if you posted a small amount of code that shows the problem.

    Exceptions raised in threads can definitely be caught in the run()
    method if the code is written correctly, so that is not in itself an issue.

    -Peter
     
    Peter Hansen, Oct 16, 2005
    #6
  7. dcrespo

    Guest

    Here's a dumbed down version of what i'm doing:

    import time
    import threading

    class threader(threading.Thread):
    def __init__(self):
    threading.Thread.__init__(self)
    pass

    def run(self):
    try:
    while 1:
    time.sleep(5)
    except SystemExit:
    print "Got Exit Message in thread"

    def killMe(self):
    raise SystemExit



    thread1 = threader()
    thread2 = threader()

    thread1.start()
    thread2.start()

    time.sleep(5)

    try:
    print "Killing thread 1"
    thread1.killMe()
    print "killing thread 2"
    thread2.killMe()
    except SystemExit:
    print "Got exceptin in main thread"



    The exception is not propogated to the threads I spawned, but instead
    comes back in the main thread.
     
    , Oct 16, 2005
    #7
  8. dcrespo

    Guest

    Nevermind. I found a better solution. I used shared memory to create
    a keep-alive flag. I then use the select function with a specified
    timeout, and recheck the keep-alive flag after each timeout.

    Thanx for all the input.
     
    , Oct 16, 2005
    #8
  9. On 15 Oct 2005 20:54:33 -0700, declaimed the
    following in comp.lang.python:


    > try:
    > print "Killing thread 1"
    > thread1.killMe()
    > print "killing thread 2"
    > thread2.killMe()
    > except SystemExit:
    > print "Got exceptin in main thread"
    >
    >
    >
    > The exception is not propogated to the threads I spawned, but instead
    > comes back in the main thread.


    No surprise there... The function invocation is being made in the
    CONTEXT of the main thread...
    --
    > ============================================================== <
    > | Wulfraed Dennis Lee Bieber KD6MOG <
    > | Bestiaria Support Staff <
    > ============================================================== <
    > Home Page: <http://www.dm.net/~wulfraed/> <
    > Overflow Page: <http://wlfraed.home.netcom.com/> <
     
    Dennis Lee Bieber, Oct 16, 2005
    #9
  10. dcrespo

    Klaas Guest

    In article <>,
    wrote:

    > Nevermind. I found a better solution. I used shared memory to create
    > a keep-alive flag. I then use the select function with a specified
    > timeout, and recheck the keep-alive flag after each timeout.
    >
    > Thanx for all the input.


    How about a threading.Event?

    -Mike
     
    Klaas, Oct 16, 2005
    #10
  11. dcrespo

    Peter Hansen Guest

    wrote:
    > Nevermind. I found a better solution. I used shared memory to create
    > a keep-alive flag. I then use the select function with a specified
    > timeout, and recheck the keep-alive flag after each timeout.


    As Dennis points out, your original attempt was destined to fail because
    you were calling the method from the main thread, not the one you wanted
    to kill. Threads don't magically execute any methods that are attached
    to the Thread instance just because they're attached. You have to
    actually call those methods *from* the thread, which means from the
    run() method or from any of the routines it calls (whether they are
    methods on that Thread or not), but it must be done in the context of
    the thread you want to raise exceptions in or it won't work.

    More importantly, you've now described your use case (and I hope that of
    the OP as well, since he hasn't replied yet): killing threads.

    This is an oft-discussed topic here, and searching the archives will
    probably give you lots more answers, but the short answer is you cannot
    kill a thread in Python (except by exiting the entire process). Instead,
    as you've discovered, you must ask it nicely to exit. The nearly
    canonical approach to doing that is as follows:

    class MyThread(threading.Thread):
    def __init__(self, *args, **kwargs):
    threading.Thread.__init__(self, *args, **kwargs)
    self._keepRunning = True

    def stop(self, timeout=None):
    self._keepRunning = False
    # optional: wait for thread to exit
    self.join(timeout=timeout)

    def run(self):
    while self._keepRunning:
    # do stuff here

    Now you can do things like this:

    thread = MyThread()
    thread.start()
    # other stuff...
    thread.stop() # block here until the thread exits

    I hope that clarifies things a little more...

    -Peter
     
    Peter Hansen, Oct 16, 2005
    #11
  12. Peter Hansen wrote:
    > wrote:
    >
    >> Nevermind. I found a better solution. I used shared memory to create
    >> a keep-alive flag. I then use the select function with a specified
    >> timeout, and recheck the keep-alive flag after each timeout.

    >
    >
    > As Dennis points out, your original attempt was destined to fail because
    > you were calling the method from the main thread, not the one you wanted

    <snip>

    A similar problem exists in .NET but they "solved" it by creating an
    Abort method that raises an exception in the thread itself, basically
    doing what you wanted to do.

    However, this leads to all sorts of problem, particular when that
    exception is raised in a finally block in that thread, which could mean
    that the finally block did not fully execute, leading to resource leaks
    or similar problems.

    They pondered a while about constructing non-interruptible blocks of
    code for .NET 2.0 but I'm not entirely sure how that worked out.

    In any case, the best way is to use some kind of signal that the thread
    reacts to, either by having it specifically wait for a signallable
    object (like an event) or just by checking a boolean variable or similar
    for a magic value that means "now is a good time for you to terminate".

    --
    Lasse Vågsæther Karlsen
    http://usinglvkblog.blogspot.com/
    mailto:
    PGP KeyID: 0x2A42A1C2
     
    =?ISO-8859-1?Q?Lasse_V=E5gs=E6ther_Karlsen?=, Oct 16, 2005
    #12
  13. Op 2005-10-14, dcrespo schreef <>:
    > Hi all,
    >
    > How can I get a raised exception from other thread that is in an
    > imported module?
    >
    > For example:


    You could try the following class, it needs ctypes and if the exception
    is raised while in a C-extention, the exception will be delayed until
    the extention is left. Some delay is unavoidable. You may also need
    to fiddle with the length of the sleep at the end of this script
    or the values in the xrange calls in the Continue threads

    ---------------------------- except.py --------------------------------

    import os
    import ctypes
    from time import sleep
    from random import randint


    class TimeOut(Exception):
    pass


    class Alarm(Exception):
    pass

    import threading

    class Xthread(threading.Thread):


    def start(self):
    self.__original_run = self.run
    self.run = self.__run
    threading.Thread.start(self)


    def __run(self):
    self.__thrd_id = threading._get_ident()
    try:
    self.__original_run()
    finally:
    self.run = self.__original_run


    def raize(self, excpt):

    Nr = ctypes.pythonapi.PyThreadState_SetAsyncExc(self.__thrd_id, ctypes.py_object(excpt))
    while Nr > 1:
    ctypes.pythonapi.PyThreadState_SetAsyncExc(self.__thrd_id, None)
    sleep(0.1)
    Nr = ctypes.pythonapi.PyThreadState_SetAsyncExc(self.__thrd_id, ctypes.py_object(excpt))

    def alarm(self, tm):

    alrm = threading.Timer(tm, self.raize, (TimeOut,))
    alrm.start()
    return alrm

    class Continue(Xthread):

    def run(self):

    self.id = os.getpid()
    print self.id, "Begin"
    i = 0
    try:
    for _ in xrange(randint(0,20)):
    for e in xrange(4 * 100000):
    i = i + e
    print self.id, "Finished"
    except Alarm:
    print self.id, "Interupted"


    lst = [Continue() for _ in xrange(10)]

    for T in lst:
    T.start()

    try:
    sleep(15)
    finally:
    for T in lst:
    T.raize(Alarm)
     
    Antoon Pardon, Oct 17, 2005
    #13
  14. <> wrote:

    > Nevermind. I found a better solution. I used shared memory to create
    > a keep-alive flag. I then use the select function with a specified
    > timeout, and recheck the keep-alive flag after each timeout.


    Definitely a better architecture. Anyway, one supported way for a
    thread to raise an exception in a different thread is function
    thread.interrupt_main(), which raises a KeyboardInterrupt in the *main*
    thread (the one thread that's running at the beginning of your program).

    There's also a supported, documented function to raise any given
    exception in any existing thread, but it's deliberately NOT directly
    exposed to Python code -- you need a few lines of C-coded extension (or
    pyrex, ctypes, etc, etc) to get at the functionality. This small but
    non-null amount of "attrition" was deliberately put there to avoid
    casual overuse of a facility intended only to help in very peculiar
    cases (essentially in debuggers &c, where the thread's code may be buggy
    and fail to check a keep-alive flag correctly...!-).


    Alex
     
    Alex Martelli, Oct 17, 2005
    #14
  15. Op 2005-10-17, Alex Martelli schreef <>:
    ><> wrote:
    >
    >> Nevermind. I found a better solution. I used shared memory to create
    >> a keep-alive flag. I then use the select function with a specified
    >> timeout, and recheck the keep-alive flag after each timeout.

    >
    > Definitely a better architecture. Anyway, one supported way for a
    > thread to raise an exception in a different thread is function
    > thread.interrupt_main(), which raises a KeyboardInterrupt in the *main*
    > thread (the one thread that's running at the beginning of your program).
    >
    > There's also a supported, documented function to raise any given
    > exception in any existing thread, but it's deliberately NOT directly
    > exposed to Python code -- you need a few lines of C-coded extension (or
    > pyrex, ctypes, etc, etc) to get at the functionality. This small but
    > non-null amount of "attrition" was deliberately put there to avoid
    > casual overuse of a facility intended only to help in very peculiar
    > cases (essentially in debuggers &c, where the thread's code may be buggy
    > and fail to check a keep-alive flag correctly...!-).


    I find this rather arrogant. It is not up to the developers of python
    to decide how the users of python will use the language. If someone
    has use of specific functionality he shouldn't be stopped because his
    use is outside the intentions of the developers.

    Just suppose you are writing a chess program. You let the program
    "think" while it is the users move, but this thinking has to be
    interrupted and some cleanup has to be made after the users move
    has made a lot of this thinking obsolete by his move. Such code
    could be mixture of loops and recursion that makes use of a simple
    flag to end it all, unpractical. Use of an exception raised from
    somewhere else would IMO in this case be an acceptable choice.

    Why should the coder of this software have to go through this
    deliberate set up attrition, to get at this functionality, just
    because it wasn't intented to be used in such a way by the
    developers?

    As far as I know, pyrex and ctypes weren't intended to get
    at the Python/C api. But they didn't create extra hurdles
    for those who could use it that way.

    --
    Antoon Pardon
     
    Antoon Pardon, Oct 17, 2005
    #15
  16. dcrespo

    Steve Holden Guest

    Antoon Pardon wrote:
    > Op 2005-10-17, Alex Martelli schreef <>:
    >
    >><> wrote:
    >>
    >>
    >>>Nevermind. I found a better solution. I used shared memory to create
    >>>a keep-alive flag. I then use the select function with a specified
    >>>timeout, and recheck the keep-alive flag after each timeout.

    >>
    >>Definitely a better architecture. Anyway, one supported way for a
    >>thread to raise an exception in a different thread is function
    >>thread.interrupt_main(), which raises a KeyboardInterrupt in the *main*
    >>thread (the one thread that's running at the beginning of your program).
    >>
    >>There's also a supported, documented function to raise any given
    >>exception in any existing thread, but it's deliberately NOT directly
    >>exposed to Python code -- you need a few lines of C-coded extension (or
    >>pyrex, ctypes, etc, etc) to get at the functionality. This small but
    >>non-null amount of "attrition" was deliberately put there to avoid
    >>casual overuse of a facility intended only to help in very peculiar
    >>cases (essentially in debuggers &c, where the thread's code may be buggy
    >>and fail to check a keep-alive flag correctly...!-).

    >
    >
    > I find this rather arrogant. It is not up to the developers of python
    > to decide how the users of python will use the language. If someone
    > has use of specific functionality he shouldn't be stopped because his
    > use is outside the intentions of the developers.
    >

    The developers have to support (at no cost) what the users do, so if
    they want to make use of certain features inaccessible to users who are
    likely to go wrong then that's up to them.

    > Just suppose you are writing a chess program. You let the program
    > "think" while it is the users move, but this thinking has to be
    > interrupted and some cleanup has to be made after the users move
    > has made a lot of this thinking obsolete by his move. Such code
    > could be mixture of loops and recursion that makes use of a simple
    > flag to end it all, unpractical. Use of an exception raised from
    > somewhere else would IMO in this case be an acceptable choice.
    >

    In *your* opinion, possibly. In my opinion it would seem to make more
    sense to have the worker thread code examine a shared flag and raise an
    exception at some convenient point in its calculation cycle. Otherwise
    you have to write the worker thread to be capable of handling
    asynchronous signals, which is a notoriously difficult task. I presume
    this is why Alex commented that he thought the OP's "better solution"
    was "definitely a better architecture."

    > Why should the coder of this software have to go through this
    > deliberate set up attrition, to get at this functionality, just
    > because it wasn't intented to be used in such a way by the
    > developers?
    >

    Because otherwise people who know no better will use the feature for
    purposes where it's not the best way to achieve the required
    functionality, leading to yet more endless discussions about why "it
    doesn't work". Asynchronous signalling between threads is an accident
    waiting to happen in the hands of an inexperienced programmer.

    > As far as I know, pyrex and ctypes weren't intended to get
    > at the Python/C api. But they didn't create extra hurdles
    > for those who could use it that way.
    >

    This seems like a complete non sequitur to me. Let's stick to the point.

    regards
    Steve
    --
    Steve Holden +44 150 684 7255 +1 800 494 3119
    Holden Web LLC www.holdenweb.com
    PyCon TX 2006 www.python.org/pycon/
     
    Steve Holden, Oct 17, 2005
    #16
  17. dcrespo

    Paul Rubin Guest

    Steve Holden <> writes:
    > Otherwise you have to write the worker thread to be capable of
    > handling asynchronous signals, which is a notoriously difficult task.


    Doing it properly needs a language extension.

    http://www.cs.williams.edu/~freund/papers/02-lwl2.ps
     
    Paul Rubin, Oct 17, 2005
    #17
  18. Op 2005-10-17, Steve Holden schreef <>:
    > Antoon Pardon wrote:
    >> Op 2005-10-17, Alex Martelli schreef <>:
    >>
    >>><> wrote:
    >>>
    >>>
    >>>>Nevermind. I found a better solution. I used shared memory to create
    >>>>a keep-alive flag. I then use the select function with a specified
    >>>>timeout, and recheck the keep-alive flag after each timeout.
    >>>
    >>>Definitely a better architecture. Anyway, one supported way for a
    >>>thread to raise an exception in a different thread is function
    >>>thread.interrupt_main(), which raises a KeyboardInterrupt in the *main*
    >>>thread (the one thread that's running at the beginning of your program).
    >>>
    >>>There's also a supported, documented function to raise any given
    >>>exception in any existing thread, but it's deliberately NOT directly
    >>>exposed to Python code -- you need a few lines of C-coded extension (or
    >>>pyrex, ctypes, etc, etc) to get at the functionality. This small but
    >>>non-null amount of "attrition" was deliberately put there to avoid
    >>>casual overuse of a facility intended only to help in very peculiar
    >>>cases (essentially in debuggers &c, where the thread's code may be buggy
    >>>and fail to check a keep-alive flag correctly...!-).

    >>
    >>
    >> I find this rather arrogant. It is not up to the developers of python
    >> to decide how the users of python will use the language. If someone
    >> has use of specific functionality he shouldn't be stopped because his
    >> use is outside the intentions of the developers.
    >>

    > The developers have to support (at no cost) what the users do,


    Only in so far that correct programs behave correctly. The developers
    are not obligated to correct buggy programs. They have every right to
    say that if a programmor can handle certain concepts he should seek
    help with that concept elsewhere.

    > so if
    > they want to make use of certain features inaccessible to users who are
    > likely to go wrong then that's up to them.


    But the way it is made inaccessible has nothing to do with knowing
    how to use the feature. That you can dable with C-extententions, pyrex
    or ctypes says nothing about your ability to handle threads that
    raise exceptions in each other.

    >> Just suppose you are writing a chess program. You let the program
    >> "think" while it is the users move, but this thinking has to be
    >> interrupted and some cleanup has to be made after the users move
    >> has made a lot of this thinking obsolete by his move. Such code
    >> could be mixture of loops and recursion that makes use of a simple
    >> flag to end it all, unpractical. Use of an exception raised from
    >> somewhere else would IMO in this case be an acceptable choice.
    >>

    > In *your* opinion, possibly. In my opinion it would seem to make more
    > sense to have the worker thread code examine a shared flag and raise an
    > exception at some convenient point in its calculation cycle.


    What it there is no such convenient point? What if you have a number
    of differnt routines that recursively call each other each with its
    own number of nested loops. If you want something that is a bit
    responsive you will have to put a check for this flag in any loop.

    > Otherwise
    > you have to write the worker thread to be capable of handling
    > asynchronous signals, which is a notoriously difficult task. I presume
    > this is why Alex commented that he thought the OP's "better solution"
    > was "definitely a better architecture."


    IMO the flag is the better architecture in this case because python
    can't interrupt a tread with an exception while that thread is in
    C-code.

    >> Why should the coder of this software have to go through this
    >> deliberate set up attrition, to get at this functionality, just
    >> because it wasn't intented to be used in such a way by the
    >> developers?
    >>

    > Because otherwise people who know no better will use the feature for
    > purposes where it's not the best way to achieve the required
    > functionality, leading to yet more endless discussions about why "it
    > doesn't work".


    Then don't react to those questions. A lot of questions about threads
    are already answered as follows: "Don't use threads", "use twised" or
    "If you have to use threads, use Queues". IMO there is nothing wrong
    with telling people that if they want to use certain features it is
    expected they know what thet are doing.

    > Asynchronous signalling between threads is an accident
    > waiting to happen in the hands of an inexperienced programmer.


    So? Do we want python to be limited to features that can't
    cause problems in the hands of an inexperienced programmer?

    --
    Antoon Pardon
     
    Antoon Pardon, Oct 17, 2005
    #18
  19. Steve Holden wrote:
    <snip>
    >> Why should the coder of this software have to go through this
    >> deliberate set up attrition, to get at this functionality, just
    >> because it wasn't intented to be used in such a way by the
    >> developers?
    >>

    > Because otherwise people who know no better will use the feature for
    > purposes where it's not the best way to achieve the required
    > functionality, leading to yet more endless discussions about why "it
    > doesn't work". Asynchronous signalling between threads is an accident
    > waiting to happen in the hands of an inexperienced programmer.


    I can't agree with that opinion. There's tons of features of existing
    programming languages and systems, Python included, that is pretty much
    guaranteed to be misused, and badly at that. We can't remove those just
    on the off chance someone might start a thread on a newsgroup about why
    this is or isn't the best way to do things. If we do that then I guess
    list comprehension is out the window because there's tons of posts (from
    me included) that get this wrong.

    The point of a programming language is not to dumb down the environment
    to a point where it is impossible to write bad code. Instead it's about
    empowering the programmers to be able to accomplish their tasks.

    In any case, this exception-in-another-thread problem has inherent
    issues which must be solved, among others:

    - C extension code can't be interrupted
    - an exception thrown at the wrong time in a finally/except block might
    cause more problems than it intends to solve

    So until a good implementation exists, there shouldn't be any point in
    actually discussing the motives of the programmers who wishes to use the
    feature.

    <snip>

    --
    Lasse Vågsæther Karlsen
    http://usinglvkblog.blogspot.com/
    mailto:
    PGP KeyID: 0x2A42A1C2
     
    =?ISO-8859-1?Q?Lasse_V=E5gs=E6ther_Karlsen?=, Oct 17, 2005
    #19
  20. dcrespo

    dcrespo Guest

    Hi Peter. Sorry for my delay. I wasn't in home the weekend.

    > Define what "get" means for your purposes. It appears that you mean you
    > want to catch the exception, but in the thread which launched the other
    > thread in the first place. If that's true, please show where you would
    > expect to catch this exception, given that when you start the thread,
    > the main thread continues running and might even finish before the other
    > thread finishes.


    In my above example, I want to catch in ProgramA the raised exception
    in ProgramB. Please, look at my example one more time.

    Thanks.
     
    dcrespo, Oct 17, 2005
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Joe Wong
    Replies:
    1
    Views:
    465
    Ype Kingma
    May 14, 2004
  2. Lonnie Princehouse
    Replies:
    8
    Views:
    540
    Lonnie Princehouse
    Oct 2, 2004
  3. pegazik

    Exception raised in wrong thread?

    pegazik, Sep 19, 2005, in forum: Python
    Replies:
    1
    Views:
    275
    Piet van Oostrum
    Sep 20, 2005
  4. northof40
    Replies:
    3
    Views:
    220
    Lie Ryan
    Feb 20, 2010
  5. Ziliang Chen
    Replies:
    3
    Views:
    156
    Ziliang Chen
    Feb 22, 2013
Loading...

Share This Page