How to get a raised exception from other thread

D

dcrespo

Hi all,

How can I get a raised exception from other thread that is in an
imported module?

For example:

---------------
programA.py
---------------

import programB

thread = programB.MakeThread()
thread.start()

---------------
programB.py
---------------
import threading, time

class SomeException(Exception):
pass

class MakeThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)

def run(self):
i = 0
while 1:
print i
i += 1
time.sleep(1) #wait a second to continue
if i>10:
raise SomeException()


Thanks

Daniel
 
J

Jeremy Moles

On non-Windows system there are a ton of ways to do it--this is almost a
whole field unto itself. :) (D-BUS, fifos, sockets, shmfs, etc.) In
Windows, I wouldn't have a clue.

I guess this is a hard question to answer without a bit more
information. :)
 
D

dcrespo

Thanks for your answer, but I think that created thread in python
should create a thread either on windows and linux.

Can you give me Python example of how to do what I want to do? Thanks
 
P

Peter Hansen

dcrespo said:
How can I get a raised exception from other thread that is in an
imported module?

Define what "get" means for your purposes. It appears that you mean you
want to catch the exception, but in the thread which launched the other
thread in the first place. If that's true, please show where you would
expect to catch this exception, given that when you start the thread,
the main thread continues running and might even finish before the other
thread finishes.
thread = programB.MakeThread()
thread.start()
....
# later code which might have completed by the time the thread finishes
....

Are you looking, for example, for some kind of
thread.waitUntilCompletionAndReRaiseExceptions() method?

-Peter
 
T

themightydoyle

I also need an answer to this problem. I'm using windows. Throwing an
exception in thread B from thread A using a callback function.

The function runs, but the thread never actually catches the exception.
The try/except block is in the run() method (over-riden from the
Thread class)
 
P

Peter Hansen

I also need an answer to this problem.

What _is_ the problem? We're still waiting for a clear description from
the OP as to what the problem really is. How would you describe what
_your_ problem is?
> I'm using windows. Throwing an
exception in thread B from thread A using a callback function.

The function runs, but the thread never actually catches the exception.
The try/except block is in the run() method (over-riden from the
Thread class)

Rather than force us all to guess what you are doing wrong, maybe it
would be better if you posted a small amount of code that shows the problem.

Exceptions raised in threads can definitely be caught in the run()
method if the code is written correctly, so that is not in itself an issue.

-Peter
 
T

themightydoyle

Here's a dumbed down version of what i'm doing:

import time
import threading

class threader(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
pass

def run(self):
try:
while 1:
time.sleep(5)
except SystemExit:
print "Got Exit Message in thread"

def killMe(self):
raise SystemExit



thread1 = threader()
thread2 = threader()

thread1.start()
thread2.start()

time.sleep(5)

try:
print "Killing thread 1"
thread1.killMe()
print "killing thread 2"
thread2.killMe()
except SystemExit:
print "Got exceptin in main thread"



The exception is not propogated to the threads I spawned, but instead
comes back in the main thread.
 
T

themightydoyle

Nevermind. I found a better solution. I used shared memory to create
a keep-alive flag. I then use the select function with a specified
timeout, and recheck the keep-alive flag after each timeout.

Thanx for all the input.
 
D

Dennis Lee Bieber

try:
print "Killing thread 1"
thread1.killMe()
print "killing thread 2"
thread2.killMe()
except SystemExit:
print "Got exceptin in main thread"



The exception is not propogated to the threads I spawned, but instead
comes back in the main thread.

No surprise there... The function invocation is being made in the
CONTEXT of the main thread...
--
 
K

Klaas

Nevermind. I found a better solution. I used shared memory to create
a keep-alive flag. I then use the select function with a specified
timeout, and recheck the keep-alive flag after each timeout.

Thanx for all the input.

How about a threading.Event?

-Mike
 
P

Peter Hansen

Nevermind. I found a better solution. I used shared memory to create
a keep-alive flag. I then use the select function with a specified
timeout, and recheck the keep-alive flag after each timeout.

As Dennis points out, your original attempt was destined to fail because
you were calling the method from the main thread, not the one you wanted
to kill. Threads don't magically execute any methods that are attached
to the Thread instance just because they're attached. You have to
actually call those methods *from* the thread, which means from the
run() method or from any of the routines it calls (whether they are
methods on that Thread or not), but it must be done in the context of
the thread you want to raise exceptions in or it won't work.

More importantly, you've now described your use case (and I hope that of
the OP as well, since he hasn't replied yet): killing threads.

This is an oft-discussed topic here, and searching the archives will
probably give you lots more answers, but the short answer is you cannot
kill a thread in Python (except by exiting the entire process). Instead,
as you've discovered, you must ask it nicely to exit. The nearly
canonical approach to doing that is as follows:

class MyThread(threading.Thread):
def __init__(self, *args, **kwargs):
threading.Thread.__init__(self, *args, **kwargs)
self._keepRunning = True

def stop(self, timeout=None):
self._keepRunning = False
# optional: wait for thread to exit
self.join(timeout=timeout)

def run(self):
while self._keepRunning:
# do stuff here

Now you can do things like this:

thread = MyThread()
thread.start()
# other stuff...
thread.stop() # block here until the thread exits

I hope that clarifies things a little more...

-Peter
 
?

=?ISO-8859-1?Q?Lasse_V=E5gs=E6ther_Karlsen?=

Peter said:
As Dennis points out, your original attempt was destined to fail because
you were calling the method from the main thread, not the one you wanted
<snip>

A similar problem exists in .NET but they "solved" it by creating an
Abort method that raises an exception in the thread itself, basically
doing what you wanted to do.

However, this leads to all sorts of problem, particular when that
exception is raised in a finally block in that thread, which could mean
that the finally block did not fully execute, leading to resource leaks
or similar problems.

They pondered a while about constructing non-interruptible blocks of
code for .NET 2.0 but I'm not entirely sure how that worked out.

In any case, the best way is to use some kind of signal that the thread
reacts to, either by having it specifically wait for a signallable
object (like an event) or just by checking a boolean variable or similar
for a magic value that means "now is a good time for you to terminate".
 
A

Antoon Pardon

Op 2005-10-14 said:
Hi all,

How can I get a raised exception from other thread that is in an
imported module?

For example:

You could try the following class, it needs ctypes and if the exception
is raised while in a C-extention, the exception will be delayed until
the extention is left. Some delay is unavoidable. You may also need
to fiddle with the length of the sleep at the end of this script
or the values in the xrange calls in the Continue threads

---------------------------- except.py --------------------------------

import os
import ctypes
from time import sleep
from random import randint


class TimeOut(Exception):
pass


class Alarm(Exception):
pass

import threading

class Xthread(threading.Thread):


def start(self):
self.__original_run = self.run
self.run = self.__run
threading.Thread.start(self)


def __run(self):
self.__thrd_id = threading._get_ident()
try:
self.__original_run()
finally:
self.run = self.__original_run


def raize(self, excpt):

Nr = ctypes.pythonapi.PyThreadState_SetAsyncExc(self.__thrd_id, ctypes.py_object(excpt))
while Nr > 1:
ctypes.pythonapi.PyThreadState_SetAsyncExc(self.__thrd_id, None)
sleep(0.1)
Nr = ctypes.pythonapi.PyThreadState_SetAsyncExc(self.__thrd_id, ctypes.py_object(excpt))

def alarm(self, tm):

alrm = threading.Timer(tm, self.raize, (TimeOut,))
alrm.start()
return alrm

class Continue(Xthread):

def run(self):

self.id = os.getpid()
print self.id, "Begin"
i = 0
try:
for _ in xrange(randint(0,20)):
for e in xrange(4 * 100000):
i = i + e
print self.id, "Finished"
except Alarm:
print self.id, "Interupted"


lst = [Continue() for _ in xrange(10)]

for T in lst:
T.start()

try:
sleep(15)
finally:
for T in lst:
T.raize(Alarm)
 
A

Alex Martelli

Nevermind. I found a better solution. I used shared memory to create
a keep-alive flag. I then use the select function with a specified
timeout, and recheck the keep-alive flag after each timeout.

Definitely a better architecture. Anyway, one supported way for a
thread to raise an exception in a different thread is function
thread.interrupt_main(), which raises a KeyboardInterrupt in the *main*
thread (the one thread that's running at the beginning of your program).

There's also a supported, documented function to raise any given
exception in any existing thread, but it's deliberately NOT directly
exposed to Python code -- you need a few lines of C-coded extension (or
pyrex, ctypes, etc, etc) to get at the functionality. This small but
non-null amount of "attrition" was deliberately put there to avoid
casual overuse of a facility intended only to help in very peculiar
cases (essentially in debuggers &c, where the thread's code may be buggy
and fail to check a keep-alive flag correctly...!-).


Alex
 
A

Antoon Pardon

Op 2005-10-17 said:
Definitely a better architecture. Anyway, one supported way for a
thread to raise an exception in a different thread is function
thread.interrupt_main(), which raises a KeyboardInterrupt in the *main*
thread (the one thread that's running at the beginning of your program).

There's also a supported, documented function to raise any given
exception in any existing thread, but it's deliberately NOT directly
exposed to Python code -- you need a few lines of C-coded extension (or
pyrex, ctypes, etc, etc) to get at the functionality. This small but
non-null amount of "attrition" was deliberately put there to avoid
casual overuse of a facility intended only to help in very peculiar
cases (essentially in debuggers &c, where the thread's code may be buggy
and fail to check a keep-alive flag correctly...!-).

I find this rather arrogant. It is not up to the developers of python
to decide how the users of python will use the language. If someone
has use of specific functionality he shouldn't be stopped because his
use is outside the intentions of the developers.

Just suppose you are writing a chess program. You let the program
"think" while it is the users move, but this thinking has to be
interrupted and some cleanup has to be made after the users move
has made a lot of this thinking obsolete by his move. Such code
could be mixture of loops and recursion that makes use of a simple
flag to end it all, unpractical. Use of an exception raised from
somewhere else would IMO in this case be an acceptable choice.

Why should the coder of this software have to go through this
deliberate set up attrition, to get at this functionality, just
because it wasn't intented to be used in such a way by the
developers?

As far as I know, pyrex and ctypes weren't intended to get
at the Python/C api. But they didn't create extra hurdles
for those who could use it that way.
 
S

Steve Holden

Antoon said:
Definitely a better architecture. Anyway, one supported way for a
thread to raise an exception in a different thread is function
thread.interrupt_main(), which raises a KeyboardInterrupt in the *main*
thread (the one thread that's running at the beginning of your program).

There's also a supported, documented function to raise any given
exception in any existing thread, but it's deliberately NOT directly
exposed to Python code -- you need a few lines of C-coded extension (or
pyrex, ctypes, etc, etc) to get at the functionality. This small but
non-null amount of "attrition" was deliberately put there to avoid
casual overuse of a facility intended only to help in very peculiar
cases (essentially in debuggers &c, where the thread's code may be buggy
and fail to check a keep-alive flag correctly...!-).


I find this rather arrogant. It is not up to the developers of python
to decide how the users of python will use the language. If someone
has use of specific functionality he shouldn't be stopped because his
use is outside the intentions of the developers.
[/QUOTE]
The developers have to support (at no cost) what the users do, so if
they want to make use of certain features inaccessible to users who are
likely to go wrong then that's up to them.
Just suppose you are writing a chess program. You let the program
"think" while it is the users move, but this thinking has to be
interrupted and some cleanup has to be made after the users move
has made a lot of this thinking obsolete by his move. Such code
could be mixture of loops and recursion that makes use of a simple
flag to end it all, unpractical. Use of an exception raised from
somewhere else would IMO in this case be an acceptable choice.
In *your* opinion, possibly. In my opinion it would seem to make more
sense to have the worker thread code examine a shared flag and raise an
exception at some convenient point in its calculation cycle. Otherwise
you have to write the worker thread to be capable of handling
asynchronous signals, which is a notoriously difficult task. I presume
this is why Alex commented that he thought the OP's "better solution"
was "definitely a better architecture."
Why should the coder of this software have to go through this
deliberate set up attrition, to get at this functionality, just
because it wasn't intented to be used in such a way by the
developers?
Because otherwise people who know no better will use the feature for
purposes where it's not the best way to achieve the required
functionality, leading to yet more endless discussions about why "it
doesn't work". Asynchronous signalling between threads is an accident
waiting to happen in the hands of an inexperienced programmer.
As far as I know, pyrex and ctypes weren't intended to get
at the Python/C api. But they didn't create extra hurdles
for those who could use it that way.
This seems like a complete non sequitur to me. Let's stick to the point.

regards
Steve
 
A

Antoon Pardon

Op 2005-10-17 said:
The developers have to support (at no cost) what the users do,

Only in so far that correct programs behave correctly. The developers
are not obligated to correct buggy programs. They have every right to
say that if a programmor can handle certain concepts he should seek
help with that concept elsewhere.
so if
they want to make use of certain features inaccessible to users who are
likely to go wrong then that's up to them.

But the way it is made inaccessible has nothing to do with knowing
how to use the feature. That you can dable with C-extententions, pyrex
or ctypes says nothing about your ability to handle threads that
raise exceptions in each other.
In *your* opinion, possibly. In my opinion it would seem to make more
sense to have the worker thread code examine a shared flag and raise an
exception at some convenient point in its calculation cycle.

What it there is no such convenient point? What if you have a number
of differnt routines that recursively call each other each with its
own number of nested loops. If you want something that is a bit
responsive you will have to put a check for this flag in any loop.
Otherwise
you have to write the worker thread to be capable of handling
asynchronous signals, which is a notoriously difficult task. I presume
this is why Alex commented that he thought the OP's "better solution"
was "definitely a better architecture."

IMO the flag is the better architecture in this case because python
can't interrupt a tread with an exception while that thread is in
C-code.
Because otherwise people who know no better will use the feature for
purposes where it's not the best way to achieve the required
functionality, leading to yet more endless discussions about why "it
doesn't work".

Then don't react to those questions. A lot of questions about threads
are already answered as follows: "Don't use threads", "use twised" or
"If you have to use threads, use Queues". IMO there is nothing wrong
with telling people that if they want to use certain features it is
expected they know what thet are doing.
Asynchronous signalling between threads is an accident
waiting to happen in the hands of an inexperienced programmer.

So? Do we want python to be limited to features that can't
cause problems in the hands of an inexperienced programmer?
 
?

=?ISO-8859-1?Q?Lasse_V=E5gs=E6ther_Karlsen?=

Steve Holden wrote:
Because otherwise people who know no better will use the feature for
purposes where it's not the best way to achieve the required
functionality, leading to yet more endless discussions about why "it
doesn't work". Asynchronous signalling between threads is an accident
waiting to happen in the hands of an inexperienced programmer.

I can't agree with that opinion. There's tons of features of existing
programming languages and systems, Python included, that is pretty much
guaranteed to be misused, and badly at that. We can't remove those just
on the off chance someone might start a thread on a newsgroup about why
this is or isn't the best way to do things. If we do that then I guess
list comprehension is out the window because there's tons of posts (from
me included) that get this wrong.

The point of a programming language is not to dumb down the environment
to a point where it is impossible to write bad code. Instead it's about
empowering the programmers to be able to accomplish their tasks.

In any case, this exception-in-another-thread problem has inherent
issues which must be solved, among others:

- C extension code can't be interrupted
- an exception thrown at the wrong time in a finally/except block might
cause more problems than it intends to solve

So until a good implementation exists, there shouldn't be any point in
actually discussing the motives of the programmers who wishes to use the
feature.

<snip>
 
D

dcrespo

Hi Peter. Sorry for my delay. I wasn't in home the weekend.
Define what "get" means for your purposes. It appears that you mean you
want to catch the exception, but in the thread which launched the other
thread in the first place. If that's true, please show where you would
expect to catch this exception, given that when you start the thread,
the main thread continues running and might even finish before the other
thread finishes.

In my above example, I want to catch in ProgramA the raised exception
in ProgramB. Please, look at my example one more time.

Thanks.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,744
Messages
2,569,483
Members
44,901
Latest member
Noble71S45

Latest Threads

Top