pyprocessing and exceptions

[garyrob]

Hi,

We're still using Python 2.5 so this question is about the
pyprocessing module rather than the multiprocessing module, but I'm
guessing the answer is the same.

I tend to use the Pool() object to create slave processes. If
something goes wrong in the slave, an exception is raised there, which
is then raised in the master or parent process, which is great.

The problem is that if the master aborts due to the exception, it
doesn't show the usual stack trace info for the slave, which would
show (among other things) the line number the error occurred on.
Instead, it shows the line in the master where work was sent to the
slave (such as a call to pool.map()).

I'm wondering what the recommended way is to write code that will
reveal what went wrong in the slave. One obvious possibility is to
have functions that are invoked in the slave incorporate their own
exception handling that prints a stack trace. But I'd rather handle
this issue in the master, rather than have to handle it in every
function in the slave module that the master may invoke.

Is there a way to do that? If not, what's the recommended approach?

Thanks,
Gary

 

[Jesse Noller]

Hi,

We're still using Python 2.5 so this question is about the
pyprocessing module rather than the multiprocessing module, but I'm
guessing the answer is the same.

I tend to use the Pool() object to create slave processes. If
something goes wrong in the slave, an exception is raised there, which
is then raised in the master or parent process, which is great.

The problem is that if the master aborts due to the exception, it
doesn't show the usual stack trace info for the slave, which would
show (among other things) the line number the error occurred on.
Instead, it shows the line in the master where work was sent to the
slave (such as a call to pool.map()).

I'm wondering what the recommended way is to write code that will
reveal what went wrong in the slave. One obvious possibility is to
have functions that are invoked in the slave incorporate their own
exception handling that prints a stack trace. But I'd rather handle
this issue in the master, rather than have to handle it in every
function in the slave module that the master may invoke.

Is there a way to do that? If not, what's the recommended approach?

Thanks,
Gary

You should handle the exception in the child.

Also, multiprocessing was backported to python 2.5 and earlier.

http://pypi.python.org/pypi/multiprocessing/

jesse