worker thread catching exceptions and putting them in queue

Discussion in 'Python' started by Paul Sijben, Mar 5, 2007.

  1. Paul Sijben

    Paul Sijben Guest

    All,

    in a worker thread setup that communicates via queues is it possible to
    catch exceptions raised by the worker executed, put them in an object
    and send them over the queue to another thread where the exception is
    raised in that scope?

    considering that an exception is an object I feel it ought to be
    possible, however I do not see how to go about it.

    does anyone have a pointer towards the solution?

    Paul
     
    Paul Sijben, Mar 5, 2007
    #1
    1. Advertising

  2. Paul Sijben wrote:

    > All,
    >
    > in a worker thread setup that communicates via queues is it possible to
    > catch exceptions raised by the worker executed, put them in an object
    > and send them over the queue to another thread where the exception is
    > raised in that scope?
    >
    > considering that an exception is an object I feel it ought to be
    > possible, however I do not see how to go about it.
    >
    > does anyone have a pointer towards the solution?



    Just raise the exception object found in the queue. Only make sure it _is_
    an exception, as you can raise everything. So in your queue-putting-code
    you might consider discriminating between the two cases, like this:


    while True:
    try:
    result = 1, work()
    except:
    result = 2, sys.exc_info()[1]
    queue.put(result)

    -------
    while True:
    kind, result = queue.get()
    if kind == 1:
    do(result)
    elif kind == 2:
    raise result


    Diez
     
    Diez B. Roggisch, Mar 5, 2007
    #2
    1. Advertising

  3. Paul Sijben

    Stargaming Guest

    Paul Sijben schrieb:
    > All,
    >
    > in a worker thread setup that communicates via queues is it possible to
    > catch exceptions raised by the worker executed, put them in an object
    > and send them over the queue to another thread where the exception is
    > raised in that scope?
    >
    > considering that an exception is an object I feel it ought to be
    > possible, however I do not see how to go about it.
    >
    > does anyone have a pointer towards the solution?
    >
    > Paul


    You're right, even exceptions are objects in Python. For further
    studies, read http://docs.python.org/lib/module-exceptions.html

    You can catch an exception like this:
    try:
    worker.do_some_work_that_may_raise_an_exception()
    except Exception, e:
    # the first argument is the type of error you want to handle
    # it is Exception here, the baseclass of all computation exceptions
    # the second argument is the variable (name) where you want to save
    # the specific exception raised to
    # it's 'e' here, a common shortcut for exception
    exception_handler.handle(e)
    # notice that you can pass e around as you like

    For further information on exceptions and how to handle them, read
    chapter 8 of the tutorial, especially starting from 8.3:
    http://docs.python.org/tut/node10.html#SECTION0010300000000000000000

    P.S. I don't know if what I told still applies to Python 3.0 -- a lot of
    changes are upcoming related to exception raising and handling.
     
    Stargaming, Mar 5, 2007
    #3
  4. Paul Sijben

    Paul Sijben Guest

    Stargaming & Diez,

    thanks very much!

    Stargaming wrote:
    > Paul Sijben schrieb:
    >> All,
    >>
    >> in a worker thread setup that communicates via queues is it possible to
    >> catch exceptions raised by the worker executed, put them in an object
    >> and send them over the queue to another thread where the exception is
    >> raised in that scope?
    >>
    >> considering that an exception is an object I feel it ought to be
    >> possible, however I do not see how to go about it.
    >>
    >> does anyone have a pointer towards the solution?
    >>
    >> Paul

    >
    > You're right, even exceptions are objects in Python. For further
    > studies, read http://docs.python.org/lib/module-exceptions.html
    >
    > You can catch an exception like this:
    > try:
    > worker.do_some_work_that_may_raise_an_exception()
    > except Exception, e:
    > # the first argument is the type of error you want to handle
    > # it is Exception here, the baseclass of all computation exceptions
    > # the second argument is the variable (name) where you want to save
    > # the specific exception raised to
    > # it's 'e' here, a common shortcut for exception
    > exception_handler.handle(e)
    > # notice that you can pass e around as you like
    >
    > For further information on exceptions and how to handle them, read
    > chapter 8 of the tutorial, especially starting from 8.3:
    > http://docs.python.org/tut/node10.html#SECTION0010300000000000000000
    >
    > P.S. I don't know if what I told still applies to Python 3.0 -- a lot of
    > changes are upcoming related to exception raising and handling.
     
    Paul Sijben, Mar 6, 2007
    #4
  5. Paul Sijben

    Aahz Guest

    In article <45ec4936$0$323$4all.nl>,
    Paul Sijben <> wrote:
    >
    >in a worker thread setup that communicates via queues is it possible to
    >catch exceptions raised by the worker executed, put them in an object
    >and send them over the queue to another thread where the exception is
    >raised in that scope?
    >
    >considering that an exception is an object I feel it ought to be
    >possible, however I do not see how to go about it.


    One caution: because exceptions keep stack frames alive, you can have
    garbage collection problems. Make sure to del the exception when you're
    done with it.
    --
    Aahz () <*> http://www.pythoncraft.com/

    "I disrespectfully agree." --SJM
     
    Aahz, Mar 8, 2007
    #5
  6. En Thu, 08 Mar 2007 13:31:14 -0300, Aahz <> escribió:

    > In article <45ec4936$0$323$4all.nl>,
    > Paul Sijben <> wrote:
    >>
    >> in a worker thread setup that communicates via queues is it possible to
    >> catch exceptions raised by the worker executed, put them in an object
    >> and send them over the queue to another thread where the exception is
    >> raised in that scope?

    >
    > One caution: because exceptions keep stack frames alive, you can have
    > garbage collection problems. Make sure to del the exception when you're
    > done with it.


    Note that it's the traceback who keeps stack frames alive, not the
    exception itself (at least on the current Python versions, might change in
    the future). If no fancy processing of the traceback is needed, maybe
    converting it to string (using the traceback module) before posting it to
    the queue is enough.

    --
    Gabriel Genellina
     
    Gabriel Genellina, Mar 8, 2007
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. alex
    Replies:
    1
    Views:
    654
    Lau Lei Cheong
    Feb 4, 2005
  2. Russell Warren

    Is Queue.Queue.queue.clear() thread-safe?

    Russell Warren, Jun 22, 2006, in forum: Python
    Replies:
    4
    Views:
    691
    Russell Warren
    Jun 27, 2006
  3. Kris
    Replies:
    0
    Views:
    494
  4. Steve Tuckner

    Catching thread exceptions

    Steve Tuckner, Aug 8, 2003, in forum: Ruby
    Replies:
    2
    Views:
    154
    Brian Candler
    Aug 8, 2003
  5. anselm
    Replies:
    2
    Views:
    178
    anselm
    Mar 22, 2007
Loading...

Share This Page