Semantics of propagated exceptions

[Thomas Lotze]

Hi,

I wonder how to solve the following problem the most pythonic way:

Suppose you have a function f which, as part of its protocol, raises some
standard exception E under certain, well-defined circumstances. Suppose
further that f calls other functions which may also raise E. How to best
distinguish whether an exception E raised by f has the meaning defined by
the protocol or just comes from details of the implementation?

As an example, let's inherit from dict and replace __getitem__. It is
supposed to raise a KeyError if an item is not found in the mapping. But
what if it does some magic to use default values:

def __getitem__(self, key):
if key in self:
return self[key]
defaults = foobar["default"]
return defaults[key]

If "default" is not in foobar, a KeyError is raised by that lookup and
propagates to the calling code. However, the problem is not "key can't be
found" but "I'm too stupid to find out whether key can be found". In a web
context where key identifies the resource requested, this might make the
difference between a 404 "Not found" and a 500 "Internal server error"
response.

Several solutions come to mind, neither of which I'm satisfied with:

- f might catch E exceptions from the implementation and raise some other
error in their stead, maybe with an appropriate message or treating the
traceback in some helpful way. This destroys the original exception.

- f might catch and re-raise E exceptions, setting some flag on them that
identifies them as protocol exceptions or not. This requires calling code
to know about the flag.

- Calling code might guess whether the exception comes from some inner
working of f from how deep in the calling stack the exception originated.
Obviously, this will not be easy or not even work at all if f calls
related functions which might also raise E with the protocol semantics.
This requires calling code to do some magic but keeps f from having to
catch and raise exceptions all over the place.

Some gut feeling tells me the first option is preferrable, but I'ld like
to read your opinions and maybe other alternatives.

 

[Lawrence D'Oliveiro]

Suppose you have a function f which, as part of its protocol, raises some
standard exception E under certain, well-defined circumstances. Suppose
further that f calls other functions which may also raise E. How to best
distinguish whether an exception E raised by f has the meaning defined by
the protocol or just comes from details of the implementation?

You can't. All the information necessary to identify the exception must be
in the exception object itself. If you want to distinguish between
exceptions raised by F and those raised by other functions called by F,
then they must be different exceptions.

 

[Marc 'BlackJack' Rintsch]

I wonder how to solve the following problem the most pythonic way:

Suppose you have a function f which, as part of its protocol, raises some
standard exception E under certain, well-defined circumstances. Suppose
further that f calls other functions which may also raise E. How to best
distinguish whether an exception E raised by f has the meaning defined by
the protocol or just comes from details of the implementation?

[…]

Several solutions come to mind, neither of which I'm satisfied with:

- f might catch E exceptions from the implementation and raise some other
error in their stead, maybe with an appropriate message or treating the
traceback in some helpful way. This destroys the original exception.

This is the way to go I think. After all it's not an ordinary `E` but one
that has a special, well defined meaning if raised by `f` itself according
to your protocol.

Put the original exception as an attribute to the new one and/or subclass
the new exception type from `E`. This way calling code can choose to to
handle both types as `E` if the distinction is not important.

Ciao,
Marc 'BlackJack' Rintsch

 

[Steve Holden]

Hi,

I wonder how to solve the following problem the most pythonic way:

Suppose you have a function f which, as part of its protocol, raises some
standard exception E under certain, well-defined circumstances. Suppose
further that f calls other functions which may also raise E. How to best
distinguish whether an exception E raised by f has the meaning defined by
the protocol or just comes from details of the implementation?

As an example, let's inherit from dict and replace __getitem__. It is
supposed to raise a KeyError if an item is not found in the mapping. But
what if it does some magic to use default values:

def __getitem__(self, key):
if key in self:
return self[key]
defaults = foobar["default"]
return defaults[key]

If "default" is not in foobar, a KeyError is raised by that lookup and
propagates to the calling code. However, the problem is not "key can't be
found" but "I'm too stupid to find out whether key can be found". In a web
context where key identifies the resource requested, this might make the
difference between a 404 "Not found" and a 500 "Internal server error"
response.

The obvious response is to make the implementation less stupid by
replacing the last statement with

try:
return defaults[key]
except KeyError:
raise KeyError("No default for key value %s" % key)

Several solutions come to mind, neither of which I'm satisfied with:

- f might catch E exceptions from the implementation and raise some other
error in their stead, maybe with an appropriate message or treating the
traceback in some helpful way. This destroys the original exception.

My "solution", of course, takes this approach. The error can readily be
recognised as different from the standard KeyError message. The original
exception is worthless in this context as your new type is clearly
supposed to provide a default for any absent key value. The handling
code should thus treat the key's absence as some sort of implementation
error.

- f might catch and re-raise E exceptions, setting some flag on them that
identifies them as protocol exceptions or not. This requires calling code
to know about the flag.

This doesn't seem particularly helpful in your case. Since the purpose
of the flag is to distinguish the actual exception thrown by your type
from the exception thrown during method execution it would seem more
direct to use a different exception altogether.

- Calling code might guess whether the exception comes from some inner
working of f from how deep in the calling stack the exception originated.
Obviously, this will not be easy or not even work at all if f calls
related functions which might also raise E with the protocol semantics.
This requires calling code to do some magic but keeps f from having to
catch and raise exceptions all over the place.

The problem with this approach is that the more complex your
implementations become, the more complex the client (calling) code has
to be. It will probably also require serious changes to the calling code
as implementation details change, which doesn't give very good isolation.

Some gut feeling tells me the first option is preferrable, but I'ld like
to read your opinions and maybe other alternatives.

Just my two cents worth.

regards
Steve

 

[Michael J. Fromberger]

Suppose you have a function f which, as part of its protocol, raises some
standard exception E under certain, well-defined circumstances. Suppose
further that f calls other functions which may also raise E. How to best
distinguish whether an exception E raised by f has the meaning defined by
the protocol or just comes from details of the implementation?

My recommendation is to define your own exception type(s), and have f
raise it (them) for errors that are truly generated by f. Or, if you
really want to use one of the existing exception types, then perhaps you
can catch the exceptions from functions called by f, within f itself,
and raise some other error (or suppress the errors, if appropriate).

Cheers,
-M

 

[Thomas Lotze]

Sorry for not answering such a long time. It's because my question
originated from a discussion within our company which moved out of focus
shortly after I posted, and over waiting for some response from them
before replying here, I forgot about it.

My "solution", of course, takes this approach.

Good to see that my "gut feeling" as to the most pythonic approach seems
to coincide with the answers I've received ;o)