Exception Handling in Python 3

[Steve Holden]

I was somewhat surprised to discover that Python 3 no longer allows an
exception to be raised in an except clause (or rather that it reports it
as a separate exception that occurred during the handling of the first).

So the following code:
.... val = d['nosuch']
.... except:
.... raise AttributeError("No attribute 'nosuch'")
....

Give the traceback I expected and wanted in Python 2:

Traceback (most recent call last):
File "<stdin>", line 4, in <module>
AttributeError: No attribute 'nosuch'

but in Python 3.1 the traceback looks like this:

Traceback (most recent call last):
File "<stdin>", line 2, in <module>
KeyError: 'nosuch'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "<stdin>", line 4, in <module>
AttributeError: No attribute 'nosuch'

Modifying the code a little allows me to change the error message, but
not much else:
.... val = d['nosuch']
.... except KeyError as e:
.... raise AttributeError("No attribute 'nosuch'") from e
....
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
KeyError: 'nosuch'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):

In a class's __getattr__() method this means that instead of being able
to say

try:
value = _attrs[name]
except KeyError:
raise AttributeError ...

I am forced to write

if name not in _attrs:
raise AttributeError ...
value = _attrs[name]

which requires an unnecessary second lookup on the attribute name. What
is the correct paradigm for this situation?

regards
Steve

 

[Lawrence D'Oliveiro]

I was somewhat surprised to discover that Python 3 no longer allows an
exception to be raised in an except clause (or rather that it reports it
as a separate exception that occurred during the handling of the first).

So what exactly is the problem? Exceptions are so easy to get wrong, it’s
just trying to report more info in a complicated situation to help you track
down the problem. Why is that bad?

In a class's __getattr__() method this means that instead of being able
to say

try:
value = _attrs[name]
except KeyError:
raise AttributeError ...

I am forced to write

if name not in _attrs:
raise AttributeError ...
value = _attrs[name]

I don’t see why. Presumably if you caught the exception in an outer try-
except clause, you would pick up your AttributeError, not the KeyError,
right? Which is what you want, right?

 

[Martin v. Loewis]

Am 24.10.2010 07:01, schrieb Steve Holden:

I was somewhat surprised to discover that Python 3 no longer allows an
exception to be raised in an except clause (or rather that it reports it
as a separate exception that occurred during the handling of the first).

I think you are misinterpreting what you are seeing. The exception being
raised actually *is* an attribute error, and it actually is the
attribute error that gets reported. It's only that reporting an
exception that has a __context__ first reports the context, then reports
the actual exception.

You may now wonder whether it is possible to set __context__ to None
somehow. See PEP 3134:

Open Issue: Suppressing Context

As written, this PEP makes it impossible to suppress '__context__',
since setting exc.__context__ to None in an 'except' or 'finally'
clause will only result in it being set again when exc is raised.

Regards,
Martin

 

[Steve Holden]

Am 24.10.2010 07:01, schrieb Steve Holden:

I think you are misinterpreting what you are seeing. The exception being
raised actually *is* an attribute error, and it actually is the
attribute error that gets reported. It's only that reporting an
exception that has a __context__ first reports the context, then reports
the actual exception.

I don't believe I *am* misinterpreting it. The fact of the matter is
that the context is irrelevant to the user, and there should be some way
to suppress it to avoid over-complicating the traceback.

This behavior is quite reasonable during testing, but I would prefer to
exclude an explicit raise directly in the except handler since that is
hardly to be construed as accidental (whereas an exception in a function
called in the handler perhaps *should* be reported).

You may now wonder whether it is possible to set __context__ to None
somehow. See PEP 3134:

Open Issue: Suppressing Context

As written, this PEP makes it impossible to suppress '__context__',
since setting exc.__context__ to None in an 'except' or 'finally'
clause will only result in it being set again when exc is raised.

I have already read that. Peter Otten has separately explained how to
suppress the behavior using sys.excepthook, which appears to be a
halfway satisfactory solution.

regards
Steve

 

[Steve Holden]

I was somewhat surprised to discover that Python 3 no longer allows an
exception to be raised in an except clause (or rather that it reports it
as a separate exception that occurred during the handling of the first).

So what exactly is the problem? Exceptions are so easy to get wrong, it’s
just trying to report more info in a complicated situation to help you track
down the problem. Why is that bad?

In a class's __getattr__() method this means that instead of being able
to say

try:
value = _attrs[name]
except KeyError:
raise AttributeError ...

I am forced to write

if name not in _attrs:
raise AttributeError ...
value = _attrs[name]

I don’t see why. Presumably if you caught the exception in an outer try-
except clause, you would pick up your AttributeError, not the KeyError,
right? Which is what you want, right?

Yes, *if the exception is caught* then it doesn't make any difference.
If the exception creates a traceback, however, I maintain that the
additional information is confusing to the consumer (while helpful to
the debugger of the consumed code).

I don't want people to think this is a big deal, however. It was just an
"eh?" that I thought must mean I was missing some way of suppressing the
additional traceback. Peter Otten has already provided a solution using
sys.except_hook().

regards
Steve

 

[Lie Ryan]

I was somewhat surprised to discover that Python 3 no longer allows an
exception to be raised in an except clause (or rather that it reports it
as a separate exception that occurred during the handling of the first).

FYI, Java has a similar behavior. In Java, however, after a certain
length, some of the older exceptions will be suppressed and will only
print message informing that there are more exceptions above it.

 

[Lawrence D'Oliveiro]

Yes, *if the exception is caught* then it doesn't make any difference.
If the exception creates a traceback, however, I maintain that the
additional information is confusing to the consumer (while helpful to
the debugger of the consumed code).

Who needs the information more?

 

[Antoine Pitrou]

You may now wonder whether it is possible to set __context__ to None
somehow. See PEP 3134:

Open Issue: Suppressing Context

As written, this PEP makes it impossible to suppress '__context__',
since setting exc.__context__ to None in an 'except' or 'finally'
clause will only result in it being set again when exc is raised.

It is not easily discoverable, but it is possible to suppress
__context__ by using a bare re-raise afterwards:
.... try: 1/0
.... except ZeroDivisionError: raise KeyError
.... except BaseException as e:
.... e.__context__ = None
.... raise
....
Traceback (most recent call last):
File "<stdin>", line 3, in <module>
KeyError


Regards

Antoine.

 

[Chris Rebert]

It is not easily discoverable, but it is possible to suppress
__context__ by using a bare re-raise afterwards:

...   try: 1/0
...   except ZeroDivisionError: raise KeyError
... except BaseException as e:
...   e.__context__ = None
...   raise
...
Traceback (most recent call last):
 File "<stdin>", line 3, in <module>
KeyError

So, how/why does that work?

Cheers,
Chris

 

[Antoine Pitrou]

So, how/why does that work?

A bare "raise" simply re-raises the currently known exception without
changing anything (neither the traceback nor the context).

This __context__ problem is mostly theoretical, anyway. If you want to
present exceptions to users in a different way, you can write a
catch-all except clause at the root of your program and use the
traceback module to do what you want.

Regards

Antoine.

 

[Martin v. Loewis]

It is not easily discoverable, but it is possible to suppress

__context__ by using a bare re-raise afterwards:

I see. I'd wrap this like this:

def raise_no_context(e):
try:
raise e
except:
e.__context__=None
raise

d = {}
try:
val = d['nosuch']
except KeyError:
raise_no_context(AttributeError("No attribute 'nosuch'"))

The downside of this is that the innermost frame will be
raise_no_context, but I suppose that's ok because even the
next-inner frame already reveals implementation details that
developers have learned to ignore.

Regards,
Martin

 

[rantingrick]

I don't want people to think this is a big deal, however.

Nonsense, this IS a big deal. (and Steve grow a spine already!) I was
not even aware of this issue until you brought it up -- although i
will admit your choice of title is completely misleading!

This new exception handling behavior is completely ridiculous and the
only thing more atrocious than the new exception handling BUG you
uncovered is the "hoop jumping" garbage people have come up with to
make the "new" work as the tried and tested "old".

I am the programmer, and when i say to my interpretor "show this
exception instead of that exception" i expect my interpretor to do
exactly as i say or risk total annihilation!! I don't want my
interpreter "interpreting" my intentions and then doing what it
"thinks" is best for me. I have a wife already, i don't need a virtual
one! We are opening a Pandora's box of mediocrity and confusion when
we start down such a slippery slope as this.

Let me enlighten you fellows... If have i ever learned anything about
programming, i can tell you one rule should always be counted on above
all rules -- that the computer will do exactly what you tell it to do
NOTHING more, and NOTHING less. End of story!

 

[MRAB]

I don't believe I *am* misinterpreting it. The fact of the matter is
that the context is irrelevant to the user, and there should be some way
to suppress it to avoid over-complicating the traceback.

This behavior is quite reasonable during testing, but I would prefer to
exclude an explicit raise directly in the except handler since that is
hardly to be construed as accidental (whereas an exception in a function
called in the handler perhaps *should* be reported).

I have already read that. Peter Otten has separately explained how to
suppress the behavior using sys.excepthook, which appears to be a
halfway satisfactory solution.

Suggestion: an explicit 'raise' in the exception handler excludes the
context, but if you want to include it then 'raise with'. For example:

# Exclude the context
try:
command_dict[command]()
except KeyError:
raise CommandError("Unknown command")

# Include the context
try:
command_dict[command]()
except KeyError:
raise with CommandError("Unknown command")

 

[Ethan Furman]

I don't believe I *am* misinterpreting it. The fact of the matter is
that the context is irrelevant to the user, and there should be some way
to suppress it to avoid over-complicating the traceback.

This behavior is quite reasonable during testing, but I would prefer to
exclude an explicit raise directly in the except handler since that is
hardly to be construed as accidental (whereas an exception in a function
called in the handler perhaps *should* be reported).

I have already read that. Peter Otten has separately explained how to
suppress the behavior using sys.excepthook, which appears to be a
halfway satisfactory solution.

Suggestion: an explicit 'raise' in the exception handler excludes the
context, but if you want to include it then 'raise with'. For example:

# Exclude the context
try:
command_dict[command]()
except KeyError:
raise CommandError("Unknown command")

# Include the context
try:
command_dict[command]()
except KeyError:
raise with CommandError("Unknown command")

+1

Presumably, this would also keep the context if an actual error occured.

~Ethan~

 

[Lawrence D'Oliveiro]

If you want to present exceptions to users in a different way ...

sys.stderr.write \
(
"Traceback (most recent call last):\n"
...
"AttributeError: blah blah blah ...\n"
)

 

[John Nagle]

Yes, *if the exception is caught* then it doesn't make any difference.
If the exception creates a traceback, however, I maintain that the
additional information is confusing to the consumer (while helpful to
the debugger of the consumed code).

If an exception propagates all the way out to the top level and
all the user gets is a system traceback, the program isn't very
good. Really. If you're concerned about the display format of
Python tracebacks seen by end users, you have bigger problems
with your code.

If it's a server-side program, you need to catch
exceptions and log them somewhere, traceback and all. If it's
a client-side GUI program, letting an exception unwind all the
way out of the program loses whatever the user was doing.

It's usually desirable to at least catch EnviromentError near
the top level of your program. If some external problem causes a
program abort, the user should get an error message, not a traceback.
If it's a program bug, you can let the traceback unwind, displaying
information that should be sent in with a bug report.

John Nagle

 

[John Ladasky]

I am the programmer, and when i say to my interpretor "show this
exception instead of that exception" i expect my interpretor to do
exactly as i say or risk total annihilation!! I don't want my
interpreter "interpreting" my intentions and then doing what it
"thinks" is best for me. I have a wife already, i don't need a virtual
one!

OK, Ranting Rick, that was funny, and worthy of your name. :^)