Exception as the primary error handling mechanism?

[Steven D'Aprano]

Why can't int('nonnumeric') return None?

It could do that, but it shouldn't, because returning magic values
instead of raising exceptions is usually a bad, bad idea.

(A related question - why can't I just go 'if record = method(): use
(record)'. Why extra lines just to trap and assign the variable before
using it?)

Because that idiom is responsible for probably the most common error in C
of all, at least one of the most common errors. Thank goodness Python
forbids such a dangerous construct.

 

[Steven D'Aprano]

You are saying I, as the programmer, cannot decide what is an error and
what is a pass-thru. The decision is made for me.

Every function decides for you what is an error and what isn't.
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: max expected 1 arguments, got 0


But I wanted it to return (sys.maxint - 7). How DARE the creator of the
language decide that it should be an error instead of returning the
arbitrary result I choose!

Not.

(Yes yes I can write int_or_None(), etc...)

That's right. As a programmer, your job is to program. If a language
doesn't provide a function you want, write your own using the primitives
available to you.

Here's a super easy example:

{ 42: 'forty two' }.get(41, None)

Because I can supply a default, I can decide what is an error and what
is .

You can ALWAYS decide what is an error.


d = {42: 'forty two'}
try:
d[41]
except KeyError:
print "All is good, no problems, dict does not contain 41"
except:
print "WARNING WARNING WARNING!!!"
print "FATAL ERROR: dict contains 41!!!"
sys.exit()


You do so by programming. As a programmer, that's your job.

Now the equivalent in a language that does not enjoy this false "Zen":

{ 42: 'forty two' }[41] # returns None

Suppose you have a dict d supplied from somewhere else. You don't know
what's in it. You do this:

d[41]

and you get a result None. Does this mean that the dict looks like this?

d = {}

or like this?

d = {41: None}

{ 42: 'forty two' }.fetch(41, None) # ibid

In Python, "fetch" is spelled "get".

{ 42: 'forty two' }.fetch(41) # raises an exception

In Python, that would be spelled {42: 'forty two'}[41]

The quicky validation is available if I _request_ it.

When you write d[41] you are requesting it. If you don't want it, use get
instead.

Because that "Zen of Python" is an empty sophistry that forces me to add
a "mere one more line of code" over and over again...

If you're writing that one line of code over and over again, that's a
good sign that you're doing it wrong and should rethink your strategy.

Don't prevent me from using a technique just because others had trouble
with it.

Any language allows and prevents certain techniques, simply by the very
nature of the language. Every language has it's own syntax: you can't
write weakly-typed stack-based concatenative code (like Forth) in Java,
or strongly-typed dynamic object-oriented Python code in Pascal. Every
language forces the programmer to use some features and avoid others.

And if bar() == foo is the superior technique anyway, because the ==
happens in chronological and lexical order after the bar() call.

That makes no sense.

 

[Benjamin Kaplan]

Here's a super easy example:

 { 42: 'forty two' }.get(41, None)

Because I can supply a default, I can decide what is an error and what
is .

Now the equivalent in a language that does not enjoy this false "Zen":

 { 42: 'forty two' }[41]  # returns None
 { 42: 'forty two' }.fetch(41, None)  # ibid
 { 42: 'forty two' }.fetch(41)  # raises an exception

Here's another super easy example.

Should

{42 : 'forty two'}[41]

be treated the same as

{41: None}[41] ?

In cases where None is a valid result, you can't use it to signal failure.

 

[Phlip]

Because that idiom is responsible for probably the most common error in C
of all, at least one of the most common errors. Thank goodness Python
forbids such a dangerous construct.

switching = for == is the "most common error in C"?

I can't tell if you are joking.

 

[Phlip]

{41: None}[41] ?

In cases where None is a valid result, you can't use it to signal failure..

Asked and answered. You change the "sentinel" in .fetch to something
else.

But y'all keep on defending the language making your programming
decisions for you!

 

[Steve Holden]

Phlip wrote:
[...]

Don't prevent me from using a technique just because others had
trouble with it.

I presume you also campaign against anti-lock braking systems (or at
least don't use cars which have them - after all, anyone who knows how
to drive should be able to brake properly, right? And even if they
don't, *you* can, so you shouldn't have to pay the extra. Of course, if
someone else happens to hit you because they *couldn't* drive safely you
will happily give up your life).

And if bar() == foo is the superior technique anyway, because the ==
happens in chronological and lexical order after the bar() call.

That's about the most specious piece of hogwash I've seen this week.
Python doesn't make any guarantees about the evaluation order of =='s
operands (unlike "and" and "or"). That's cargo cult programming right
there. Readability is surely the only reason to decide how to write that
"if", and I suspect most people would choose "if foo == bar()" for that
reason, though the difference is slight.

regards
Steve

 

[Steve Holden]

Phlip wrote:
[...]

Don't prevent me from using a technique just because others had
trouble with it.

I presume you also campaign against anti-lock braking systems (or at
least don't use cars which have them - after all, anyone who knows how
to drive should be able to brake properly, right? And even if they
don't, *you* can, so you shouldn't have to pay the extra. Of course, if
someone else happens to hit you because they *couldn't* drive safely you
will happily give up your life).

And if bar() == foo is the superior technique anyway, because the ==
happens in chronological and lexical order after the bar() call.

That's about the most specious piece of hogwash I've seen this week.
Python doesn't make any guarantees about the evaluation order of =='s
operands (unlike "and" and "or"). That's cargo cult programming right
there. Readability is surely the only reason to decide how to write that
"if", and I suspect most people would choose "if foo == bar()" for that
reason, though the difference is slight.

regards
Steve

 

[Grant Edwards]

NO! It's a rude way to start a sentence don't you think?

No. When somebody asks a yes/no question, answering yes or no
seems quite polite to me. Following the yes/no answer with an
explanation of the answer is always nice, and I've little doubt
that's what happened.

Just because you're correcting someone doesn't mean you have
to be combative and try and make them feel small.

Answering a yes/no question with "no" doesn't seem to me to be
combative if the correct answer is indeed "no". But I've lost
track of the post you found objectionable...

 

[Steve Holden]

switching = for == is the "most common error in C"?

I can't tell if you are joking.

Well, there's also the "indenting code without surrounding it by braces"
error, but y'all just keep defending the approach to programming that
*you* think is best. Just don't expect others to agree. The FAQ clearly
documents why Python is a statement-based, and not an expression-based,
language.

Me, I think Guido is a pretty good language designer, and while we've
had our differences I think that Python is a *much* better language than
C and most others.

regards
Steve

 

[Bruno Desthuilliers]

Phlip a écrit :

switching = for == is the "most common error in C"?

I can't tell if you are joking.

It's at least a _very_ common error in all languages that allow this
construct. In C, it's common enough to gave birth to the "BestPractice"
you described, ie swapping operand orders in equality test to have the
compiler detect the problem - at least when one of the operand is a
function call expression or constant (it obviously won't 'work' when
both operands are variables).

Anyway: in Python, assignment is not an expression, and this isn't going
to change anytime soon.

 

[Ben Kaplan]

{41: None}[41] ?

In cases where None is a valid result, you can't use it to signal failure.

Asked and answered. You change the "sentinel" in .fetch to something
else.

When did I specify a sentinel value? You're saying that if None is a
valid value in the dict, you can't use slice syntax for it.

But y'all keep on defending the language making your programming
decisions for you!

Your example is the one where the language makes programming decisions
for you. Your language is deciding that if I use slice syntax, I want
the sentinel to be None. Python is deciding that unless I give it a
sentinel, there is no sentinel.

 

[Phlip]

y'all just keep defending the approach to programming that
*you* think is best.

Speak for yourself...

 

[Lie Ryan]

{41: None}[41] ?

In cases where None is a valid result, you can't use it to signal failure..

Asked and answered. You change the "sentinel" in .fetch to something
else.

I believe Ben Kaplan's point is that if dict slicing returns sentinel
for missing keys, the slicing syntax would be unusable to differentiate
{41: sentinel} and {}; if the default sentinel had been None, that would
make it too easy to forget to reset the sentinel and wrote a buggy code.

But y'all keep on defending the language making your programming
decisions for you!

When designing a language, there are two different ways to signal a
missing key in a dict: 1) raise an exception or 2) return a sentinel.
Both are useful for different purpose. One or the other must be the
default behavior, and the other must give way.

Python decided that the default behavior should be raising exception and
sentinel have to use the dict.get() method. Simple and clear. The other
possible behavior (i.e. slicing returns a sentinel while dict.get()
raises an exception) is arguably just as simple and just as clear; but
python doesn't do it that way. Why? Because it's part of what makes
python Python[1].

But y'all keep on defending the language making your programming
decisions for you!

Indeed, the language makes decisions for you; why would you want to use
a language that nags you for every possible different behavior? I want a
reasonable default and I want a reasonably easy way to access the
non-default behaviors.

[1] the essence of which is emboldened as the Zen, which is the broad
design guideline for both python's syntax and codes written in python.
Specifically, "Errors should never pass silently" because missing key
indicates a possible error and "Simple is better than complex" because
returning sentinel for missing keys implies the need for an interface to
change the sentinel[2].
[2] or become like Java, whose HashMap cannot reliably store null and
doesn't allow you to specify the sentinel.

 

[Lie Ryan]

Speak for yourself...

Everyone speaks for themselves, is that a problem?

 

[Grant Edwards]

Speak for yourself...

Everyone speaks for themselves, [...]

Except for the Lorax. He speaks for the trees.

 

[Phlip]

Everyone speaks for themselves, is that a problem?

Of course not. I was pointing out that Steve is telling me not to
force my programming opinions on everyone...

....while defending Python enforcing programming opinions on everyone.

And now, if everyone will excuse me, I have to get back to writing a
unit-test-to-code ratio of 2:1. Have fun being rigorous, everyone!

 

[Terry Reedy]

Python decided that the default behavior should be raising exception and
sentinel have to use the dict.get() method. Simple and clear. The other
possible behavior (i.e. slicing returns a sentinel while dict.get()
raises an exception) is arguably just as simple and just as clear; but
python doesn't do it that way. Why? Because it's part of what makes
python Python[1].

The altermatives are not quite equivalent. The current way lets one
specify the sentinel whereas the alternative does not. There is hardly
any reason to specify the exception.

tjr

 

[Steven D'Aprano]

And now, if everyone will excuse me, I have to get back to writing a
unit-test-to-code ratio of 2:1.

In my experience, that's about half as many unit-tests as needed for full
code coverage for even a simple class. If you're trying to impress us,
you have failed.

Have fun being rigorous, everyone!

You say that as if writing correct code was a bad thing.

 

[Steve Holden]

Of course not. I was pointing out that Steve is telling me not to
force my programming opinions on everyone...

...while defending Python enforcing programming opinions on everyone.

That's because Python is not a person, and Guido is a better language
designer than both of us put together. No languagecan be all things to
all programmers, and Python represents a set of pragmatic and useful
choices.

And now, if everyone will excuse me, I have to get back to writing a
unit-test-to-code ratio of 2:1. Have fun being rigorous, everyone!

Consider yourself excused ;-)

regards
Steve

 

[Michi]

I'm glad we agree on that, but I wonder why you previously emphasised
machine efficiency so much, and correctness almost not at all, in your
previous post?

Uh… Because the original poster quoted one small paragraph out of a
large article and that paragraph happened to deal with this particular
(and minor) point of API design?

If all you're argument is that we shouldn't write crappy APIs, then I
agree with you completely.

Well, yes: the article was precisely about that. And the point about
exception efficiency was a minor side remark in that article.

Your argument seems to be
that we should avoid exceptions by default, and only use them if
unavoidable. I think that is backwards.

I never made that argument. After 25 years as a software engineer, I
well and truly have come to appreciate exceptions as a superior form
of error handling. I simply stated that throwing an exception when
none should be thrown is a pain and often inefficient on top of that.
That's all, really.

I wouldn't say that's normal. If you don't care about the function's
result, why are you calling it? For the side-effects?

printf returns a value that is almost always ignored. And, yes, a
function such as printf is inevitable called for its side effects. We
could argue that printf is misdesigned (I would): the return value is
not useful, otherwise it would be used more. And if printf threw an
exception when something didn't work, that would be appropriate
because it fails so rarely that, if it does fail, I probably want to
know.

There's that premature micro-optimization again.

Let's be clear here: the entire discussion is about *inappropriate*
use of exceptions. This isn't a premature optimisation. It's about
deciding when an exception is appropriate and when not. If I throw an
exception when I shouldn't, I make the API harder to use *and* less
efficient. The real crime isn't the loss of efficiency though, it's
the inappropriate exception.

I've been wondering when you would reach the conclusion that an API
should offer both forms. For example, Python offers both key-lookup that
raises exceptions (dict[key]) and key-lookup that doesn't (dict.get(key))..

The danger of this is that it complicates the API, leads to a more
complex implementation, and may result in duplicated code (if the two
functions have independent implementations).

Offering a choice in some form can be appropriate for some APIs. I'm
not advocating it as a panacea, and I'm aware of the down-side in
increased complexity, learning curve, etc. (BTW, the article discusses
this issue in some detail.)

Well, obviously I agree that you should only make things be an exception
if they actually should be an exception. I don't quite see where the
implication is

In the context of the original article, I argued that throwing
exceptions that are inappropriate is one of the many things that API
designers get wrong. To many people, that's stating the obvious. The
number of APIs that still make exactly this mistake suggests that the
point is worth making though.

Anyway, some of the early posts implied that I was arguing against
exception handling per-se because exceptions can be less efficient. I
responded to correct that misconception. What the article really said
is that throwing an exception when none should be thrown is bad API
design, and inefficient to boot. I stand by that statement.

Cheers,

Michi.