Comparison with False - something I don't understand

[Steven D'Aprano]

I find a bit cumbersome
that exceptions are advocated for certain conditions which can be sanely
worked around in the application's logic and even avoided, rather than
waiting for them to get caught and providing an unsatisfactory result.

That's surprisingly rare in Python. In fact, I'd go so far as to say that
in Python there is *nothing* that you can test for and then have a
*guarantee* that it will succeed.

Of course, this is mainly of theoretical concern. In practice, "Look
Before You Leap" (test first, then process) is often fine. But there are
traps to look out for. For example, unless you are running a single-
process machine, the following code is subject to race conditions and is
not safe:

if os.exists(pathname):
fp = open(pathname)
else:
handle_missing_file()

Just because the file is there when os.exists() looks for it, doesn't
mean it still exists a microsecond later when you try opening it.

Or consider this code:

if y != 0:
result = x/y
else:
handle_division_by_zero()


This is also unsafe unless you know the type of y. Suppose y is an
interval quantity that straddles zero, then division by y may fail even
though y != 0.

 

[Harishankar]

Of course, this is mainly of theoretical concern. In practice, "Look
Before You Leap" (test first, then process) is often fine. But there are
traps to look out for. For example, unless you are running a single-
process machine, the following code is subject to race conditions and is
not safe:

if os.exists(pathname):
fp = open(pathname)
else:
handle_missing_file()

Just because the file is there when os.exists() looks for it, doesn't
mean it still exists a microsecond later when you try opening it.

I understand this line of thinking. And it makes sense to see why it
would matter to leave the exception handling mechanism deal with such
issues.

Or consider this code:

if y != 0:
result = x/y
else:
handle_division_by_zero()


This is also unsafe unless you know the type of y. Suppose y is an
interval quantity that straddles zero, then division by y may fail even
though y != 0.

Of course in each of these cases the in-built exceptions are used to
verify the result of certain system level or lower level operations. My
object was not to deprecate the system-level or other low level
exceptions thrown by Python, but to consider whether such a mechanism
would be a preferable method of handling your own programs error-
conditions.

The issue to be considered by every programmer is to define what can be
defined as the exceptional condition and what is a condition that merits
merely different treatment without causing disruption of the normal flow
of the program.

 

[Tim Harig]

Of course in each of these cases the in-built exceptions are used to
verify the result of certain system level or lower level operations. My
object was not to deprecate the system-level or other low level
exceptions thrown by Python, but to consider whether such a mechanism
would be a preferable method of handling your own programs error-
conditions.

Whether you happen to like the exception mechanism and syntax or not, it is
the idiomatic way of handling errors in Python. Using two different
conventions in your code will lead to confusion. I come from a long C
background as well. I have come to appreciate the power the Python's
exception handling provides. It does everything that you need to do with
passing values in C and more.

The issue to be considered by every programmer is to define what can be
defined as the exceptional condition and what is a condition that merits
merely different treatment without causing disruption of the normal flow
of the program.

That is an issue much harder to define. Anything it is an obvious
error *should* throw an exception. Invalid input is an error.
Unusable hardware states are errors. Any invalid request to an object,
is an error. Essentially anything that deviates from a normal flow of
a program, to handle an exceptional condition, is an error

Where it becomes less obvious is when you start using exceptions as
part normal control flow. An example is a try it and see methodology.
You might for instance have a group of file objects which might or might
not support a particular method attribute. You might have a preference for
using the attribute; but, have a fallback plan if it does not. One way to
handle this is to try to use the attribute and catch the exception raised
if it is not present to execute your backup method. I have found this
*essential* in some marsaling enviroments where you might not have access to
the meta-data of the object that you are working with.

Another, questionable but useful use, is to ignore the complex accounting
of your position inside of a complex data structure. You can continue
moving through the structure until an exception is raised indicating
that you have reached a boundary of the structure.

Whether you accept uses of exceptions like these is more of a personal
quesion. Like many good tools, they can be useful in ways that they were
never really designed to be and I would hate to proclude some of these
really useful features.

This can, of course, be easily abused. I was once writing code, involving
complex object marshaling like I described above, with a partner who
wasn't totally familiar with Python. We came to a situation where it
was impossible to know ahead of time what kind of object (one of two
possiblities) we would receive from another marshalled object and had no
meta-data to be able to figure out before attempting to access the object.
I used a try/except clause to resolve the problem. The next day, I
found several poorly conceived try/except blocks in the codebase that
my partner had used for control structures using dictionaries because
he didn't know of dict.has_key(). I was not so pleased.

 

[Tim Harig]

Another, questionable but useful use, is to ignore the complex accounting
of your position inside of a complex data structure. You can continue
moving through the structure until an exception is raised indicating
that you have reached a boundary of the structure.

Here is another example in this vein. A friend was trying to derive a
mathematical formula for determining the possibly distribution of results
from rolling arbitrariy numbers of m n-sided dice and needed several sets
of data in different directions from which to draw conclusions.

I created objects for dice and roles which contained and manipulated
multiple dice. To generate a listing of all (non-uniq) possible roles,
I would call the first dices increment method read and read the dice
faces into a log until the first dice threw an exception that it could
not be further incremented. Then I would call reset() on the first dice
and increment the second and so on much like the odometer of a car.

By using exceptions rather then checking the return value of increment,
the state information of the dice was completely isolated to the dice
and did not polute into the role structure; the logic for incrementing
the dice, logging the role state, and rolling over the dice where
all completely seperated and independent of any state; and therefore
reseting multiple previous dice as the higher values on the odometer were
incremented functioned automatically as each dice threw its own exception
recursively rather then requiring logic to handle these multiple rollovers.

 

[Paul Rubin]

A friend was trying to derive a mathematical formula for determining
the possibly distribution of results from rolling arbitrariy numbers
of m n-sided dice

http://en.wikipedia.org/wiki/Multinomial_distribution

To generate a listing of all (non-uniq) possible roles, I would call
the first dices increment method read and read the dice faces into a
log until the first dice threw an exception that it could not be
further incremented. Then I would call reset() on the first dice and
increment the second and so on much like the odometer of a car.

from itertools import product
m, n = 5, 2
for roll in product(*(xrange(1,m+1) for i in xrange(n))):
print roll

 

[Steven D'Aprano]

Anything it is an obvious
error *should* throw an exception.

Well, maybe... there are good use-cases for returning a sentinel. E.g.
str.find, or the use of quiet NANs in IEEE floating point and decimal
maths.

NANs and INFs in floating point maths are a good example of the right way
to do it. If you forget to check for a NAN, it will propagate through
your calculation. INF will, under some circumstances where it is
mathematically valid to do so, will disappear leaving a normal result.
This means you only need to check your result at the very end of the
calculation, not after every step.

str.find is more troublesome, because the sentinel -1 doesn't propagate
and is a common source of errors:

result = string[string.find(delim):]

will return a plausible-looking but incorrect result if delim is missing
from string. But the convenience and familiarity of str.find means it
will probably be around forever.

 

[Tim Harig]

http://en.wikipedia.org/wiki/Multinomial_distribution

I sure he rediscovered much of that. Working that out for himeself was
probably far more educational then simply reading an article on the
solution.

from itertools import product
m, n = 5, 2
for roll in product(*(xrange(1,m+1) for i in xrange(n))):
print roll

The fact that I bothered to create classes for the dice and roles, rather
then simply iterating over a list of numbers, should tell you that I
produced was of a far more flexible nature; including the support for
roles with dice having different numbers of sides. I basically created
a DSL that he could use to generate and automatically calculate the
properties of series of roles defined by one or more varying property.

I merely posted a simplied description of the dice-role objects because I
thought that it demonstrated how exceptions can provide eligance of control
for situations that don't involve what would traditionally be defined as an
error.

 

[Paul Rubin]

The fact that I bothered to create classes for the dice and roles, rather
then simply iterating over a list of numbers, should tell you that I
produced was of a far more flexible nature; including the support for
roles with dice having different numbers of sides.

from itertools import product
def n_sided_die(n): return xrange(1, n+1)

# make one 3-sided die, one 4-sided die, and one 5-sided die
dice = (n_sided_die(3), n_sided_die(4), n_sided_die(5))
for roll in product(*dice):
print roll

I merely posted a simplied description of the dice-role objects
because I thought that it demonstrated how exceptions can provide
eligance of control for situations that don't involve what would
traditionally be defined as an error.

Exceptions (though sometimes necessary) are messy and I'm having a hard
time trying to envision that code being cleaner with them than without
them. If you post the actual code maybe that will help me understand.

 

[Tim Harig]

from itertools import product
def n_sided_die(n): return xrange(1, n+1)

# make one 3-sided die, one 4-sided die, and one 5-sided die
dice = (n_sided_die(3), n_sided_die(4), n_sided_die(5))
for roll in product(*dice):
print roll

Notice that you had to change the structure of your program to accomodate
the new possiblity; and, if I throw further requirements at you, you
will have to change them again. I didn't want that. What the dice
returned may or may not have returned an easy to compute sequence.
In fact, for all of the rest of the logic cared, the dice could have
computed their value from a previous role of another dice. All of the
logic of about what the dice may have returned when asked for their
value and how they derived, was encapsilated in the dice themselves.
It did not need to be known anywhere else in the program logic.

The DSL effectively provided a way do define how the dice decided how
to increment themselves, how to choose the value that they returned for
their face, and how to know when they could no longer be incremented.
The DSL parser generated the dice set from the description given.
Creating new dice objects was much easier then attempting to change the
logic of how they were rolled.

Exceptions (though sometimes necessary) are messy and I'm having a hard
time trying to envision that code being cleaner with them than without
them. If you post the actual code maybe that will help me understand.

Let me get this straight, the same person that was trying to tell me
setjmp/longjmp weren't messy thinks exceptions are messy? I have used
both. I much prefer the exceptions. I not have to code here to post.

The cleanliness of using the exception and calling the dice increments
recursively, was that there was no need to figure out which dice needed
to be incremented whenever the first die needed to be reset. When a dice
needed to be reset, it would raise an exception. This exception would
rise through the recursion stack, and thus through the dice, resetting
each along the way until it found the one which needed to be incremented
or raised past the top call indicating that all of the combinations has
been exhausted. There, once the reset condition for the previous dice
had been effectively handled, it would be supprested.

Had this been done using in band data:

1. The roll object would have needed logic to determine when
a reset condition needed to take place, effectively
leaking some of the logic from the dice object to the
role object.

2. The roll object would have needed logic to determine how to
follow the dice which needed to be reset until it found
which one needed incrementing. Once again, this logic
was better left to the dice walking the resets was
automatically handled by the progression of the exception.

Even if it wasn't an error, the resets were effectively a exceptional
condition from the normal flow of the role object (the primary flow simply
being to increment the first die). By using exceptions, I effectively
isolated each into its own separate independent flow; and, because they
where called separatly, neither needed to have control conditions to detect
which was needed. They simply allowed the dice object to decide.

 

[Tim Harig]

Notice that you had to change the structure of your program to accomodate
the new possiblity; and, if I throw further requirements at you, you
will have to change them again. I didn't want that. What the dice
returned may or may not have returned an easy to compute sequence.
In fact, for all of the rest of the logic cared, the dice could have
computed their value from a previous role of another dice. All of the
logic of about what the dice may have returned when asked for their
value and how they derived, was encapsilated in the dice themselves.
It did not need to be known anywhere else in the program logic.

The DSL effectively provided a way do define how the dice decided how
to increment themselves, how to choose the value that they returned for
their face, and how to know when they could no longer be incremented.
The DSL parser generated the dice set from the description given.
Creating new dice objects was much easier then attempting to change the
logic of how they were rolled.


Let me get this straight, the same person that was trying to tell me
setjmp/longjmp weren't messy thinks exceptions are messy? I have used
both. I much prefer the exceptions. I not have to code here to post.

The cleanliness of using the exception and calling the dice increments
recursively, was that there was no need to figure out which dice needed
to be incremented whenever the first die needed to be reset. When a dice
needed to be reset, it would raise an exception. This exception would
rise through the recursion stack, and thus through the dice, resetting
each along the way until it found the one which needed to be incremented
or raised past the top call indicating that all of the combinations has
been exhausted. There, once the reset condition for the previous dice
had been effectively handled, it would be supprested.

Had this been done using in band data:

1. The roll object would have needed logic to determine when
a reset condition needed to take place, effectively
leaking some of the logic from the dice object to the
role object.

2. The roll object would have needed logic to determine how to
follow the dice which needed to be reset until it found
which one needed incrementing. Once again, this logic
was better left to the dice walking the resets was
automatically handled by the progression of the exception.

Even if it wasn't an error, the resets were effectively a exceptional
condition from the normal flow of the role object (the primary flow simply
being to increment the first die). By using exceptions, I effectively
isolated each into its own separate independent flow; and, because they
where called separatly, neither needed to have control conditions to detect
which was needed. They simply allowed the dice object to decide.

Okay, it occures to me that you don't really need to see much to know what was
going on, here is the basic idea of how the role function of the object would
have looked like:

def role(self, dice):

try:
self.role(dice.previous())
except diceReset:
dice.increment()
except endOfDice:
raise diceReset

 

[Tim Chase]

On Sun, 05 Dec 2010 04:13:02 +0000, Tim Harig wrote:
str.find is more troublesome, because the sentinel -1 doesn't propagate
and is a common source of errors:

result = string[string.find(delim):]

will return a plausible-looking but incorrect result if delim is missing
from string. But the convenience and familiarity of str.find means it
will probably be around forever.

Fortunately, string objects offer both .find() and .index() so
you can choose whether you want sentinels or exceptions according
to your use-case.

-tkc

 

[Martin v. Loewis]

result = myfunction (vars)

if not result:
# error condition

Now above I first realized that the function can also return an empty
list under some conditions and so changed it to

If your function returns a list when successful, it should not return
False in the error case. Instead, it should return None (indicating that
there is no list).

Then the condition changes to

result = myfunction()
if result is None:
# error condition

Using None for "no result available" is very common in Python. Using
False for the same purpose (i.e. returning either a list or False)
is not. If you return False from a function, the only other possible
result should be True.

Regards,
Martin

 

[MRAB]

If your function returns a list when successful, it should not return
False in the error case. Instead, it should return None (indicating that
there is no list).

Then the condition changes to

result = myfunction()
if result is None:
# error condition

Using None for "no result available" is very common in Python. Using
False for the same purpose (i.e. returning either a list or False)
is not. If you return False from a function, the only other possible
result should be True.

As an example, the re module uses both two approaches.

If you ask it to compile a regex:

rgx = re.compile(regex)

it either returns a PatternObject (if the regex is valid) or raises an
exception (if the regex isn't valid).

If you ask it to search a string:

rgx.search(string)

it returns either a MatchObject (if the match is successful) or None
(if the match isn't successful).

 

[Lie Ryan]

Here is another example in this vein.

I had another example where using Exception as a control structure
proves to be the most elegant solution.

The problem was a logic puzzle solver (e.g. for Sudoku, Einstein's Logic
problem, etc). The algorithm used is recursive backtracking solver (yes,
I know there are more efficient constraint-based solver, but that's
besides the point).

The function modifies the `puzzle` list in-place such that after the
call `puzzle` list will contain the solution to the puzzle (if any
exists). The solving "in-place" is important since this solver thread
runs in a "solver thread" and there is another "drawing thread" that
draws the currently tested board asynchronously. This means we do not
make a copy of the game board. No locking is done, since it is fine for
the drawing thread to draw malformed board, and we do not want to
compromise the solver's thread's speed.

The two versions of using return value and exception:

def solve_return(puzzle, index):
""" return True when puzzle is solved,
return False when backtracking or when no solution exists
"""
# recursion base case
if all cells are filled and puzzle does not violate game rules:
return True # solution found

if puzzle violates game rules:
return False # backtrack

if puzzle[index] is unfilled:
for i in possible_candidates(puzzle, index):
puzzle[index] = i
if solve(puzzle, index+1):
# solution already found, unwinding the stack
return True

# all candidates failed, backtrack
puzzle[r][c] = unfilled
return False
else: # the current cell is part of the base clue
return solve(puzzle, index+1) # skip it

def main_return():
puzzle = [...]
if solve_return(puzzle, 0):
print('solution found')
else:
print('no solution found')

def solve_raise(puzzle, index):
""" no return value
throws SolutionFound when solution is found
"""
# recursion base case
if all cells are filled and puzzle does not violate game rules:
raise SolutionFound

if puzzle[index] is unfilled:
for i in possible_candidates(puzzle, index):
puzzle[index] = i
if puzzle does not violate game rules:
solve(puzzle, index+1)

# all candidates failed, backtrack
puzzle[r][c] = unfilled
else: # the current cell is part of the base clue
solve(puzzle, index+1) # skip it

def main_raise():
puzzle = [...]
try:
solve_raise(puzzle, 0)
except SolutionFound:
print('solution found')
else:
print('no solution found')


Personally, I've found the logic in the exception version clearer than
the return version. Also, the exception version is easier to extend, if
I wanted to add a smarter algorithm that can deterministically infer
certain cell's value (this creates indirect recursion, solve() may call
either infer() or solve() which may call either infer() or solve()), it
can work beside the existing mechanism without explicitly handling the
return flag when a solution is found.

If we suppose that end-of-line (e.g. the semicolon, in C-like language)
is a single-step forward control structure, the if-statement is a n-step
forward control structure, and looping is a n-step backward control
structure. Now, if we suppose function call as a single-step downward
control structure, and function return as a single-step upward control
structure, then exception is a n-step upward control structure. It
throws control upwards of the function call stack, while doing cleanup
along its way up.