Comparison with False - something I don't understand

[Harishankar]

When I run pychecker through my modules I get the message that
comparisons with "False" is not necessary and that it might yield
unexpected results.

Yet in some situations I need to specifically check whether False was
returned or None was returned. Why is comparison with False so bad?

# example code which matches both False and None
if not var:
# do something

# specifically check if False is returned
# I don't want None
if var == False:
# do something

So how do you get around this? My functions return False and None under
different circumstances. Should I raise exceptions instead? I feel it's
unnecessary clutter to use exceptions unless absolutely no other solution
is available and yet I have doubts about the "False" value.

 

[Nobody]

When I run pychecker through my modules I get the message that
comparisons with "False" is not necessary and that it might yield
unexpected results.

Yet in some situations I need to specifically check whether False was
returned or None was returned. Why is comparison with False so bad?

The behaviour may be counterintuitive.

One might expect that "x == False" is equivalent to "not x". Sometimes it
is, sometimes it isn't.

E.g. 0 and 0.0 are equal to False and are equivalent to False when
converted to booleans:

> 0 == False
True
> not 0
True

> 0.0 == False
True
> not 0.0
True

[], "" and None aren't equal to False but are equivalent to False when
converted to booleans:

> [] == False
False
> not []
True

> "" == False
False
> not ""
True

> None == False
False
> not None
True

The boolean conversions are what's relevant for "if x ...", "while x ...",
etc.

If you want to test specifically for True, False or None, use "is" rather
than an equality check. This eliminates the warning and doesn't
risk misleading someone reading the code.

 

[Harishankar]

If you want to test specifically for True, False or None, use "is"
rather than an equality check. This eliminates the warning and doesn't
risk misleading someone reading the code.

Thanks so much for this very specific answer. I guess "is" is what I am
looking for.

 

[Harishankar]

More details of the problem you're trying to solve would help with
giving specific advice.

I'm writing functions with multiple points of failure exits. I use return
False as a way to flag the error condition rather than raising
exceptions. But under certain circumstances, the function can also return
empty lists which equate to false when using the condition like:

# myfunction () can return a list of tuples, but can also return an empty
# list under certain conditions since it's query a database and the result
# can be empty also

result = myfunction (vars)

if not result:
# error condition

Now above I first realized that the function can also return an empty
list under some conditions and so changed it to

if result == False:
# error condition


But now I realize that it's better to use "is"

if result is False:
# error condition

That is how my problem arose.

-
Harishankar (http://harishankar.org http://lawstudentscommunity.com)

 

[Steve Holden]

I'm writing functions with multiple points of failure exits. I use return
False as a way to flag the error condition rather than raising
exceptions. But under certain circumstances, the function can also return
empty lists which equate to false when using the condition like:

# myfunction () can return a list of tuples, but can also return an empty
# list under certain conditions since it's query a database and the result
# can be empty also

result = myfunction (vars)

if not result:
# error condition

Now above I first realized that the function can also return an empty
list under some conditions and so changed it to

if result == False:
# error condition


But now I realize that it's better to use "is"

if result is False:
# error condition

That is how my problem arose.

Did you think about using exceptions to handle exceptional conditions?
If you are new to Python it may not be the obvious soltuion, but it can
greatly simplify program logic.

regards
Steve

 

[Grant Edwards]

Did you think about using exceptions to handle exceptional conditions?
If you are new to Python it may not be the obvious soltuion, but it can
greatly simplify program logic.

If you're not used to using exceptions it may at first seem like they
take extra effort to use. But usually, in the end, they end up being
less work (and more importantly easier to read and less bugs).

 

[Harishankar]

Did you think about using exceptions to handle exceptional conditions?
If you are new to Python it may not be the obvious soltuion, but it can
greatly simplify program logic.

regards
Steve

I am not new to Python but I am not a fan of exceptions either. I prefer
to avoid writing my own exceptions because it feels too heavy and clunky
for simple error checking. Most times I find simple error checking ample
for my purposes.

Of course, I use the built-in exception objects when I have no choice,
but I hate try blocks. They add clunkiness to code and besides exception
objects seem to be fairly heavy-duty for simple error conditions where a
true/false flag would probably suffice.

I am also wary of using larger catch-all try blocks or try blocks with
multiple exception exits (which seem to make tracking subtle bugs
harder). I prefer the philosophy of dealing with errors immediately as
they arise, rather than delegate them to exception mechanism. Of course,
I could wrap single statements in try blocks, but that makes the code
even messier without any significant benefits.

 

[Tim Harig]

I am also wary of using larger catch-all try blocks or try blocks with
multiple exception exits (which seem to make tracking subtle bugs
harder). I prefer the philosophy of dealing with errors immediately as

If you are using exceptions to try to catch bug then you are using them
improperly. Exceptions (with the exception (no pun intended) of
AssertionError) are designed to catch error conditions, not bugs.

harder). I prefer the philosophy of dealing with errors immediately as
they arise, rather than delegate them to exception mechanism. Of course,
I could wrap single statements in try blocks, but that makes the code
even messier without any significant benefits.

Actually, finer grained error handling commonly covers up bugs. If you
want to find bugs, you want to make the program prone to crashing if
a bug is present. It is all too easy to accidently mistake the return
value of a function as error condition and handle it rather the letting
the program crash. By separating the results from the transmission of
error conditions (in effect taking error conditions out of band) then
you make it much harder to make such a mistake because you have to
explicity indicate which error conditions your code is capable of
generating.

 

[Harishankar]

If you are using exceptions to try to catch bug then you are using them
improperly. Exceptions (with the exception (no pun intended) of
AssertionError) are designed to catch error conditions, not bugs.

I agree. But more specifically some exceptions seem too broad to catch
specific errors that occur in MY code rather than in a system call or a
library call.

Also writing my own exception objects and raising exceptions seem to add
too much of complexity to what is essentially a simple problem. This is
what I was trying to explain.

Actually, finer grained error handling commonly covers up bugs. If you
want to find bugs, you want to make the program prone to crashing if a
bug is present. It is all too easy to accidently mistake the return
value of a function as error condition and handle it rather the letting
the program crash. By separating the results from the transmission of
error conditions (in effect taking error conditions out of band) then
you make it much harder to make such a mistake because you have to
explicity indicate which error conditions your code is capable of
generating.

Doesn't the same finer grained exception mechanism make it prone to the
same problems?

Actually return values of functions which I write myself can be as
specific and to the point. I could make it as fuzzy or as precise as I
like. This is no doubt, something that I could emulate with an exception
as well, but only make it slightly more complex with no obvious benefit.

As I said before, the way exceptions are caught seem to me to be the most
confusing bit. Non-atomic operations always worry me. What if my function
which is wrapped inside a try block has two different statements that
raised the same exception but for different reasons? With error handling
I could probably handle it right below the statement which was called and
thus reduce the problem???

 

[Tim Harig]

Doesn't the same finer grained exception mechanism make it prone to the
same problems?

Exception handling is a move away from fine grained handling. If in
IOError is raised by a parser, I don't need to know at exactly which
line the error occured. All I need to know is that the file is somehow
unreadable; and that is why the parser failed. Therefore, none of
the parser code should be handling IOError because it is being handled
higher up in the stack. Since I expected that the parser might have
problemes reading files, for problems that have nothing to do with my
code, I explicity catch that error if it occurs. I am not expecting an
IndexError from the parser and I don't bother to catch it. If the parser
code does raise an IndexError, then my program crashes and I know that I
have a bug in the parsing code. The call trace will tell me where that
error occurs. I can watch that section of code in debugger to find out
exactly what went wrong.

Actually return values of functions which I write myself can be as
specific and to the point. I could make it as fuzzy or as precise as I
like. This is no doubt, something that I could emulate with an exception
as well, but only make it slightly more complex with no obvious benefit.

You seem to be making it complex because you are still trying to be too
fine grained in handling each exception where it occurs as opposed to
handing where the logic makes sense that it should be handled and because
you are trying to code too defensively against your own code. Exception
handling does require a different focus from handling errors from return
values alone.

As I said before, the way exceptions are caught seem to me to be the most
confusing bit. Non-atomic operations always worry me. What if my function
which is wrapped inside a try block has two different statements that
raised the same exception but for different reasons? With error handling
I could probably handle it right below the statement which was called and
thus reduce the problem???

If you are having that issue, then you are likely placing the try blocks
at too low of a level in your code. In general you will find that
most systems have a gateway function as an entry point to the system.
If there is not one already, then create such a function in you code.
The parse function in my code above would be an example of such a
gateway function. Beneath that function, you don't need to know exactly
where the error occured, you just need to know the nature of the error and
have general error handling procedures for each kind of error that you
expect might occur.

 

[Jean-Michel Pichavant]

As I said before, the way exceptions are caught seem to me to be the most
confusing bit. Non-atomic operations always worry me. What if my function
which is wrapped inside a try block has two different statements that
raised the same exception but for different reasons? With error handling
I could probably handle it right below the statement which was called and
thus reduce the problem???

Exception actually are the solution as you can give an unlimitted
quantity of information:

def myfunction(self):
error = myfuncException('a meaninful message')
# error 1
error.blame = whateverobjectresponsibleoftheerror
# error 2
error.blame = anotherobject_anothercause

raise error

try:
myfunction()
except myfuncException, exception:
cause = exception.blame
# you can inspect the 'cause' for specific handling


In a more general and simple manner, you can tune the error feedback of
exceptions by changing the message of the exception. Using different
exception classes is also an obvious way to do it.

But my point was that you can also set attributes to the exception
you're raising with reference to almost anything at the time the
exception occurs. And that is a very cool way to give precise feedback
to exception handlers.

JM

 

[Harishankar]

Raise exceptions for exceptional cases, and define the function
interface so that it's doing one clear job only. Often that involves
breaking a complicated function into several collaborating functions
with simpler interfaces.

This is probably what I should try to do. Of course my function returns
only a list in most circumstances. Only in error does it return False. I
mis-represented the None type for the empty list in my previous post, my
apologies.

 

[Harishankar]

If you are having that issue, then you are likely placing the try blocks
at too low of a level in your code. In general you will find that most
systems have a gateway function as an entry point to the system. If
there is not one already, then create such a function in you code. The
parse function in my code above would be an example of such a gateway
function. Beneath that function, you don't need to know exactly where
the error occured, you just need to know the nature of the error and
have general error handling procedures for each kind of error that you
expect might occur.

I think I might very well by using try blocks rather defensively rather
than letting the code reach its logical conclusion in normal
circumstances. This is why I think I find it a bit clunky.

I think I understand the general trend of what you're saying. It
definitely requires a mindset change. I still feel that user-defined
exception classes might not be the way, but maybe I should allow the
built-in exceptions which are thrown by library functions to follow its
natural path upwards till it reaches the top level where it could be
handled.

Maybe I should handle the error only at the highest level (UI level)
rather than returning False to flag errors.

One of the reasons why I feared to do this is because I need to know each
and every exception that might be thrown by the function and litter my
top-level code with too many exception handlers.

 

[Tim Harig]

I think I might very well by using try blocks rather defensively rather
than letting the code reach its logical conclusion in normal
circumstances. This is why I think I find it a bit clunky.

That was the conclusion I was coming to.

I think I understand the general trend of what you're saying. It
definitely requires a mindset change. I still feel that user-defined
exception classes might not be the way, but maybe I should allow the
built-in exceptions which are thrown by library functions to follow its
natural path upwards till it reaches the top level where it could be
handled.

Look at it this way, in C you were constrained to place your error
handling code around ever function that might fail. Now you are free
to place the error handling code wherever it makes sense to do so.
As a general rule, if, in C, your function would handle the error by
passing an error return value to the calling function, then the error
handling code should be higher up.

Maybe I should handle the error only at the highest level (UI level)
rather than returning False to flag errors.

I don't write many UIs; but, I normally consider the UI code to be yet
another subsystem. In general, I find the best place to place error
handling code in the high level business logic of your application (which
might be what you actually meant by UI, to me the UI code is the code that
actually draws the interface), in the high level logic of the systems,
and in the bounderies between subsystems. The exceptions caught at each
level depend on where the logic to handle the error is best applied.

One of the reasons why I feared to do this is because I need to know each
and every exception that might be thrown by the function and litter my
top-level code with too many exception handlers.

Each exception has a place where it is better handled. Wherever you
find boundaries between subsystems, think about what error conditions
that subsystem might encounter. Subsystems dealing with are likely
to encounter io related errors, network subsystems network errors,
parsers validation errors etc. Logic exceptions that indicate errors
in your code should be left alone entirely so that they may be easily
found. Look at the exceptions pertaining to these subsystems.

For each error reaching the boundery, think about whether you have enough
information within the module to handle the error in a constructive
manner or whether the error handling would benefit from information
further up in the program. If you have all of the information that you
need then handle it in the main logic of that subsystem. If not, pass
it up to the error handlers on top of the boundry. When you get there,
make the same decision.

In general you only need to catch a handful of exceptions at each level.
The easy excpetions will be handled at lower levels. The most difficult
exceptions will rise towards the top of the program until only the terminal
exceptions, that cannot be resolved are left with the inevitable result that
you should notify the user and exit, will remain.

 

[Mel]

I think I understand the general trend of what you're saying. It
definitely requires a mindset change. I still feel that user-defined
exception classes might not be the way, but maybe I should allow the
built-in exceptions which are thrown by library functions to follow its
natural path upwards till it reaches the top level where it could be
handled.

User-defined exception classes are no big deal, and I think they're helpful.
At the minimum, they're just a few lines in a module, e.g.:

class SumpError (StandardError): '''Errors raised by the SUMP client.'''
class SumpIdError (SumpError): '''The wrong string was returned by an ID
request.'''
class SumpFlagsError (SumpError): '''Illegal combination of flags.'''
class SumpStageError (SumpError): '''Illegal trigger stage setting.'''

This is from a module to drive some special hardware through a serial
connection. At this stage in development, I don't even have try/except
statements for these. It's enough that some typo will not silently put the
hardware into an illegal state, and instead will report

Traceback (most recent call last):
File "logic_sniffer.py", line 184, in OnDeviceCapture
set_sump_options (grabber)
File "logic_sniffer.py", line 21, in set_sump_options
sump.set_flags (demux=True, filter=True, channel_groups=0x0,
external=False, inverted=False) # only 1 channel group
File "/home/mwilson/sandbox/sump-analyzer/sump.py", line 160, in set_flags
raise SumpFlagsError
sump.SumpFlagsError

Because these are subclasses of StandardError, they'll be caught by any
`except StandardError`, which may or may not turn out to be a mistake.

Once development is done, try/except for these will be in some window
methods as part of a wxPython GUI, several call levels above the code that
would raise the exceptions, up where a human user would take steps to change
the way things are being done, or submit a bug report (more likely), or
something.

Maybe I should handle the error only at the highest level (UI level)
rather than returning False to flag errors.

One of the reasons why I feared to do this is because I need to know each
and every exception that might be thrown by the function and litter my
top-level code with too many exception handlers.

The advantage to the exceptions, is that they only need to be recognized and
caught and handled at the UI level. They don't have to be recognized and
passed back up the call chain from level to level till they get to the right
place -- the way out-of-band error returns have to be.

Mel.

 

[Steve Holden]

One of the reasons why I feared to do this is because I need to know each
and every exception that might be thrown by the function and litter my
top-level code with too many exception handlers.

You appear to be suffering from the delusion that all exceptions must be
caught and handled. This is far from being the case. But still, better
to have your top-level code "littered with exception handlers" than to
have your functions "littered with if statements".

Quite often it's impossible for the function to know what needs to be
done when a specific conditions arises, in which case (presumably) you
have to return some error code and test for that ...

regards
Steve

 

[Steve Holden]

One of the reasons why I feared to do this is because I need to know each
and every exception that might be thrown by the function and litter my
top-level code with too many exception handlers.

You appear to be suffering from the delusion that all exceptions must be
caught and handled. This is far from being the case. But still, better
to have your top-level code "littered with exception handlers" than to
have your functions "littered with if statements".

Quite often it's impossible for the function to know what needs to be
done when a specific conditions arises, in which case (presumably) you
have to return some error code and test for that ...

regards
Steve

 

[Harishankar]

You appear to be suffering from the delusion that all exceptions must be

caught and handled. This is far from being the case. But still, better
to have your top-level code "littered with exception handlers" than to
have your functions "littered with if statements".

Of course not. But going by the replies here, it appears that Python has
made exceptions as the "norm" for error handling which is ironical
considering the meaning of the word "exception". I find a bit cumbersome
that exceptions are advocated for certain conditions which can be sanely
worked around in the application's logic and even avoided, rather than
waiting for them to get caught and providing an unsatisfactory result.

Quite often it's impossible for the function to know what needs to be
done when a specific conditions arises, in which case (presumably) you
have to return some error code and test for that ...

Not necessarily. I wasn't talking about low-level or built-in exceptions.
I was talking about using exceptions in my programming where often the
function is reasonably confident of the kind of errors it is likely to
incur. I did not start this as a criticism of Python's exceptions as
such. I just expressed my personal aversion to using them in my own code.

However, in my next project I have started using exceptions and will keep
an open mind on how it turns out. So far it doesn't seem too bad.

 

[D'Arcy J.M. Cain]

Of course not. But going by the replies here, it appears that Python has
made exceptions as the "norm" for error handling which is ironical
considering the meaning of the word "exception". I find a bit cumbersome
that exceptions are advocated for certain conditions which can be sanely
worked around in the application's logic and even avoided, rather than
waiting for them to get caught and providing an unsatisfactory result.

It just seems to me that you have a semantic issue rather than a
technical one. If the word "exception" was replaced by "check" or
something else would that make the process easier to swallow?

try:
somefunc()
check ValueError:
handle_error()

Whatever it's called it's just flow control.

Not necessarily. I wasn't talking about low-level or built-in exceptions.
I was talking about using exceptions in my programming where often the
function is reasonably confident of the kind of errors it is likely to
incur. I did not start this as a criticism of Python's exceptions as
such. I just expressed my personal aversion to using them in my own code.

However, in my next project I have started using exceptions and will keep
an open mind on how it turns out. So far it doesn't seem too bad.

Open minds are good.

 

[Terry Reedy]

Of course not. But going by the replies here, it appears that Python has
made exceptions as the "norm" for error handling which is ironical
considering the meaning of the word "exception".

In communications parlance, 'exception' = out-of-band signal or return
value, while 'return'ed value = in-band signal. A fake in-band return
value, like returning None (ok) or False (worse) to *signal* 'I cannot
return a list' is still an exception signal, even if 'in-band'.

The advantage of out-of-band signals is that they cannot be mistaken for
valid in-band signals (return values). If a caller neglects to catch an
exception, the process stops, as it should. If a caller neglects to
check return values, the process goes on (at least for a while) under
the pretense that error codes (in-band exception signals) are valid
return values.

Neglecting to check return values for error codes is a common bug in C
code. At worst, the process eventually return a bad value or performs a
bad action. At best, it crashes sometime later, making the bug hard to find.

Or a function is called without even storing, let alone checking the
return value. This is common for i/o functions. A program may 'finish'
without any indication that it failed. If one does the same with Python
functions (equally common), any exceptions *will* be passed up until
either caught or displayed on the screen with an informative traceback
(assuming that the screen is the not source of the error).