question of style

[Steven D'Aprano]

Anyway, Python's overloading of bool(...) is yet another misfeature and
although it's convenient, the "explicit is better than implicit"
principle indicates to avoid that sort of trick.

"Overloading of bool()"?

I don't understand what that means -- you mean you dislike the ability to
overload __bool__ (__nonzero__ in older Pythons)? That seems strange.
Python allows you to overload __int__ and __str__ and __float__, why is
__bool__ a misfeature? If that's what you mean, then I'm perplexed.

Or do you mean that all Python objects are interpretable in a truth
context? If that's what you mean, then I'm not perplexed, I'm sure you're
utterly wrong. Certain people -- a tiny minority -- keep trying to argue
that the ability to say "if obj" for arbitrary objects is somehow a bad
thing, and their arguments seem to always boil down to:

"If you write code that assumes that only bools have a truth value, then
surprising things will happen because all objects have a truth value."

Well duh.

If you think you have a better reason for calling the truth value of
arbitrary objects a "misfeature", I'd like to hear it.

 

[Paul Rubin]

Certain people -- a tiny minority -- keep trying to argue
that the ability to say "if obj" for arbitrary objects is somehow a bad
thing, and their arguments seem to always boil down to:
"If you write code that assumes that only bools have a truth value, then
surprising things will happen because all objects have a truth value."

I'd put it under the general rubric of "explicit is better than
implicit". The language shouldn't do silly automatic typecasts all
over the place. Yes, it saves a few keystrokes to say "if x:" instead
of "if len(x)==0:" or even "if bool(x):", but if I program in a style
where I like to think I know the type of something when I use it, I'd
like the interpreter to let me know when I'm wrong instead of
proceeding silently.

 

[Steven D'Aprano]

I'd put it under the general rubric of "explicit is better than
implicit".

"if x" is explicit. It's difficult to see how a branch could be anything
other than explicit, but never mind.

The language shouldn't do silly automatic typecasts all over
the place.

"if x" doesn't involve a typecast. Python doesn't have typecasts, except
possibly for the special case of myobject.__class__ = Another_Class.

If you mean Python shouldn't do silly automatic type conversions all over
the place, I absolutely agree with you! Fortunately, testing the truth
value of an object isn't a silly automatic type conversion.

Yes, it saves a few keystrokes to say "if x:" instead of "if
len(x)==0:" or even "if bool(x):",

It's not about saving keystrokes -- that's a furphy. It's about
encapsulation. Objects are in a better position to recognise when they
are "something" (true) or "nothing" (false) than you are.

Given an arbitrary object x, how do you know if it's something or
nothing? In general, you can't tell -- but the object can, provided it's
well written. (The conspicuous exception is iterators, but that's a
special case.)

"if len(x) == 0" is wasteful. Perhaps I've passed you a list-like
iterable instead of a list, and calculating the actual length is O(N).
Why spend all that effort to find out the length of the object is
59,872,819 only to throw that away? My object knows when it's empty, but
instead you make foolish assumptions about the object and consequently
write wastefully slow code.

but if I program in a style where I
like to think I know the type of something when I use it, I'd like the
interpreter to let me know when I'm wrong instead of proceeding
silently.

Oh come on now... that's a silly objection. If you want strongly-typed
variables in Python, say so, don't pretend this is a failure of truth-
testing. If you write len(x)==0 Python doesn't complain if x is a dict
instead of the list you were expecting. Why is it acceptable to duck-type
len(x) but not truth-testing?

 

[Paul Rubin]

It's not about saving keystrokes -- that's a furphy. It's about
encapsulation. Objects are in a better position to recognise when they
are "something" (true) or "nothing" (false) than you are.

I don't know what a furphy is, but I don't accept that "somethingness"
vs. "nothingness" is the same distinction as truth vs falsehood. True
and False are values in a specific datatype (namely bool), not
abstract qualities of arbitrary data structures. The idea that the
"if" statement selects between "somethingness" and "nothingness"
rather than between True and False is a bogus re-imagining of the
traditional function of an "if" statement and has been an endless
source of bugs in Python code. Look how much confusion it causes here
in the newsgroup all the time.

"if len(x) == 0" is wasteful. Perhaps I've passed you a list-like
iterable instead of a list, and calculating the actual length is O(N).

That doesn't happen in any of Python's built-in container types. I
could see some value to having a generic "is_empty" predicate on
containers though, to deal with this situation. Your iterable could
support that predicate. In fact maybe all iterables should support
that predicate. They don't (and can't) all support "len".

If you write len(x)==0 Python doesn't complain if x is a dict
instead of the list you were expecting. Why is it acceptable to
duck-type len(x) but not truth-testing?

I haven't seen the amount of bugs coming from generic "len" as from
something-vs-nothing confusion.

 

[Lie Ryan]

That doesn't happen in any of Python's built-in container types. I
could see some value to having a generic "is_empty" predicate on
containers though, to deal with this situation. Your iterable could
support that predicate. In fact maybe all iterables should support
that predicate. They don't (and can't) all support "len".

That doesn't happen because all python's built-in container keep track
of its own length and are able to quickly determine its own length. But
outside builtins, certain iterables cannot determine its own length
quickly, e.g. iterators, but may have alternative ways to determine
whether itself is empty (through a "private" attributes). If you peek
through this private attribute, you're breaking encapsulation. Although
python is a language of consenting adults and doesn't really have a real
private, codes that breaks encapsulation is prone to bugs.

I don't know what a furphy is, but I don't accept that "somethingness"
vs. "nothingness" is the same distinction as truth vs falsehood. True
and False are values in a specific datatype (namely bool), not
abstract qualities of arbitrary data structures. The idea that the
"if" statement selects between "somethingness" and "nothingness"
rather than between True and False is a bogus re-imagining of the
traditional function of an "if" statement and has been an endless
source of bugs in Python code. Look how much confusion it causes here
in the newsgroup all the time.

Neither python's `if` nor `if` in formal logic is about testing True vs.
False. `if` in python and formal logic receives a statement. The
statement must be evaluatable to True or False, but does not have to be
True or False themselves. It just happens that True evaluates to True
and False evaluates to False. For example, the statement:

P := `e = 2.7182818284590451`
Q := `e = m*c**2`
----------------------------------
P -> Q

P -> Q evaluates to:
`e = 2.7182818284590451` -> `e = m*c**2`

Note that `e = 2.7182818284590451` is a statement, not a boolean value.
The truth value of `e = 2.7182818284590451` is determined by "calling"
(note the double quotes) `e = 2.7182818284590451`.statement_is_true(),
which when written in python syntax becomes: (e ==
2.7182818284590451).__bool__()

 

[Paul Rubin]

No, wait, I tell I lie... re.search() sometimes bites me, because
sometimes it returns None and sometimes it returns a matchobject and I
don't use re often enough to have good habits with it yet.

re is a common source of this type of bug but there are others.

There are three natural approaches to (say) re.search() for dealing with
failure:

(1) return a sentinel value like None;
(2) return a matchobject which tests False;
(3) raise an exception.

4. Have re.search return a bool and possible matchobject separately:

put_match_here = []
if re.search(pat, s, target=put_match_here):
do_something_with(put_match_here[0])

or alternatively (cleaner), have a new type of object which supports
search operations while self-updating with the match object:

mt = re.match_target()
...
if mt.search(pat, s):
do_something_with(mt.match)
if mt.search(pat2, s):
do_another_thing_with(mt.match)
...

This is sort of inspired by what Perl does. I often do something like
this because it makes it cleaner to chain a series of matches.

 

[Steven D'Aprano]

I don't know what a furphy is,

Is your Google broken? *wink*

http://en.wikipedia.org/wiki/Furphy

but I don't accept that "somethingness"
vs. "nothingness" is the same distinction as truth vs falsehood.

It's the distinction used by Python since the dawn of time. Python only
grew a bool type a few versions back.

True
and False are values in a specific datatype (namely bool), not abstract
qualities of arbitrary data structures.

I'm not talking about the constants True and False (nouns), but about
true and false values (adjectives).

The idea that the "if"
statement selects between "somethingness" and "nothingness" rather than
between True and False is a bogus re-imagining of the traditional
function of an "if" statement

There's nothing bogus about it.

Some languages such as Pascal and Java require a special Boolean type for
if-branches. Other languages, like Forth, C, Lisp and Ruby do not.

http://en.wikipedia.org/wiki/Boolean_data_type

and has been an endless source of bugs in Python code.

I wonder why these "endless" bugs weren't important enough to be
mentioned in the rationale to PEP 285:

http://www.python.org/dev/peps/pep-0285/

You'd think something as vital as "if x Considered Harmful" would have
made it into the PEP, but no. Instead Guido *explicitly* stated that he
had no intention of forcing `if` to require a bool, describing `if x` as
the "correct form" and calling scrapping such a feature as "crippling the
language".

Look how much confusion it causes here in the newsgroup all the time.

The only confusion is that you're causing me. Would you care to link to
some?

That doesn't happen in any of Python's built-in container types.

And if they were the only types possible in Python, that might be
relevant.

I
could see some value to having a generic "is_empty" predicate on
containers though, to deal with this situation.

We have that already. It's spelled __bool__ or __nonzero__, and it
applies to any object, not just containers.

Your iterable could
support that predicate. In fact maybe all iterables should support that
predicate. They don't (and can't) all support "len".

Iterators are a special case, because in general they can't tell if
they're exhausted until they try to yield a value.

I haven't seen the amount of bugs coming from generic "len" as from
something-vs-nothing confusion.

Again with these alleged bugs.

 

[Steven D'Aprano]

Neither python's `if` nor `if` in formal logic is about testing True vs.
False. `if` in python and formal logic receives a statement. The
statement must be evaluatable to True or False, but does not have to be
True or False themselves. It just happens that True evaluates to True
and False evaluates to False.

I think your explanation is a little confused, or at least confusing.

`if` implements a two-way branch. Some languages, like Pascal and Java,
requires the switch value to take one of two specific enumerable values
conventionally spelled TRUE and FALSE (modulo variations in case).

Other languages don't require specific enumerable values, and instead
accept (e.g.) any integer, or any object, with rules for how to interpret
such values in such a context. Forth, for example, branches according to
whether the word on the stack is zero or non-zero: "nothing" or
"something". Lisp branches according to empty list or non-empty list:
"nothing" or "something" again.

Other languages, like Ruby, have less intuitive rules. That's their
problem.

 

[Steven D'Aprano]

There are three natural approaches to (say) re.search() for dealing
with failure:

(1) return a sentinel value like None; (2) return a matchobject which
tests False; (3) raise an exception.

4. Have re.search return a bool and possible matchobject separately:

put_match_here = []
if re.search(pat, s, target=put_match_here):
do_something_with(put_match_here[0])

Wow. I never for the life of me thought I'd see an experienced Python
programmer re-implement Pascal's VAR parameters.

That is... hideous. Returning a (flag, matchobject) tuple is the height
of beauty in comparison.

 

[Lie Ryan]

I think your explanation is a little confused, or at least confusing.

Indeed, partially because I said "statement" when I really meant
"expression".

Other languages don't require specific enumerable values, and instead
accept (e.g.) any integer, or any object, with rules for how to interpret
such values in such a context.

That was what I was wanting to say, except that I stretched that to
formal logic (mathematical logic). Even in formal logic `if` receives
any arbitrary expression that can be -- according to certain rules --
interpreted as True or False (i.e. the expressions themselves are not
required to be a boolean value).

The conclusion is python's `if` does not deviate from `if`'s semantic in
mathematical sense.

 

[Terry Reedy]

I don't know what a furphy is, but I don't accept that "somethingness"
vs. "nothingness" is the same distinction as truth vs falsehood. True
and False are values in a specific datatype (namely bool), not
abstract qualities of arbitrary data structures. The idea that the
"if" statement selects between "somethingness" and "nothingness"
rather than between True and False is a bogus re-imagining of the
traditional function of an "if" statement and has been an endless
source of bugs in Python code. Look how much confusion it causes here
in the newsgroup all the time.

You appear to be confusing a specific interpretation of an abstraction
with the abstraction itself. Or perhaps better, you seem to be confusing
a specific example of a general process with the general process.

A boolean variable is a variable that is in one of two states -- a
binary variable -- a variable that carries one bit of information. The
two states are the marked state and the unmarked or default state. Which
is to say, when we draw a distinction to distinguish two states, we mark
one of them to distinguish one from the other. The if statement tests
whether an object is in the marked state (or not).

Truth and falsity of propositions are one possible interpretation of
marked and unmarked, giving us propositional logic. But they are only
one of many. So are in and out of a particular class and member or not
of a set or subclass or not of a set, giving us class and set logic. So
are closed and open, said of gates or switches, or on and off, giving us
switching logic. So are non-zero and zero, said of numbers. Or done and
not done, said of an algorithmic process.

Counts 0 and 1, and their representations '0' and '1', taken in
themselves, are as good as any distinct pair as a pair of labels for the
two distinct states. They have some computational advantages, including
making it easy to count the number of objects in a collection in the
marked state (and, given the total number, the number in the unmarked
state). They have one disadvantage, though. If I say 'x = 1', do I mean
1 versus 0 or 1 versus all possible ints? Similarly, If 'print(x)'
prints 1, does it mean 1 versus 0 or 1 versus all other ints?
Recognizing this, Guido decided to subclass them and give them alternate
names. He could have chosen 'Marked' and 'Unmarked', or any of several
other pairs, but did choose the conventional 'True' and 'False',
referring to the common propositional interpretation. However, he
specifically disclaimed any intention to restrict 'if' to testing
specific logic propositions, as opposed to the general proposition
'object is in the marked state'.

Terry Jan Reedy

 

[Paul Rubin]

It's the distinction used by Python since the dawn of time. Python only
grew a bool type a few versions back.

That's true, part of the situation we have now is an artifact of that
history.

I'm not talking about the constants True and False (nouns), but about
true and false values (adjectives).

But, it seems to me, the constants True and False are the only values
to which the adjectives "true" and "false" should be applicable to.

There's nothing bogus about it.

I wonder why these "endless" bugs weren't important enough to be
mentioned in the rationale to PEP 285:

Because adding the bool type doesn't really fix those bugs.

describing `if x` as the "correct form" and calling scrapping such a
feature as "crippling the language".

Certainly, changing "if" would have broken an immense amount of code
and been a completely unworkable approach. We are using a fairly
mature language by now; it has a culture and history that carries
certain baggage, as one should expect.

The only confusion is that you're causing me. Would you care to link to
some?

This current discussion (about bools) came from such confusion just a
few posts up in this very thread:

From: upwestdon <[email protected]>
Date: Fri, 3 Jul 2009 23:03:39 -0700 (PDT)
How about just:

if not (self.higher and self.lower):
return self.higher or self.lower

That test was designed to treat None as a boolean False, without
noticing that numeric 0 is also treated as False and could make the
test do the wrong thing. This is an extremely common type of error.

We have that already. It's spelled __bool__ or __nonzero__

That's fine, but under the "explicit is better than implicit"
principle, it's preferable to call that predicate explicitly:
"if bool(x): ..." rather than "if x:". Also, after many years of
fixing bugs caused by the mushing together of None and False, it seems
to me that we'd have been better off if bool(None) raised an exception
rather than returning false. None is supposed to denote a nonexistent
value, not a false or empty value. The only valid way to check
whether something is None should have been "if x is None". However,
it is of course way too late to do this differently.

Iterators are a special case, because in general they can't tell if
they're exhausted until they try to yield a value.

Right, it would be nice if they supported a lookahead slot, though
that would complicate a lot of __iter__ methods. There's been various
kludgy attempts to wrap them in ways that support this, though.

 

[Paul Rubin]

BTW, Paul, kind of a tangent: I reimplemented the same algorithm but
using tuples instead of instances (and empty tuples for "NULL"
values.) I was trying to mess around in the space you seemed to
indicate existed, i.e. a better implementation using other datatypes,
but I didn't have a clear idea what I was doing and, as I said, I
started by simply re-implementing with a different datatype.

Much to my surprise and delight, I discovered the tuple-based BTree
was /already/ a "persistent data type"! It was both awesome and a bit
of an anti-climax. :]

Cool ;-). It also seems to me a bit irregular to require every tree
to have a node with optional children, rather than allowing trees to be
compleely empty. I think the irregularity complicated the code somewhat.

 

[Albert van der Horst]

Anyway, Python's overloading of bool(...) is yet another misfeature
and although it's convenient, the "explicit is better than implicit"
principle indicates to avoid that sort of trick.

Well, trading "tricks" (I would call it idiom) for explicitness,
made some people (you know ..) hate Pascal.
Einstein introduced the summation convention for indices that
are used twice. Leaving out summation signs is absolutely hideous,
but it has saved generations of physicists of loosing track (and
their minds.)

Also ...

For me there is nothing more clear than a regular expression
search that returns None if nothing matches.

Groetjes Albert

 

[Albert van der Horst]

That's true, part of the situation we have now is an artifact of that
history.


But, it seems to me, the constants True and False are the only values
to which the adjectives "true" and "false" should be applicable to.


Because adding the bool type doesn't really fix those bugs.


Certainly, changing "if" would have broken an immense amount of code
and been a completely unworkable approach. We are using a fairly
mature language by now; it has a culture and history that carries
certain baggage, as one should expect.


This current discussion (about bools) came from such confusion just a
few posts up in this very thread:

From: upwestdon <[email protected]>
Date: Fri, 3 Jul 2009 23:03:39 -0700 (PDT)
How about just:

if not (self.higher and self.lower):
return self.higher or self.lower

That test was designed to treat None as a boolean False, without
noticing that numeric 0 is also treated as False and could make the
test do the wrong thing. This is an extremely common type of error.


That's fine, but under the "explicit is better than implicit"
principle, it's preferable to call that predicate explicitly:
"if bool(x): ..." rather than "if x:". Also, after many years of

Maybe I'm missing something here, but if self.higher contains 0,
wouldn't bool(self.higher) evaluate to False?
So how does this help?

Groetjes Albert

 

[koranthala]

That test was designed to treat None as a boolean False, without

noticing that numeric 0 is also treated as False and could make the
test do the wrong thing. This is an extremely common type of error.

Actually, I felt that 0 not being considered False would be a better
option.
I had lot of instances where 0 is a valid value and always I had to
put checks specifically for that.
For me, None and False should be equal to False in if checks, every
thing else being considered True.

Can someone let me know why 0 is considered equal to False?
There should be real good reasons for it, only that I cannot think of
one.

 

[Chris Rebert]

Actually, I felt that 0 not being considered False would be a better
option.
I had lot of instances where 0 is a valid value and always I had to
put checks specifically for that.
For me, None and False should be equal to False in if checks, every
thing else being considered True.

Can someone let me know why 0 is considered equal to False?
There should be real good reasons for it, only that I cannot think of
one.

* Because that's how C does/did it (primary motivation)
- Because if Boolean algebra is implemented using numbers, False is 0
- Because philosophically, 0 is the "empty" or null value for numbers,
and empty values are by convention considered false in Python
- Because it's the behavior people want 75% of the time.

FWIW, your paradigm of true/false is used by Lisp and Lisp wannabes
such as Ruby.

Cheers,
Chris

 

[Steven D'Aprano]

Actually, I felt that 0 not being considered False would be a better
option.
I had lot of instances where 0 is a valid value and always I had to put
checks specifically for that.
For me, None and False should be equal to False in if checks, every
thing else being considered True.

I believe it would be a mistake for None == False to return True.

As far as having only None and False to be considered equivalent in truth
contexts, that's awfully limiting. It is very useful to be able to write
e.g.:

if header or body or footer:
print assemble_page(header, body, footer)

and have empty strings to be equivalent to False.


Lisp and (I think) Ruby behaves as you want. Python doesn't. We think
it's Lisp and Ruby that got it wrong

Can someone let me know why 0 is considered equal to False? There should
be real good reasons for it, only that I cannot think of one.

Having 0 considered false goes back to branching in machine language,
which I understand are typically. The Null value (e.g. nil in Pascal, nul
in C) are usually implemented as pointer values equal to an address of
zero.

It essentially boils down to this: the behaviour of if...else is a
convention. The language designer might choose to have an explicit
Boolean type, and prohibit "if None" (e.g. Pascal does this). It might
choose to treat flags as integers, and branch according to the integer
being zero (false) or non-zero (true) (e.g. Forth does this). Python
allows any object to be tested by if, and branches according to whether
it is Something or Nothing. It's a logical dichotomy, and treating 0 and
[] as Something instead of Nothing would make it arbitrary and strange.

 

[Paul Rubin]

It is very useful to be able to write e.g.:
if header or body or footer:
print assemble_page(header, body, footer)
and have empty strings to be equivalent to False.

Why doesn't assemble_page properly handle the case where header, body,
and footer are all empty? That would let you eliminate the if. "Make
sure your code 'does nothing' gracefully" (Kernighan and Plauger).

 

[Piet van Oostrum]

k> Actually, I felt that 0 not being considered False would be a better
k> option.
k> I had lot of instances where 0 is a valid value and always I had to
k> put checks specifically for that.
k> For me, None and False should be equal to False in if checks, every
k> thing else being considered True.

k> Can someone let me know why 0 is considered equal to False?
k> There should be real good reasons for it, only that I cannot think of
k> one.

Originally Python didn't have booleans. Integers were used instead (but
not exclusively). 0 was False, everything else True (with a slight
preference for 1 as True). Same as in C. Nowadays this is reflected in
bool being a subtype of int with False == 0 and True == 1. Actually it
is even closer: False and True are just 0 and 1 cast to bool, so to say.