__nonzero__ of iterators

[Christian Eder]

Hi,

I just discovered the following pitfall in Python 2.3.
Consider the following code :
True

So, an "empty" iterator evaluates to True in boolean context.
At a first glance, this is not what one would expect. This causes
several problems, e.g. if you operate on something expected to be
sequence, and you guard your code with "if seq :" to avoid crashing into
an empty sequence, you still crash if you get an empty iterator, even if
the rest of your code is able to deal with an iterator as well as with
any other sequence type.

At a second glance, the behaviour is clear, since the iterator object
does not know wheter it is able to provide the next element, before
having done so.

Anyway, although I don't know how the iterator protocol is implemented
in Python 2.3, I suppose the __nonzero__ could be changed to check
whether a next element can be provided without actually consuming it.

I would like to read some comments on this topic as I'm sure that I'm
not the first one wondering whether the current behaviour should be that
way.

thanks
chris

 

[Yermat]

Hi,

I just discovered the following pitfall in Python 2.3.
Consider the following code :

True

So, an "empty" iterator evaluates to True in boolean context.
At a first glance, this is not what one would expect. This causes
several problems, e.g. if you operate on something expected to be
sequence, and you guard your code with "if seq :" to avoid crashing into
an empty sequence, you still crash if you get an empty iterator, even if
the rest of your code is able to deal with an iterator as well as with
any other sequence type.

At a second glance, the behaviour is clear, since the iterator object
does not know wheter it is able to provide the next element, before
having done so.

Anyway, although I don't know how the iterator protocol is implemented
in Python 2.3, I suppose the __nonzero__ could be changed to check
whether a next element can be provided without actually consuming it.

I would like to read some comments on this topic as I'm sure that I'm
not the first one wondering whether the current behaviour should be that
way.

thanks
chris

In fact, what is missing is a 'hasNext' function...

But for your example, the behavior is quite clear.
bool(a.keys()) == bool([]) == False
Whereas:
bool(a.iterkeys()) = bool(anIterator) == (anIterator != None) == True

So what is really missing is a hasNext method on Iterator. It would also
be usefull in some loop.

But what to do with Generator ?

Yermat

 

[John Roth]

Hi,

I just discovered the following pitfall in Python 2.3.
Consider the following code :

True

So, an "empty" iterator evaluates to True in boolean context.
At a first glance, this is not what one would expect. This causes
several problems, e.g. if you operate on something expected to be
sequence, and you guard your code with "if seq :" to avoid crashing into
an empty sequence, you still crash if you get an empty iterator, even if
the rest of your code is able to deal with an iterator as well as with
any other sequence type.

At a second glance, the behaviour is clear, since the iterator object
does not know wheter it is able to provide the next element, before
having done so.

Anyway, although I don't know how the iterator protocol is implemented
in Python 2.3, I suppose the __nonzero__ could be changed to check
whether a next element can be provided without actually consuming it.

I would like to read some comments on this topic as I'm sure that I'm
not the first one wondering whether the current behaviour should be that
way.

One of the issues is that the iterator many not be able, even in
principle, be able to figure out whether there is, in fact, a next
element. Consider a pipe or an internet connection.

However, you're talking about the special iterator that the
dictionary object returns. I'm not sure whether this behavior
is a bug or a feature. It would clearly be possible for that
iterator object to proxy the __nonzero__ call to the basic
dictionary object, and from what you show, it doesn't do so.

I belive this is a quite different issue from a "has next" type
of function.

John Roth

 

[Michele Simionato]

<snip>
I would like to read some comments on this topic as I'm sure that I'm
not the first one wondering whether the current behaviour should be that
way.

Indeed. You may want to read this thread of last summer:

http://groups.google.it/groups?hl=i...mpty+iterator&meta=group%3Dcomp.lang.python.*

Michele Simionato

 

[Raymond Hettinger]

[Christian Eder]
This was fixed in Py2.4:

Python 2.4a0 (#46, Apr 4 2004, 05:21:08) [MSC v.1200 32 bit (Intel)]
on win32
IDLE 1.1a0

d = dict(a=1, b=2, c=3)
it = d.iterkeys()
bool(it) True
list(it) ['a', 'c', 'b']
bool(it)

False



[John Roth]

One of the issues is that the iterator many not be able, even in
principle, be able to figure out whether there is, in fact, a next
element. Consider a pipe or an internet connection.

However, you're talking about the special iterator that the
dictionary object returns. I'm not sure whether this behavior
is a bug or a feature. It would clearly be possible for that
iterator object to proxy the __nonzero__ call to the basic
dictionary object, and from what you show, it doesn't do so.

I belive this is a quite different issue from a "has next" type
of function.

Right. So, the only iterators that provide knowledge of their length
are the ones like dictionaries that know what lies ahead. For the
rest, the behavior is unchanged (i.e. they do not provide a __len__()
method).


Raymond Hettinger