__nonzero__ of iterators

Discussion in 'Python' started by Christian Eder, Apr 1, 2004.

  1. Hi,

    I just discovered the following pitfall in Python 2.3.
    Consider the following code :

    >>> a = {}
    >>> bool (a.keys ())

    False
    >>> bool (a.iterkeys ())

    True

    So, an "empty" iterator evaluates to True in boolean context.
    At a first glance, this is not what one would expect. This causes
    several problems, e.g. if you operate on something expected to be
    sequence, and you guard your code with "if seq :" to avoid crashing into
    an empty sequence, you still crash if you get an empty iterator, even if
    the rest of your code is able to deal with an iterator as well as with
    any other sequence type.

    At a second glance, the behaviour is clear, since the iterator object
    does not know wheter it is able to provide the next element, before
    having done so.

    Anyway, although I don't know how the iterator protocol is implemented
    in Python 2.3, I suppose the __nonzero__ could be changed to check
    whether a next element can be provided without actually consuming it.

    I would like to read some comments on this topic as I'm sure that I'm
    not the first one wondering whether the current behaviour should be that
    way.

    thanks
    chris
    Christian Eder, Apr 1, 2004
    #1
    1. Advertising

  2. Christian Eder

    Yermat Guest

    Christian Eder wrote:
    > Hi,
    >
    > I just discovered the following pitfall in Python 2.3.
    > Consider the following code :
    >
    > >>> a = {}
    > >>> bool (a.keys ())

    > False
    > >>> bool (a.iterkeys ())

    > True
    >
    > So, an "empty" iterator evaluates to True in boolean context.
    > At a first glance, this is not what one would expect. This causes
    > several problems, e.g. if you operate on something expected to be
    > sequence, and you guard your code with "if seq :" to avoid crashing into
    > an empty sequence, you still crash if you get an empty iterator, even if
    > the rest of your code is able to deal with an iterator as well as with
    > any other sequence type.
    >
    > At a second glance, the behaviour is clear, since the iterator object
    > does not know wheter it is able to provide the next element, before
    > having done so.
    >
    > Anyway, although I don't know how the iterator protocol is implemented
    > in Python 2.3, I suppose the __nonzero__ could be changed to check
    > whether a next element can be provided without actually consuming it.
    >
    > I would like to read some comments on this topic as I'm sure that I'm
    > not the first one wondering whether the current behaviour should be that
    > way.
    >
    > thanks
    > chris
    >


    In fact, what is missing is a 'hasNext' function...

    But for your example, the behavior is quite clear.
    bool(a.keys()) == bool([]) == False
    Whereas:
    bool(a.iterkeys()) = bool(anIterator) == (anIterator != None) == True

    So what is really missing is a hasNext method on Iterator. It would also
    be usefull in some loop.

    But what to do with Generator ?

    Yermat
    Yermat, Apr 2, 2004
    #2
    1. Advertising

  3. Christian Eder

    John Roth Guest

    "Christian Eder" <> wrote in message
    news:c4gjup$615$...
    > Hi,
    >
    > I just discovered the following pitfall in Python 2.3.
    > Consider the following code :
    >
    > >>> a = {}
    > >>> bool (a.keys ())

    > False
    > >>> bool (a.iterkeys ())

    > True
    >
    > So, an "empty" iterator evaluates to True in boolean context.
    > At a first glance, this is not what one would expect. This causes
    > several problems, e.g. if you operate on something expected to be
    > sequence, and you guard your code with "if seq :" to avoid crashing into
    > an empty sequence, you still crash if you get an empty iterator, even if
    > the rest of your code is able to deal with an iterator as well as with
    > any other sequence type.
    >
    > At a second glance, the behaviour is clear, since the iterator object
    > does not know wheter it is able to provide the next element, before
    > having done so.
    >
    > Anyway, although I don't know how the iterator protocol is implemented
    > in Python 2.3, I suppose the __nonzero__ could be changed to check
    > whether a next element can be provided without actually consuming it.
    >
    > I would like to read some comments on this topic as I'm sure that I'm
    > not the first one wondering whether the current behaviour should be that
    > way.


    One of the issues is that the iterator many not be able, even in
    principle, be able to figure out whether there is, in fact, a next
    element. Consider a pipe or an internet connection.

    However, you're talking about the special iterator that the
    dictionary object returns. I'm not sure whether this behavior
    is a bug or a feature. It would clearly be possible for that
    iterator object to proxy the __nonzero__ call to the basic
    dictionary object, and from what you show, it doesn't do so.

    I belive this is a quite different issue from a "has next" type
    of function.

    John Roth
    >
    > thanks
    > chris
    >
    John Roth, Apr 2, 2004
    #3
  4. Michele Simionato, Apr 4, 2004
    #4
  5. [Christian Eder]
    > > I just discovered the following pitfall in Python 2.3.
    > > Consider the following code :
    > >
    > > >>> a = {}
    > > >>> bool (a.keys ())

    > False
    > > >>> bool (a.iterkeys ())

    > > True
    > >
    > > So, an "empty" iterator evaluates to True in boolean context.
    > > At a first glance, this is not what one would expect. This causes
    > > several problems, e.g. if you operate on something expected to be
    > > sequence, and you guard your code with "if seq :" to avoid crashing into
    > > an empty sequence, you still crash if you get an empty iterator, even if
    > > the rest of your code is able to deal with an iterator as well as with
    > > any other sequence type.


    This was fixed in Py2.4:

    Python 2.4a0 (#46, Apr 4 2004, 05:21:08) [MSC v.1200 32 bit (Intel)]
    on win32
    IDLE 1.1a0
    >>> d = dict(a=1, b=2, c=3)
    >>> it = d.iterkeys()
    >>> bool(it)

    True
    >>> list(it)

    ['a', 'c', 'b']
    >>> bool(it)

    False



    [John Roth]
    > One of the issues is that the iterator many not be able, even in
    > principle, be able to figure out whether there is, in fact, a next
    > element. Consider a pipe or an internet connection.
    >
    > However, you're talking about the special iterator that the
    > dictionary object returns. I'm not sure whether this behavior
    > is a bug or a feature. It would clearly be possible for that
    > iterator object to proxy the __nonzero__ call to the basic
    > dictionary object, and from what you show, it doesn't do so.
    >
    > I belive this is a quite different issue from a "has next" type
    > of function.


    Right. So, the only iterators that provide knowledge of their length
    are the ones like dictionaries that know what lies ahead. For the
    rest, the behavior is unchanged (i.e. they do not provide a __len__()
    method).


    Raymond Hettinger
    Raymond Hettinger, Apr 5, 2004
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Ken Sprague
    Replies:
    4
    Views:
    671
  2. Russ Perry Jr
    Replies:
    2
    Views:
    4,100
    Russ Perry Jr
    Aug 20, 2004
  3. Marcin Kaliciñski

    Iterators and reverse iterators

    Marcin Kaliciñski, May 8, 2005, in forum: C++
    Replies:
    1
    Views:
    477
    Kai-Uwe Bux
    May 8, 2005
  4. Sergey Kishchenko
    Replies:
    6
    Views:
    305
    Terry Reedy
    Jan 9, 2009
  5. , India
    Replies:
    10
    Views:
    1,060
    James Kanze
    Aug 8, 2009
Loading...

Share This Page