A quirk/gotcha of for i, x in enumerate(seq) when seq is empty

[Alex Willmer]

This week I was slightly surprised by a behaviour that I've not
considered before. I've long used

for i, x in enumerate(seq):
# do stuff

as a standard looping-with-index construct. In Python for loops don't
create a scope, so the loop variables are available afterward. I've
sometimes used this to print or return a record count e.g.

for i, x in enumerate(seq):
# do stuff
print 'Processed %i records' % i+1

However as I found out, if seq is empty then i and x are never
created. The above code will raise NameError. So if a record count is
needed, and the loop is not guaranteed to execute the following seems
more correct:

i = 0
for x in seq:
# do stuff
i += 1
print 'Processed %i records' % i

Just thought it worth mentioning, curious to hear other options/
improvements/corrections.

 

[Steven D'Aprano]

This week I was slightly surprised by a behaviour that I've not
considered before. I've long used

for i, x in enumerate(seq):
# do stuff

as a standard looping-with-index construct. In Python for loops don't
create a scope, so the loop variables are available afterward. I've
sometimes used this to print or return a record count e.g.

for i, x in enumerate(seq):
# do stuff
print 'Processed %i records' % i+1

However as I found out, if seq is empty then i and x are never created.

This has nothing to do with enumerate. It applies to for loops in
general: the loop variable is not initialised if the loop never runs.
What value should it take? Zero? Minus one? The empty string? None?
Whatever answer Python choose would be almost always wrong, so it refuses
to guess.

The above code will raise NameError. So if a record count is needed, and
the loop is not guaranteed to execute the following seems more correct:

i = 0
for x in seq:
# do stuff
i += 1
print 'Processed %i records' % i

What fixes the problem is not avoiding enumerate, or performing the
increments in slow Python instead of fast C, but that you initialise the
loop variable you care about before the loop in case it doesn't run.

i = 0
for i,x in enumerate(seq):
# do stuff

is all you need: the addition of one extra line, to initialise the loop
variable i (and, if you need it, x) before hand.

 

[Paul Rubin]

i = 0
for x in seq:
# do stuff
i += 1
print 'Processed %i records' % i

Just thought it worth mentioning, curious to hear other options/
improvements/corrections.

Stephen gave an alternate patch, but you are right, it is a pitfall that
can be easy to miss in simple testing.

A more "functional programming" approach might be:

def do_stuff(x): ...

n_records = sum(1 for _ in imap(do_stuff, seq))

 

[Ethan Furman]

This has nothing to do with enumerate. It applies to for loops in
general: the loop variable is not initialised if the loop never runs.
What value should it take? Zero? Minus one? The empty string? None?
Whatever answer Python choose would be almost always wrong, so it refuses
to guess.



What fixes the problem is not avoiding enumerate, or performing the
increments in slow Python instead of fast C, but that you initialise the
loop variable you care about before the loop in case it doesn't run.

i = 0
for i,x in enumerate(seq):
# do stuff

is all you need: the addition of one extra line, to initialise the loop
variable i (and, if you need it, x) before hand.

Actually,

i = -1

or his reporting will be wrong.

~Ethan~

 

[Mark Lawrence]

Actually,

i = -1

or his reporting will be wrong.

~Ethan~

Methinks an off by one error

 

[Peter Otten]

Actually,

i = -1

or his reporting will be wrong.

Yes, either

i = -1
for i, x in enumerate(seq):
...
print "%d records" % (i+1)

or

i = 0
for i, x in enumerate(seq, 1):
...
print "%d records" % i

 

[Rick Johnson]

[...]
as a standard looping-with-index construct. In Python for loops don't
create a scope, so the loop variables are available afterward. I've
sometimes used this to print or return a record count e.g.

for i, x in enumerate(seq):
   # do stuff
print 'Processed %i records' % i+1

You could employ the "else clause" of "for loops" to your advantage;
(psst: which coincidentally are working pro-bono in this down
economy!)

for x in []:

.... print x
.... else:
.... print 'Empty Iterable'
Empty Iterable

for i,o in enumerate([]):

.... print i, o
.... else:
.... print 'Empty Iterable'
Empty Iterable

 

[Peter Otten]

[...]
as a standard looping-with-index construct. In Python for loops don't
create a scope, so the loop variables are available afterward. I've
sometimes used this to print or return a record count e.g.

for i, x in enumerate(seq):
# do stuff
print 'Processed %i records' % i+1

You could employ the "else clause" of "for loops" to your advantage;

for x in []:

... print x
... else:
... print 'Empty Iterable'
Empty Iterable

for i,o in enumerate([]):

... print i, o
... else:
... print 'Empty Iterable'
Empty Iterable
No:

for i in []:

.... pass
.... else:
.... print "else"
....
else

for i in [42]:

.... pass
.... else:
.... print "else"
....
else

for i in [42]:

.... break
.... else:
.... print "else"
....
The code in the else suite executes only when the for loop is left via
break. A non-empty iterable is required but not sufficient.

 

[Peter Otten]

The code in the else suite executes only when the for loop is left via
break.

Oops, the following statement is nonsense:

A non-empty iterable is required but not sufficient.

Let me try again:

A non-empty iterable is required but not sufficient to *skip* the else-suite
of a for loop.

 

[Steven D'Aprano]

for i in []:

... pass
... else:
... print "else"
...
else

for i in [42]:

... pass
... else:
... print "else"
...
else

for i in [42]:

... break
... else:
... print "else"
...The code in the else suite executes only when the for loop is left via
break. A non-empty iterable is required but not sufficient.

You have a typo there. As your examples show, the code in the else suite
executes only when the for loop is NOT left via break (or return, or an
exception). The else suite executes regardless of whether the iterable is
empty or not.


for...else is a very useful construct, but the name is misleading. It
took me a long time to stop thinking that the else clause executes when
the for loop was empty.

In Python 4000, I think for loops should be spelled:

for name in iterable:
# for block
then:
# only if not exited with break
else:
# only if iterable is empty

and likewise for while loops.

Unfortunately we can't do the same now, due to the backward-incompatible
change in behaviour for "else".

 

[Arnaud Delobelle]

for...else is a very useful construct, but the name is misleading. It
took me a long time to stop thinking that the else clause executes when
the for loop was empty.

This is why I think we should call this construct "for / break / else"
rather than "for / else".

 

[Peter Otten]

You have a typo there. As your examples show, the code in the else suite
executes only when the for loop is NOT left via break (or return, or an
exception). The else suite executes regardless of whether the iterable is
empty or not.

Yup, sorry for the confusion.

 

[Rick Johnson]

for...else is a very useful construct, but the name is misleading. It
took me a long time to stop thinking that the else clause executes when
the for loop was empty.

Agreed. This is a major stumbling block for neophytes.

In Python 4000, I think for loops should be spelled:

for name in iterable:
    # for block
then:
    # only if not exited with break
else:
    # only if iterable is empty

and likewise for while loops.

I like this syntax better than the current syntax, however, it is
STILL far too confusing!

for name in iterable:
    # for block

this part is okay

then:
    # only if not exited with break

I only know how the "then" clause works if you include that comment
each and every time!

else:
    # only if iterable is empty

Again. I need more info before this code becomes intuitive. Too much
guessing is required. Not to mention that the try/except/else suite
treats "else" differently.

try:
do_this()
except EXCEPTION:
recover()
else NO_EXCEPTION:
okay_do_this_also().

for x in iterable:
do_this()
except EXCEPTION:
recover()
else NO_EXCEPTION:
do_this_also()

while LOOPING:
do_this()
except EXCEPTION:
break or recover()
else NO_EXCEPTION:
do_this_also()

In this manner "else" will behave consistently between exception
handling and looping.

But this whole idea of using an else clause is ridiculous anyway
because all you've done is to "break up" the code visually. Not to
mention; breaking the cognitive flow of a reader!

try:
do_this()
okay_do_this_also()
what_the_heck.do_this_too()
except EXCEPTION:
recover()
finally:
always_do_this()

Loop syntax can drop the "else" and adopt "then/finally" -- if you
think we even need a finally!?!?

for x in iterable:
do_this()
except EXCEPTION:
recover()
then:
okay_do_this_also()
what_the_heck.do_this_too()
finally:
always_do_this()

while LOOPING:
do_this()
except EXCEPTION:
recover()
then:
okay_do_this_also()
what_the_heck.do_this_too()
finally:
always_do_this()

 

[Chris Angelico]

I like this syntax better than the current syntax, however, it is
STILL far too confusing!

Absolutely, it's FAR too confusing. Every syntactic structure should
have the addition of a "foo:" suite, which will run when the
programmer expects it to and no other time. This would solve a LOT of
problems.

ChrisA

 

[Steven D'Aprano]

Every syntactic structure should
have the addition of a "foo:" suite, which will run when the programmer
expects it to and no other time. This would solve a LOT of problems.

Indeed, when I design my killer language, the identifiers "foo"
and "bar" will be reserved words, never used, and not even
mentioned in the reference manual. Any program using one will
simply dump core without comment. Multitudes will rejoice.
-- Tim Peters, 29 Apr 1998