List comprehension + lambdas - strange behaviour

[Artur Siekielski]

Hello.
I found this strange behaviour of lambdas, closures and list
comprehensions:

funs = [lambda: x for x in range(5)]
[f() for f in funs]

[4, 4, 4, 4, 4]

Of course I was expecting the list [0, 1, 2, 3, 4] as the result. The
'x' was bound to the final value of 'range(5)' expression for ALL
defined functions. Can you explain this? Is this only counterintuitive
example or an error in CPython?


Regards,
Artur

 

[Raymond Hettinger]

Hello.
I found this strange behaviour of lambdas, closures and list
comprehensions:

funs = [lambda: x for x in range(5)]
[f() for f in funs]

[4, 4, 4, 4, 4]

Of course I was expecting the list [0, 1, 2, 3, 4] as the result. The
'x' was bound to the final value of 'range(5)' expression for ALL
defined functions. Can you explain this? Is this only counterintuitive
example or an error in CPython?

Try binding the value of x for each of the inner functions:

funs = [lambda x=x: x for x in range(5)]
[f() for f in funs]

[0, 1, 2, 3, 4]

Otherwise, the 'x' is just a global value and the lambdas look it up
at when the function is invoked. Really, not surprising at all:
.... return x
....20


Raymond

 

[Emile van Sebille]

On 5/6/2010 12:34 PM Artur Siekielski said...

Hello.
I found this strange behaviour of lambdas, closures and list
comprehensions:

funs = [lambda: x for x in range(5)]

funs is now a list of lambda functions that return 'x' (whatever it
currently is from whereever it's accessible when invoked)

>>> [f() for f,x in zip(funs,range(5))] [0, 1, 2, 3, 4]
>>> del x
>>> [f() for f in funs]

Traceback (most recent call last):
File "<stdin>", line 1, in ?

Emile

 

[Benjamin Peterson]

Of course I was expecting the list [0, 1, 2, 3, 4] as the result. The
'x' was bound to the final value of 'range(5)' expression for ALL
defined functions. Can you explain this? Is this only counterintuitive
example or an error in CPython?

The former. Closures are rebound in a loop.

 

[Terry Reedy]

Hello.
I found this strange behaviour of lambdas, closures and list
comprehensions:

funs = [lambda: x for x in range(5)]
[f() for f in funs]

[4, 4, 4, 4, 4]

You succumbed to lambda hypnosis, a common malady ;-).
The above will not work in 3.x, which does not leak comprehension
iteration variables. It is equivalent to

funs = [lambda: x for y in range(5)]
del y # only for 2.x. y is already gone in 3.x
x = 4
[f() for f in funs]

Now, I am sure, you would expect what you got.

and nearly equivalent to

def f(): return x
x=8
funs = [f for x in range(5)]
[f() for f in funs]

# [8,8,8,8,8] in 3.x

Ditto

Terry Jan Reedy

 

[Neil Cerutti]

Hello.
I found this strange behaviour of lambdas, closures and list
comprehensions:

funs = [lambda: x for x in range(5)]
[f() for f in funs]

[4, 4, 4, 4, 4]

You succumbed to lambda hypnosis, a common malady ;-). The
above will not work in 3.x, which does not leak comprehension
iteration variables.

It functions the same in 3.1.

Python 3.1.1 (r311:74483, Aug 17 2009, 17:02:12) [MSC v.1500 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.

funs = [lambda: x for x in range(5)]
[f() for f in funs]

[4, 4, 4, 4, 4]

 

[Terry Reedy]

Hello.
I found this strange behaviour of lambdas, closures and list
comprehensions:

funs = [lambda: x for x in range(5)]
[f() for f in funs]
[4, 4, 4, 4, 4]

You succumbed to lambda hypnosis, a common malady ;-). The
above will not work in 3.x, which does not leak comprehension
iteration variables.

It functions the same in 3.1.

Python 3.1.1 (r311:74483, Aug 17 2009, 17:02:12) [MSC v.1500 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.

funs = [lambda: x for x in range(5)]
[f() for f in funs]

[4, 4, 4, 4, 4]
Ok.

Traceback (most recent call last):
File "<pyshell#1>", line 1, in <module>
x
NameError: name 'x' is not defined
#only in 3.x

But because the list comp is implemented in 3.x as an anonymous
function, which is then called and discarded (an implementation that I
believe is not guaranteed by the language ref), the lambda expression
defines a nested function which captures the (final) value of x.

>>> funs[0].__closure__[0].cell_contents

4

So it works (runs without exception), but somewhat accidentally and for
a different reason than in 2.x, where 'x' is 4 at the global level.

Terry Jan Reedy