# List comprehension + lambdas - strange behaviour

Discussion in 'Python' started by Artur Siekielski, May 6, 2010.

1. ### Artur SiekielskiGuest

Hello.
I found this strange behaviour of lambdas, closures and list
comprehensions:

>>> funs = [lambda: x for x in range(5)]
>>> [f() for f in funs]

[4, 4, 4, 4, 4]

Of course I was expecting the list [0, 1, 2, 3, 4] as the result. The
'x' was bound to the final value of 'range(5)' expression for ALL
defined functions. Can you explain this? Is this only counterintuitive
example or an error in CPython?

Regards,
Artur

Artur Siekielski, May 6, 2010

2. ### Raymond HettingerGuest

On May 6, 9:34 pm, Artur Siekielski <>
wrote:
> Hello.
> I found this strange behaviour of lambdas, closures and list
> comprehensions:
>
> >>> funs = [lambda: x for x in range(5)]
> >>> [f() for f in funs]

>
> [4, 4, 4, 4, 4]
>
> Of course I was expecting the list [0, 1, 2, 3, 4] as the result. The
> 'x' was bound to the final value of 'range(5)' expression for ALL
> defined functions. Can you explain this? Is this only counterintuitive
> example or an error in CPython?

Try binding the value of x for each of the inner functions:

>>> funs = [lambda x=x: x for x in range(5)]
>>> [f() for f in funs]

[0, 1, 2, 3, 4]

Otherwise, the 'x' is just a global value and the lambdas look it up
at when the function is invoked. Really, not surprising at all:

>>> x = 10
>>> def f():

.... return x
....
>>> x = 20
>>> f()

20

Raymond

Raymond Hettinger, May 6, 2010

3. ### Emile van SebilleGuest

On 5/6/2010 12:34 PM Artur Siekielski said...
> Hello.
> I found this strange behaviour of lambdas, closures and list
> comprehensions:
>
>>>> funs = [lambda: x for x in range(5)]

funs is now a list of lambda functions that return 'x' (whatever it
currently is from whereever it's accessible when invoked)

>>> [f() for f,x in zip(funs,range(5))]

[0, 1, 2, 3, 4]
>>> del x
>>> [f() for f in funs]

Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "<stdin>", line 1, in <lambda>
NameError: global name 'x' is not defined
>>>

Emile

Emile van Sebille, May 6, 2010
4. ### Benjamin PetersonGuest

Artur Siekielski <artur.siekielski <at> gmail.com> writes:
>
> Of course I was expecting the list [0, 1, 2, 3, 4] as the result. The
> 'x' was bound to the final value of 'range(5)' expression for ALL
> defined functions. Can you explain this? Is this only counterintuitive
> example or an error in CPython?

The former. Closures are rebound in a loop.

Benjamin Peterson, May 6, 2010
5. ### Terry ReedyGuest

On 5/6/2010 3:34 PM, Artur Siekielski wrote:
> Hello.
> I found this strange behaviour of lambdas, closures and list
> comprehensions:
>
>>>> funs = [lambda: x for x in range(5)]
>>>> [f() for f in funs]

> [4, 4, 4, 4, 4]

You succumbed to lambda hypnosis, a common malady ;-).
The above will not work in 3.x, which does not leak comprehension
iteration variables. It is equivalent to

funs = [lambda: x for y in range(5)]
del y # only for 2.x. y is already gone in 3.x
x = 4
[f() for f in funs]

Now, I am sure, you would expect what you got.

and nearly equivalent to

def f(): return x
x=8
funs = [f for x in range(5)]
[f() for f in funs]

# [8,8,8,8,8] in 3.x

Ditto

Terry Jan Reedy

Terry Reedy, May 7, 2010
6. ### Neil CeruttiGuest

On 2010-05-07, Terry Reedy <> wrote:
> On 5/6/2010 3:34 PM, Artur Siekielski wrote:
>> Hello.
>> I found this strange behaviour of lambdas, closures and list
>> comprehensions:
>>
>>>>> funs = [lambda: x for x in range(5)]
>>>>> [f() for f in funs]

>> [4, 4, 4, 4, 4]

>
> You succumbed to lambda hypnosis, a common malady ;-). The
> above will not work in 3.x, which does not leak comprehension
> iteration variables.

It functions the same in 3.1.

Python 3.1.1 (r311:74483, Aug 17 2009, 17:02:12) [MSC v.1500 32 bit (Intel)] on
win32
>>> funs = [lambda: x for x in range(5)]
>>> [f() for f in funs]

[4, 4, 4, 4, 4]

--
Neil Cerutti
*** Your child was bitten by a Bat-Lizard. ***

Neil Cerutti, May 7, 2010
7. ### Terry ReedyGuest

On 5/7/2010 8:31 AM, Neil Cerutti wrote:
> On 2010-05-07, Terry Reedy<> wrote:
>> On 5/6/2010 3:34 PM, Artur Siekielski wrote:
>>> Hello.
>>> I found this strange behaviour of lambdas, closures and list
>>> comprehensions:
>>>
>>>>>> funs = [lambda: x for x in range(5)]
>>>>>> [f() for f in funs]
>>> [4, 4, 4, 4, 4]

>>
>> You succumbed to lambda hypnosis, a common malady ;-). The
>> above will not work in 3.x, which does not leak comprehension
>> iteration variables.

>
> It functions the same in 3.1.
>
> Python 3.1.1 (r311:74483, Aug 17 2009, 17:02:12) [MSC v.1500 32 bit (Intel)] on
> win32
>>>> funs = [lambda: x for x in range(5)]
>>>> [f() for f in funs]

> [4, 4, 4, 4, 4]

Ok.

>>> x

Traceback (most recent call last):
File "<pyshell#1>", line 1, in <module>
x
NameError: name 'x' is not defined
#only in 3.x

But because the list comp is implemented in 3.x as an anonymous
function, which is then called and discarded (an implementation that I
believe is not guaranteed by the language ref), the lambda expression
defines a nested function which captures the (final) value of x.

>>> funs[0].__closure__[0].cell_contents

4

So it works (runs without exception), but somewhat accidentally and for
a different reason than in 2.x, where 'x' is 4 at the global level.

Terry Jan Reedy

Terry Reedy, May 7, 2010