# Functional programming gotcha

Discussion in 'Python' started by Ed Schofield, Oct 24, 2003.

1. ### Ed SchofieldGuest

Hi all,

I find this strange:

>>> flist = []
>>> for i in range(3):

.... f = lambda x: x+i
.... flist.append(f)
....
>>> [f(1) for f in flist]

[3, 3, 3]
>>>

What I expect is:

>>> [f(1) for f in flist]

[1,2,3]
>>>

Is this a bug in Python? It happens on my builds of
Python 2.3 and 2.2.3.

Replacing the lambda function by a named function, as in

flist = []
for i in range(3):
def f(x):
return x + i
flist.append(f)

[f(1) for f in flist]

gives the same result.

I have a workaround (of sorts) adapted from the Python tutorial:

>>> def make_incrementor(n):

.... return lambda x: x + n
....
>>> flist = [make_incrementor(i) for i in range(3)]
>>>
>>> [f(1) for f in flist]

[1, 2, 3]
>>>

but I'd prefer the flexibility of the first approach. Any ideas?

Any explanations for why Python does this? Any justifications for why
Python _should_ do this? Who believes this is a bug?

Kind regards,
Ed Schofield

Ed Schofield, Oct 24, 2003

2. ### David C. FoxGuest

Ed Schofield wrote:

> Hi all,
>
> I find this strange:
>
>
>>>>flist = []
>>>>for i in range(3):

>
> ... f = lambda x: x+i
> ... flist.append(f)
> ...
>
>>>>[f(1) for f in flist]

>
> [3, 3, 3]
>
>
> What I expect is:
>
>
>>>>[f(1) for f in flist]

>
> [1,2,3]
>
>
> Is this a bug in Python? It happens on my builds of
> Python 2.3 and 2.2.3.

I guess the nested scopes changes in Python 2.1/2.2 (see
http://www.python.org/peps/pep-0227.html) apply to functions but not
loops. You can use the same workaround that people used to use because
of the lack of nested scopes:

f = lambda x, i = i: x + i

That way, the second, optional parameter has a default value which is
evaluated at the time the lambda is created.

>
> Replacing the lambda function by a named function, as in
>
> flist = []
> for i in range(3):
> def f(x):
> return x + i
> flist.append(f)
>
> [f(1) for f in flist]
>
> gives the same result.
>
> I have a workaround (of sorts) adapted from the Python tutorial:
>
>
>>>>def make_incrementor(n):

>
> ... return lambda x: x + n
> ...
>
>>>>flist = [make_incrementor(i) for i in range(3)]
>>>>
>>>>[f(1) for f in flist]

>
> [1, 2, 3]
>
>
> but I'd prefer the flexibility of the first approach. Any ideas?
>
> Any explanations for why Python does this? Any justifications for why
> Python _should_ do this? Who believes this is a bug?
>
> Kind regards,
> Ed Schofield
>

David C. Fox, Oct 24, 2003

3. ### Robin BeckerGuest

In article <>, Ed
Schofield <> writes
>
>Hi all,
>
>I find this strange:
>
>>>> flist = []
>>>> for i in range(3):

>... f = lambda x: x+i

try using f = lambda x,i=i: x+i ie bind at the definition point

>... flist.append(f)
>...
>>>> [f(1) for f in flist]

>[3, 3, 3]
>>>>

>
>What I expect is:
>
>>>> [f(1) for f in flist]

>[1,2,3]
>>>>

>
>Is this a bug in Python? It happens on my builds of
>Python 2.3 and 2.2.3.
>
>Replacing the lambda function by a named function, as in
>
>flist = []
>for i in range(3):
> def f(x):
> return x + i
> flist.append(f)
>
>[f(1) for f in flist]
>
>gives the same result.
>
>I have a workaround (of sorts) adapted from the Python tutorial:
>
>>>> def make_incrementor(n):

>... return lambda x: x + n
>...
>>>> flist = [make_incrementor(i) for i in range(3)]
>>>>
>>>> [f(1) for f in flist]

>[1, 2, 3]
>>>>

>
>but I'd prefer the flexibility of the first approach. Any ideas?
>
>Any explanations for why Python does this? Any justifications for why
>Python _should_ do this? Who believes this is a bug?
>
>Kind regards,
>Ed Schofield
>

--
Robin Becker

Robin Becker, Oct 24, 2003
4. ### David C. FoxGuest

David C. Fox wrote:

> Ed Schofield wrote:
>
>> Hi all,
>>
>> I find this strange:
>>
>>
>>>>> flist = []
>>>>> for i in range(3):

>>
>>
>> ... f = lambda x: x+i
>> ... flist.append(f)
>> ...
>>
>>>>> [f(1) for f in flist]

>>
>>
>> [3, 3, 3]
>>
>>
>> What I expect is:
>>
>>
>>>>> [f(1) for f in flist]

>>
>>
>> [1,2,3]
>>
>>
>> Is this a bug in Python? It happens on my builds of
>> Python 2.3 and 2.2.3.

>
>
> I guess the nested scopes changes in Python 2.1/2.2 (see
> http://www.python.org/peps/pep-0227.html) apply to functions but not
> loops.

I take back that explanation. The problem is that the i in the lambda
*does* refer to the local variable i, whose value changes as you go
through the loop, not to a new variable whose value is equal to the
value of i at the time when the lambda was created.

The workaround is still the same:

> loops. You can use the same workaround that people used to use because
> of the lack of nested scopes:
>
> f = lambda x, i = i: x + i
>
> That way, the second, optional parameter has a default value which is
> evaluated at the time the lambda is created.
>

David

David C. Fox, Oct 24, 2003
5. ### Gregor LinglGuest

David C. Fox schrieb:
.....

>
> I guess the nested scopes changes in Python 2.1/2.2 (see
> http://www.python.org/peps/pep-0227.html) apply to functions but not
> loops. You can use the same workaround that people used to use because
> of the lack of nested scopes:
>
> f = lambda x, i = i: x + i
>
> That way, the second, optional parameter has a default value which is
> evaluated at the time the lambda is created.
>

Isn't this the intended behaviour of nested scopes (regardless
of loops or not)?

>>> def fun():

i = 1
def f():
print i
f()
i+=1
f()

>>> fun()

1
2
>>>

Gregor

Gregor Lingl, Oct 24, 2003
6. ### Terry ReedyGuest

"Ed Schofield" <> wrote in message
news...
>
> Hi all,
>
> I find this strange:
>
> >>> flist = []
> >>> for i in range(3):

> ... f = lambda x: x+i
> ... flist.append(f)
> ...
> >>> [f(1) for f in flist]

> [3, 3, 3]
> >>>

>
> What I expect is:
>
> >>> [f(1) for f in flist]

> [1,2,3]

Others have given the solution, but let's think about the why of it.
If a function in a dyanamic language (one that allows runtime
construction of functions) has a free variable (one not set within the
function), the question arises "When do we get (capture) the value of
the variable? At definition/construction time or at call/execution
time? Or, to put it another way, what do we capture? The (current)
value of the variable or its name (for later evaluation). Either
could be what we want.

Python lets you choose which capture method (evaluation time) you
want. Do nothing to get the name captured for later evaluation. Or
put v=v in the arglist to capture the current value and give it the
same name (w=v, to use a new name, is legal too, of course).

Terry J. Reedy

Terry Reedy, Oct 25, 2003
7. ### Georgy PrussGuest

"Terry Reedy" <> wrote in message news:...
>
> <...>
>
> Others have given the solution, but let's think about the why of it.
> If a function in a dyanamic language (one that allows runtime
> construction of functions) has a free variable (one not set within the
> function), the question arises "When do we get (capture) the value of
> the variable? At definition/construction time or at call/execution
> time? Or, to put it another way, what do we capture? The (current)
> value of the variable or its name (for later evaluation). Either
> could be what we want.
>
> Python lets you choose which capture method (evaluation time) you
> want. Do nothing to get the name captured for later evaluation. Or
> put v=v in the arglist to capture the current value and give it the
> same name (w=v, to use a new name, is legal too, of course).
>
> Terry J. Reedy
>

Very good and simple explanation! Thank you.

Georgy

Georgy Pruss, Oct 25, 2003

### Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.