Functional programming gotcha

[Ed Schofield]

Hi all,

I find this strange:

flist = []
for i in range(3):

.... f = lambda x: x+i
.... flist.append(f)
....

[f(1) for f in flist] [3, 3, 3]

What I expect is:

[f(1) for f in flist] [1,2,3]

Is this a bug in Python? It happens on my builds of
Python 2.3 and 2.2.3.

Replacing the lambda function by a named function, as in

flist = []
for i in range(3):
def f(x):
return x + i
flist.append(f)

[f(1) for f in flist]

gives the same result.

I have a workaround (of sorts) adapted from the Python tutorial:
.... return lambda x: x + n
....

flist = [make_incrementor(i) for i in range(3)]

[f(1) for f in flist] [1, 2, 3]

but I'd prefer the flexibility of the first approach. Any ideas?

Any explanations for why Python does this? Any justifications for why
Python _should_ do this? Who believes this is a bug?

Kind regards,
Ed Schofield

 

[David C. Fox]

Hi all,

I find this strange:

flist = []
for i in range(3):

... f = lambda x: x+i
... flist.append(f)
...

[f(1) for f in flist]

[3, 3, 3]


What I expect is:

[f(1) for f in flist]

[1,2,3]


Is this a bug in Python? It happens on my builds of
Python 2.3 and 2.2.3.

I guess the nested scopes changes in Python 2.1/2.2 (see
http://www.python.org/peps/pep-0227.html) apply to functions but not
loops. You can use the same workaround that people used to use because
of the lack of nested scopes:

f = lambda x, i = i: x + i

That way, the second, optional parameter has a default value which is
evaluated at the time the lambda is created.

Replacing the lambda function by a named function, as in

flist = []
for i in range(3):
def f(x):
return x + i
flist.append(f)

[f(1) for f in flist]

gives the same result.

I have a workaround (of sorts) adapted from the Python tutorial:


... return lambda x: x + n
...

flist = [make_incrementor(i) for i in range(3)]

[f(1) for f in flist]

[1, 2, 3]


but I'd prefer the flexibility of the first approach. Any ideas?

Any explanations for why Python does this? Any justifications for why
Python _should_ do this? Who believes this is a bug?

Kind regards,
Ed Schofield

 

[Robin Becker]

Hi all,

I find this strange:

flist = []
for i in range(3):

... f = lambda x: x+i

try using f = lambda x,i=i: x+i ie bind at the definition point

... flist.append(f)
...

[f(1) for f in flist] [3, 3, 3]

What I expect is:

[f(1) for f in flist] [1,2,3]

Is this a bug in Python? It happens on my builds of
Python 2.3 and 2.2.3.

Replacing the lambda function by a named function, as in

flist = []
for i in range(3):
def f(x):
return x + i
flist.append(f)

[f(1) for f in flist]

gives the same result.

I have a workaround (of sorts) adapted from the Python tutorial:
... return lambda x: x + n
...

flist = [make_incrementor(i) for i in range(3)]

[f(1) for f in flist] [1, 2, 3]

but I'd prefer the flexibility of the first approach. Any ideas?

Any explanations for why Python does this? Any justifications for why
Python _should_ do this? Who believes this is a bug?

Kind regards,
Ed Schofield

 

[David C. Fox]

Hi all,

I find this strange:

flist = []
for i in range(3):

... f = lambda x: x+i
... flist.append(f)
...

[f(1) for f in flist]

[3, 3, 3]


What I expect is:

[f(1) for f in flist]

[1,2,3]


Is this a bug in Python? It happens on my builds of
Python 2.3 and 2.2.3.

I guess the nested scopes changes in Python 2.1/2.2 (see
http://www.python.org/peps/pep-0227.html) apply to functions but not
loops.

I take back that explanation. The problem is that the i in the lambda
*does* refer to the local variable i, whose value changes as you go
through the loop, not to a new variable whose value is equal to the
value of i at the time when the lambda was created.

The workaround is still the same:

loops. You can use the same workaround that people used to use because
of the lack of nested scopes:

f = lambda x, i = i: x + i

That way, the second, optional parameter has a default value which is
evaluated at the time the lambda is created.

David

 

[Gregor Lingl]

David C. Fox schrieb:
.....

I guess the nested scopes changes in Python 2.1/2.2 (see
http://www.python.org/peps/pep-0227.html) apply to functions but not
loops. You can use the same workaround that people used to use because
of the lack of nested scopes:

f = lambda x, i = i: x + i

That way, the second, optional parameter has a default value which is
evaluated at the time the lambda is created.

Isn't this the intended behaviour of nested scopes (regardless
of loops or not)?
i = 1
def f():
print i
f()
i+=1
f()

1
2

Gregor

 

[Terry Reedy]

Hi all,

I find this strange:

flist = []
for i in range(3):

... f = lambda x: x+i
... flist.append(f)
...

[f(1) for f in flist] [3, 3, 3]

What I expect is:

[f(1) for f in flist]

[1,2,3]

Others have given the solution, but let's think about the why of it.
If a function in a dyanamic language (one that allows runtime
construction of functions) has a free variable (one not set within the
function), the question arises "When do we get (capture) the value of
the variable? At definition/construction time or at call/execution
time? Or, to put it another way, what do we capture? The (current)
value of the variable or its name (for later evaluation). Either
could be what we want.

Python lets you choose which capture method (evaluation time) you
want. Do nothing to get the name captured for later evaluation. Or
put v=v in the arglist to capture the current value and give it the
same name (w=v, to use a new name, is legal too, of course).

Terry J. Reedy

 

[Georgy Pruss]

<...>

Others have given the solution, but let's think about the why of it.
If a function in a dyanamic language (one that allows runtime
construction of functions) has a free variable (one not set within the
function), the question arises "When do we get (capture) the value of
the variable? At definition/construction time or at call/execution
time? Or, to put it another way, what do we capture? The (current)
value of the variable or its name (for later evaluation). Either
could be what we want.

Python lets you choose which capture method (evaluation time) you
want. Do nothing to get the name captured for later evaluation. Or
put v=v in the arglist to capture the current value and give it the
same name (w=v, to use a new name, is legal too, of course).

Terry J. Reedy

Very good and simple explanation! Thank you.

Georgy