lambda functions within list comprehensions

[Max Rybinsky]

Hello!

Please take a look at the example.

a = [(x, y) for x, y in map(None, range(10), range(10))] # Just a list of tuples
a

[(0, 0), (1, 1), (2, 2), (3, 3), (4, 4), (5, 5), (6, 6), (7, 7), (8,
8), (9, 9)]

Now i want to get a list of functions x*y/n, for each (x, y) in a:

funcs = [lambda n: x * y / n for x, y in a]

It looks consistent!
[<function <lambda> at 0x010F3DF0>, <function <lambda> at 0x010F7CF0>,
<function <lambda> at 0x010F7730>, <function <lambda> at 0x010FD270>,
<function <lambda> at 0x010FD0B0>, <function <lambda> at 0x010FD5B0>,
<function <lambda> at 0x010FD570>, <function <lambda> at 0x010FD630>,

funcs[0](1)

[/QUOTE]
81

But they aren't!
.... print func(1)
....
81
81
81
81
81
81
81
81
81
81

It seems, all functions have x and y set to 9.
What's wrong with it? Is it a bug?

.... return lambda n: x * y / n
....

funcs = [buldFunc(x, y) for x, y in a]

.... and it does work!

But why not to save several lines of code?

Thanks in advance.

 

[Alex Martelli]

funcs = [lambda n: x * y / n for x, y in a]

...
It seems, all functions have x and y set to 9.
What's wrong with it? Is it a bug?

It's known as *late binding*: names x and y are looked up when the
lambda's body is executing, and at that time they're both set to the
value 9. You appear to have expected *early binding*, with the names
being somehow looked up at the time the lambda keyword executed, but
that's just not Python semantics (and would interfere with many other
cases where late binding is exactly what one wants).

You've already indicated what's probably the best solution -- a factory
function instead of the lambda. There are other ways to request early
binding, and since you appear to value compactness over clarity the most
compact way is probably:

funcs = [lambda n, x=x, y=y: x*y/n for x, y in a]

it's not perfect, because the resulting functions can take up to 3
arguments, so that if you called funcs[1](2,3) you'd get an unwanted
result rather than a TypeError exception. If you're keen on getting the
exception in such cases, you can use a lambda factory in the same role
as the much clearer and more readable factory function you had (which I
keep thinking is the _sensible_ solution)...:

funcs = [ (lambda x,y: lambda n: x*y/n)(x,y) for x,y in a ]


Alex

 

[Max Rybinsky]

Thank you for explanation, Alex.
It appears that almost every beginner to Python gets in trouble with
this ...feature.

 

[Alex Martelli]

Thank you for explanation, Alex.
It appears that almost every beginner to Python gets in trouble with
this ...feature.

Almost every beginner to Python gets in trouble by expecting "do what
I'm thinking of RIGHT NOW"-binding, which no language offers: in other
words, such beginners sometimes expect late binding where Python binds
early, and, vice versa, they at other times expect early binding where
Python binds late. Not ALWAYS, mind you -- what they expect depends on
what would appear to them to be most convenient for their immediate
needs on each separate occasion. Some other languages try to follow
beginners and offer "do what I mean" semantics -- when using such
languages, one ends up in a battle of wit against the compiler's guesses
about one's intentions. Python instead offers extremely simple rules,
such as: any name is looked up each and every time it's evaluated (and
at no other times); evaluation of function headers happens completely at
the time the 'def' or 'lambda' evaluates, while evaluation of function
bodies happens completely at the time the function is _called_. By
learning and applying such simple rules there can be no surprise about
what is evaluated (and, in particular, looked up) when. E.g., consider
the difference between the following two functions:

def early(x=whatever()):
...

def late():
x=whatever()
...

In 'early', the call to whatever() is part of the function's header, and
therefore happens at the time the 'def' statement executes -- and thus
name 'whatever' means whatever it means at THAT time (if at that time
it's not bound to anything, the 'def' statement fails with an
exception).

In 'late', the call to whatever() is part of the function's body, and
therefore happens each time the function is called -- and thus name
'whatever' means whatever it means at THAT time (if at that time it's
not bound to anything, the call fails with an exception).


Alex

 

[Max Rybinsky]

OK.
The thing i've got is an obscure semantic bug, occured because of my
unawareness of the following Python "features":
1. (In major)
http://mail.python.org/pipermail/python-dev/2005-September/056508.html
2. "late" bindings of the function's body

Got to know!

Thanks for your attention.

 

[Max Rybinsky]

Valid link in my previews message is

http://mail.python.org/pipermail/python-dev/2005-September/056669.html

Sorry.