lambda in list comprehension acting funny

[Daniel Fetchinson]

funcs = [ lambda x: x**i for i in range( 5 ) ]
print funcs[0]( 2 )
print funcs[1]( 2 )
print funcs[2]( 2 )

This gives me

16
16
16

When I was excepting

1
2
4

Does anyone know why?

Cheers,
Daniel

 

[Colin J. Williams]

funcs = [ lambda x: x**i for i in range( 5 ) ]
print funcs[0]( 2 )
print funcs[1]( 2 )
print funcs[2]( 2 )

This gives me

16
16
16

When I was excepting

1
2
4

Does anyone know why?

Cheers,
Daniel

I don't understand why you would expect 1, 2, 4.

Perhaps parentheses will help the order of evaluation:

funcs = [(lambda x: x**i) for i in range( 5 )]

This gives:
1
16
81

Colin W.

 

[Ian Kelly]

I don't understand why you would expect 1, 2, 4.

Because:

funcs[0](2) == 2 ** 0 == 1
funcs[1](2) == 2 ** 1 == 2
funcs[2](2) == 2 ** 2 == 4

Perhaps parentheses will help the order of evaluation:

funcs = [(lambda x: x**i) for i in range( 5 )]

This gives:
1
16
81

No, that gives 16, 16, 16 just like the original. I don't understand
why you would expect 1, 16, 81, unless you have misread the code.

 

[woooee]

You should not be using lambda in this case
..for x in [2, 3]:
.. funcs = [x**ctr for ctr in range( 5 )]
.. for p in range(5):
.. print x, funcs[p]
.. print

 

[John Ladasky]

Exactly. It's threads like these which remind me why I never use lambda. I would rather give a function an explicit name and adhere to the familiar Python syntax, despite the two extra lines of code. I don't even like the name "lambda". It doesn't tell you what it is (unless you're John McCarthy), a function that you won't re-use and so you don't really need to give ita persistent name.

I haven't seen any lambdas in any Python library code, or in any of the third-party modules I use (numpy, matplotlib, Biopython). Do they exist? Because I have not been forced to do so, I haven't retained a space in the topdrawer of my programming brain for lambda.

I know the historical reason that Python ended up with lambda, it was requested by people in the Lisp community. While I appreciate some of the Lisp-like features which did find their way into Python (especially being able to treat code as data, and functions as objects), I've found that lambda does nothing for me.

 

[Hans Mulder]

You should not be using lambda in this case
.for x in [2, 3]:
. funcs = [x**ctr for ctr in range( 5 )]
. for p in range(5):
. print x, funcs[p]
. print

The list is called "funcs" because it is meant to contain functions.
Your code does not put functions in the list, so it doesn't do what
he wants.

He could do:

funcs = []
for i in range(5):
def f(x):
return x**i
funcs.append(f)
print funcs[0]( 2 )
print funcs[1]( 2 )
print funcs[2]( 2 )

..... and it will print 16, 16, 16 for the same reason as the lambda
version. On the other hand, this does what he wants:

funcs = []
for i in range(5):
def f(x, i=i):
return x**i
funcs.append(f)
print funcs[0]( 2 )
print funcs[1]( 2 )
print funcs[2]( 2 )

The lambda keyword is a red herring. The question is really about
functions, and how they handle non-local variables. The good news
is that 'lambda' and 'def' work exactly the same in that regards.
So once you understand the finer points about 'def', you no longer
have a reason to avoid 'lambda'.

Hope this helps,

-- HansM

 

[Steven D'Aprano]

You should not be using lambda in this case
.for x in [2, 3]:
. funcs = [x**ctr for ctr in range( 5 )]
. for p in range(5):
. print x, funcs[p]
. print

If you change the requirements, it's always easy to solve problems. But
it is the wrong problem that you have solved.

The problem we have been asked to solve is to create a sequence of
function objects, so that they can be called later, when needed, *not* to
pre-calculate the results.

In this case, the most obvious solution is to store a local variable in
each function object with the value you want.

funcs = [(lambda x, i=i: x**i) for i in range(5)]

creates a list of five functions:

lambda x, i=0: x**i
lambda x, i=1: x**i
lambda x, i=2: x**i
and so on.

In this case, each function has two local variables, x and i, with i
having a default value. The function parameter i is bound to the value of
i when the function was created.

Because parameter defaults are calculated once, when the function is
created, this causes the value of i to stick to the newly created
function, and we get the result we need.

What happens if you don't use a parameter with a default value?

funcs = [(lambda x: x**i) for i in range(5)]

In this case, each function body contains one local variable, x, and one
non-local or global variable, i.

Because i is a non-local, the function doesn't store a value for it.
Instead, the function stores a lump of data pointing to just enough of
the environment to fetch the current value of the non-local i when
needed. Since all five functions are in the same environment, they all
see the same value of i when you call them, regardless of what the value
of i was when they were created.

This is little different from doing this:

i = 1
def f1(x): return x**i

i = 2
def f2(x): return x**i

i = 3
def f3(x): return x**i

Is there any surprise that all three functions return the same value?
They all point to the same global variable i. I'm not sure what it is
about lambda that fools people into thinking that it is different (I've
even been fooled myself!) but it is not.

 

[Steven D'Aprano]

Exactly. It's threads like these which remind me why I never use
lambda. I would rather give a function an explicit name and adhere to
the familiar Python syntax, despite the two extra lines of code.

lambda is familiar Python syntax, or at least it should be if you
consider yourself an expert Python programmer.

And besides, this is not a problem with lambda. You get the *exact* same
problem with "ordinary" function definitions. Of course you do -- lambdas
*are* ordinary functions, they just have a more compact syntax.

funcs = []
for i in range(5):
def pwr(x):
return x**i
print("Testing 3**%d:" % i, pwr(3))
funcs.append(pwr)

for j in range(5):
print("Expected", 3**j, "got", funcs[j](3))

gives you *exactly* the same issue as with the lambda. This is not a
problem with lambdas, this is a scoping issue.

I don't even like the name "lambda". It doesn't tell you what it is

Whereas "def" does?

If you can learn that "def" defines a function (as opposed to a class, a
module, or anything else really) than it's not that hard to learn that
"lambda" also defines a function.

Although I do wonder whether people would be less scared of lambda, and
less likely to imagine that it creates something with strange mysterious
properties different from "regular functions", if the keyword was called
"function" instead of lambda?

(unless you're John McCarthy), a function that you won't re-use and so
you don't really need to give it a persistent name.

Whether or not a function has a persistent name has nothing to do with
reuse. Anonymous functions can be reused. The driving motivation for
lambdas is for callback functions, which are frequently reused, but you
don't need or want to fill your global namespace up with potentially
hundreds of callback functions.

Lambda syntax is only incidentally good for saving a line or two. If the
only benefit of lambda was to save a line of code, that would be stupid.
The real reason for lambda is to avoid polluting your namespace with
unnecessary names.

Python allows us to easily and simply use anonymous strings, anonymous
ints, anonymous objects of all types. Why should functions be treated as
second-class?

I haven't seen any lambdas in any Python library code, or in any of the
third-party modules I use (numpy, matplotlib, Biopython). Do they
exist? Because I have not been forced to do so, I haven't retained a
space in the top drawer of my programming brain for lambda.

There are over 800 uses of lambda in the standard library:

[steve@ando ~]$ cd /usr/local/lib/python3.2
[steve@ando python3.2]$ grep lambda *.py */*.py | wc -l
829


(although some of the above are comments, doctests, etc.)

See, for example, configparser, decimal, functools, gettext, inspect,
pydoc, symtable, uuid, and others. pydoc and idlelib in particular make
heavy use of lambda, and the test suite is overflowing with them -- over
600 uses in total.

 

[Dennis Lee Bieber]

I know the historical reason that Python ended up with lambda, it was requested by people in the Lisp community. While I appreciate some of the Lisp-like features which did find their way into Python (especially being able to treat code as data, and functions as objects), I've found that lambda does nothing for me.

It can be useful when defining callbacks for GUIs that need a
predefined argument... eg; a calculator application: 10 buttons for 0-9.
Why define 10 separate callbacks when you only need one callback with a
parameter containing the numeric value for the button.

lambda x=#: number_button_callback(x)

where # is the (integral) value of the specific button.


{non sequitur: I still recall my archaic C++ class with the OOAD
assignment of designing said calculator -- we never had to implement
one, just design the basic classes/methods/attributes [on 3x5 cards] for
a four-banger. I managed to persuade the team I was on that an RPN
calculator would be simpler to (potentially) implement... And THEN
persuaded them to NOT use the equivalent of an ALU class, but to put the
math work into each operation button... Justification when presented to
class: one could turn that four-banger into a scientific [or financial]
calculator by just adding classes for the additional function buttons
[and of course, buttons on screen] rather than having to rewrite an
"ALU" class to understand all functions.}

 

[Daniel Fetchinson]

You should not be using lambda in this case

.for x in [2, 3]:
. funcs = [x**ctr for ctr in range( 5 )]
. for p in range(5):
. print x, funcs[p]
. print

If you change the requirements, it's always easy to solve problems. But
it is the wrong problem that you have solved.

The problem we have been asked to solve is to create a sequence of
function objects, so that they can be called later, when needed, *not* to
pre-calculate the results.

In this case, the most obvious solution is to store a local variable in
each function object with the value you want.

funcs = [(lambda x, i=i: x**i) for i in range(5)]

creates a list of five functions:

lambda x, i=0: x**i
lambda x, i=1: x**i
lambda x, i=2: x**i
and so on.

In this case, each function has two local variables, x and i, with i
having a default value. The function parameter i is bound to the value of
i when the function was created.

Because parameter defaults are calculated once, when the function is
created, this causes the value of i to stick to the newly created
function, and we get the result we need.

What happens if you don't use a parameter with a default value?

funcs = [(lambda x: x**i) for i in range(5)]

In this case, each function body contains one local variable, x, and one
non-local or global variable, i.

Because i is a non-local, the function doesn't store a value for it.
Instead, the function stores a lump of data pointing to just enough of
the environment to fetch the current value of the non-local i when
needed. Since all five functions are in the same environment, they all
see the same value of i when you call them, regardless of what the value
of i was when they were created.

This is little different from doing this:

i = 1
def f1(x): return x**i

i = 2
def f2(x): return x**i

i = 3
def f3(x): return x**i

Is there any surprise that all three functions return the same value?
They all point to the same global variable i. I'm not sure what it is
about lambda that fools people into thinking that it is different (I've
even been fooled myself!) but it is not.

Thank you Steve!
Precise and clear, as always!

Cheers,
Daniel

 

[88888 Dihedral]

On Wed, Jul 11, 2012 at 4:28 AM, Colin J. Williams <[email protected]> wrote:
> I don't understand why you would expect 1, 2, 4.

Because:

funcs[0](2) == 2 ** 0 == 1
funcs[1](2) == 2 ** 1 == 2
funcs[2](2) == 2 ** 2 == 4

> Perhaps parentheses will help the order of evaluation:
>
> funcs = [(lambda x: x**i) for i in range( 5 )]
>
> This gives:
> 1
> 16
> 81

No, that gives 16, 16, 16 just like the original. I don't understand
why you would expect 1, 16, 81, unless you have misread the code.

I'll contribute my way of python programming:

def powerb(x, b): #
return x**b


One functor is enough!

 

[88888 Dihedral]

On Wed, Jul 11, 2012 at 4:28 AM, Colin J. Williams <[email protected]> wrote:
> I don't understand why you would expect 1, 2, 4.

Because:

funcs[0](2) == 2 ** 0 == 1
funcs[1](2) == 2 ** 1 == 2
funcs[2](2) == 2 ** 2 == 4

> Perhaps parentheses will help the order of evaluation:
>
> funcs = [(lambda x: x**i) for i in range( 5 )]
>
> This gives:
> 1
> 16
> 81

No, that gives 16, 16, 16 just like the original. I don't understand
why you would expect 1, 16, 81, unless you have misread the code.

I'll contribute my way of python programming:

def powerb(x, b): #
return x**b


One functor is enough!

 

[Steven D'Aprano]

I'll contribute my way of python programming:

def powerb(x, b): #
return x**b

Here's a shorter version:

py> pow

One functor is enough!

Nothing we have been discussing in this thread has been a functor, either
in the Haskell sense or the C++ sense.

 

[Steven D'Aprano]

{non sequitur: I still recall my archaic C++ class with the OOAD
assignment of designing said calculator -- we never had to implement
one, just design the basic classes/methods/attributes [on 3x5 cards] for
a four-banger. I managed to persuade the team I was on that an RPN
calculator would be simpler to (potentially) implement... And THEN
persuaded them to NOT use the equivalent of an ALU class, but to put the
math work into each operation button...

"ALU class"?

Googling gives me no clue.

 

[Steven D'Aprano]

funcs = [ lambda x: x**i for i in range( 5 ) ]

Here's another solution:

from functools import partial
funcs = [partial(lambda i, x: x**i, i) for i in range(5)]


Notice that the arguments i and x are defined in the opposite order.
That's because partial only applies positional arguments from the left.
If there was a "right partial" that applies from the right, we could use
the built-in instead:

funcs = [rpartial(pow, i) for i in range(5)]

 

[Terry Reedy]

{non sequitur: I still recall my archaic C++ class with the OOAD
assignment of designing said calculator -- we never had to implement
one, just design the basic classes/methods/attributes [on 3x5 cards] for
a four-banger. I managed to persuade the team I was on that an RPN
calculator would be simpler to (potentially) implement... And THEN
persuaded them to NOT use the equivalent of an ALU class, but to put the
math work into each operation button...

"ALU class"?

Googling gives me no clue.

4th hit: Arithmetic Logic Unit (hardware) -- Wikipedia
ALU class -- software version or simulation thereof.

The hardware version of Dennis's distributed design would be
interesting. Spreadsheets are based in part on the same idea of
distributed logic and arithmetic.

 

[Dennis Lee Bieber]

"ALU class"?

Googling gives me no clue.

Arithmetic/Logic Unit

http://en.wikipedia.org/wiki/Arithmetic_logic_unit
http://en.wikipedia.org/wiki/74181
{diversion: http://en.wikipedia.org/wiki/VAX-11/780 -- incredible...
that used to be considered a "super-mini" when I worked on them; and now
would be shamed by most laptops except for the ability to support so
many users concurrently (let me know when a Windows laptop supports 32
VT-100 class connections <G>)}

 

[88888 Dihedral]

On Wed, 11 Jul 2012 20:39:45 -0700, 88888 Dihedral wrote:

&gt; I'll contribute my way of python programming:
&gt;
&gt; def powerb(x, b): #
&gt; return x**b

Here's a shorter version:

py&gt; pow
&lt;built-in function pow&gt;


&gt; One functor is enough!

Nothing we have been discussing in this thread has been a functor, either
in the Haskell sense or the C++ sense.

Well, I encountered this kind of problems before in OOP.

I have to make sure my functor to keep the state variable values
for different objects that call the same functor to behave correctly
in order to avoid passing extra parameters in various objects using the same functor.

 

[John O'Hagan]

Exactly. It's threads like these which remind me why I never use lambda. I
would rather give a function an explicit name and adhere to the familiar
Python syntax, despite the two extra lines of code. I don't even like the
name "lambda". It doesn't tell you what it is (unless you're John McCarthy),
a function that you won't re-use and so you don't really need to give it a
persistent name.

I haven't seen any lambdas in any Python library code, or in any of the
third-party modules I use (numpy, matplotlib, Biopython). Do they exist?
Because I have not been forced to do so, I haven't retained a space in the
top drawer of my programming brain for lambda.

[...]

+1 about the name, -1 about the usefulness. For example it's a clear and
concise way to pass a key argument to min, max, sort and friends:

sort(seq, key=lambda x: x.a)

The alternative is to have trivial function definitions floating around the
namespace which readers of the code have to search for.

 

[Steven D'Aprano]

I have to make sure my functor to keep the state variable values for
different objects that call the same functor to behave correctly in
order to avoid passing extra parameters in various objects using the
same functor.

Yo dawg, I heard you liked functors, so I put a functor in your functor
so you can train your markov chains with extra functor parameters to
functor objects that are functor factory decorator functors.