# lambda in list comprehension acting funny

Discussion in 'Python' started by Daniel Fetchinson, Jul 11, 2012.

1. ### Daniel FetchinsonGuest

funcs = [ lambda x: x**i for i in range( 5 ) ]
print funcs[0]( 2 )
print funcs[1]( 2 )
print funcs[2]( 2 )

This gives me

16
16
16

When I was excepting

1
2
4

Does anyone know why?

Cheers,
Daniel

--
Psss, psss, put it down! - http://www.cafepress.com/putitdown

Daniel Fetchinson, Jul 11, 2012

2. ### Colin J. WilliamsGuest

On 11/07/2012 2:41 AM, Daniel Fetchinson wrote:
> funcs = [ lambda x: x**i for i in range( 5 ) ]
> print funcs[0]( 2 )
> print funcs[1]( 2 )
> print funcs[2]( 2 )
>
> This gives me
>
> 16
> 16
> 16
>
> When I was excepting
>
> 1
> 2
> 4
>
> Does anyone know why?
>
> Cheers,
> Daniel
>
>

I don't understand why you would expect 1, 2, 4.

Perhaps parentheses will help the order of evaluation:

funcs = [(lambda x: x**i) for i in range( 5 )]

This gives:
1
16
81

Colin W.

Colin J. Williams, Jul 11, 2012

3. ### Ian KellyGuest

On Wed, Jul 11, 2012 at 4:28 AM, Colin J. Williams <> wrote:
> I don't understand why you would expect 1, 2, 4.

Because:

funcs[0](2) == 2 ** 0 == 1
funcs[1](2) == 2 ** 1 == 2
funcs[2](2) == 2 ** 2 == 4

> Perhaps parentheses will help the order of evaluation:
>
> funcs = [(lambda x: x**i) for i in range( 5 )]
>
> This gives:
> 1
> 16
> 81

No, that gives 16, 16, 16 just like the original. I don't understand
why you would expect 1, 16, 81, unless you have misread the code.

Ian Kelly, Jul 11, 2012
4. ### woooeeGuest

You should not be using lambda in this case
..for x in [2, 3]:
.. funcs = [x**ctr for ctr in range( 5 )]
.. for p in range(5):
.. print x, funcs[p]
.. print

woooee, Jul 11, 2012

Exactly. It's threads like these which remind me why I never use lambda. I would rather give a function an explicit name and adhere to the familiar Python syntax, despite the two extra lines of code. I don't even like the name "lambda". It doesn't tell you what it is (unless you're John McCarthy), a function that you won't re-use and so you don't really need to give ita persistent name.

I haven't seen any lambdas in any Python library code, or in any of the third-party modules I use (numpy, matplotlib, Biopython). Do they exist? Because I have not been forced to do so, I haven't retained a space in the topdrawer of my programming brain for lambda.

I know the historical reason that Python ended up with lambda, it was requested by people in the Lisp community. While I appreciate some of the Lisp-like features which did find their way into Python (especially being able to treat code as data, and functions as objects), I've found that lambda does nothing for me.

6. ### Hans MulderGuest

On 11/07/12 20:38:18, woooee wrote:
> You should not be using lambda in this case
> .for x in [2, 3]:
> . funcs = [x**ctr for ctr in range( 5 )]
> . for p in range(5):
> . print x, funcs[p]
> . print

The list is called "funcs" because it is meant to contain functions.
Your code does not put functions in the list, so it doesn't do what
he wants.

He could do:

funcs = []
for i in range(5):
def f(x):
return x**i
funcs.append(f)
print funcs[0]( 2 )
print funcs[1]( 2 )
print funcs[2]( 2 )

..... and it will print 16, 16, 16 for the same reason as the lambda
version. On the other hand, this does what he wants:

funcs = []
for i in range(5):
def f(x, i=i):
return x**i
funcs.append(f)
print funcs[0]( 2 )
print funcs[1]( 2 )
print funcs[2]( 2 )

The lambda keyword is a red herring. The question is really about
functions, and how they handle non-local variables. The good news
is that 'lambda' and 'def' work exactly the same in that regards.
So once you understand the finer points about 'def', you no longer
have a reason to avoid 'lambda'.

Hope this helps,

-- HansM

Hans Mulder, Jul 11, 2012
7. ### Steven D'ApranoGuest

On Wed, 11 Jul 2012 11:38:18 -0700, woooee wrote:

> You should not be using lambda in this case
> .for x in [2, 3]:
> . funcs = [x**ctr for ctr in range( 5 )]
> . for p in range(5):
> . print x, funcs[p]
> . print

If you change the requirements, it's always easy to solve problems. But
it is the wrong problem that you have solved.

The problem we have been asked to solve is to create a sequence of
function objects, so that they can be called later, when needed, *not* to
pre-calculate the results.

In this case, the most obvious solution is to store a local variable in
each function object with the value you want.

funcs = [(lambda x, i=i: x**i) for i in range(5)]

creates a list of five functions:

lambda x, i=0: x**i
lambda x, i=1: x**i
lambda x, i=2: x**i
and so on.

In this case, each function has two local variables, x and i, with i
having a default value. The function parameter i is bound to the value of
i when the function was created.

Because parameter defaults are calculated once, when the function is
created, this causes the value of i to stick to the newly created
function, and we get the result we need.

What happens if you don't use a parameter with a default value?

funcs = [(lambda x: x**i) for i in range(5)]

In this case, each function body contains one local variable, x, and one
non-local or global variable, i.

Because i is a non-local, the function doesn't store a value for it.
Instead, the function stores a lump of data pointing to just enough of
the environment to fetch the current value of the non-local i when
needed. Since all five functions are in the same environment, they all
see the same value of i when you call them, regardless of what the value
of i was when they were created.

This is little different from doing this:

i = 1
def f1(x): return x**i

i = 2
def f2(x): return x**i

i = 3
def f3(x): return x**i

Is there any surprise that all three functions return the same value?
They all point to the same global variable i. I'm not sure what it is
about lambda that fools people into thinking that it is different (I've
even been fooled myself!) but it is not.

--
Steven

Steven D'Aprano, Jul 12, 2012
8. ### Steven D'ApranoGuest

> Exactly. It's threads like these which remind me why I never use
> lambda. I would rather give a function an explicit name and adhere to
> the familiar Python syntax, despite the two extra lines of code.

lambda is familiar Python syntax, or at least it should be if you
consider yourself an expert Python programmer.

And besides, this is not a problem with lambda. You get the *exact* same
problem with "ordinary" function definitions. Of course you do -- lambdas
*are* ordinary functions, they just have a more compact syntax.

funcs = []
for i in range(5):
def pwr(x):
return x**i
print("Testing 3**%d:" % i, pwr(3))
funcs.append(pwr)

for j in range(5):
print("Expected", 3**j, "got", funcs[j](3))

gives you *exactly* the same issue as with the lambda. This is not a
problem with lambdas, this is a scoping issue.

> I don't even like the name "lambda". It doesn't tell you what it is

Whereas "def" does?

If you can learn that "def" defines a function (as opposed to a class, a
module, or anything else really) than it's not that hard to learn that
"lambda" also defines a function.

Although I do wonder whether people would be less scared of lambda, and
less likely to imagine that it creates something with strange mysterious
properties different from "regular functions", if the keyword was called

> (unless you're John McCarthy), a function that you won't re-use and so
> you don't really need to give it a persistent name.

Whether or not a function has a persistent name has nothing to do with
reuse. Anonymous functions can be reused. The driving motivation for
lambdas is for callback functions, which are frequently reused, but you
don't need or want to fill your global namespace up with potentially
hundreds of callback functions.

Lambda syntax is only incidentally good for saving a line or two. If the
only benefit of lambda was to save a line of code, that would be stupid.
The real reason for lambda is to avoid polluting your namespace with
unnecessary names.

Python allows us to easily and simply use anonymous strings, anonymous
ints, anonymous objects of all types. Why should functions be treated as
second-class?

> I haven't seen any lambdas in any Python library code, or in any of the
> third-party modules I use (numpy, matplotlib, Biopython). Do they
> exist? Because I have not been forced to do so, I haven't retained a
> space in the top drawer of my programming brain for lambda.

There are over 800 uses of lambda in the standard library:

[steve@ando ~]\$ cd /usr/local/lib/python3.2
[steve@ando python3.2]\$ grep lambda *.py */*.py | wc -l
829

(although some of the above are comments, doctests, etc.)

See, for example, configparser, decimal, functools, gettext, inspect,
pydoc, symtable, uuid, and others. pydoc and idlelib in particular make
heavy use of lambda, and the test suite is overflowing with them -- over
600 uses in total.

--
Steven

Steven D'Aprano, Jul 12, 2012
9. ### Dennis Lee BieberGuest

<> declaimed the following in
gmane.comp.python.general:

> I know the historical reason that Python ended up with lambda, it was requested by people in the Lisp community. While I appreciate some of the Lisp-like features which did find their way into Python (especially being able to treat code as data, and functions as objects), I've found that lambda does nothing for me.

It can be useful when defining callbacks for GUIs that need a
predefined argument... eg; a calculator application: 10 buttons for 0-9.
Why define 10 separate callbacks when you only need one callback with a
parameter containing the numeric value for the button.

lambda x=#: number_button_callback(x)

where # is the (integral) value of the specific button.

{non sequitur: I still recall my archaic C++ class with the OOAD
assignment of designing said calculator -- we never had to implement
one, just design the basic classes/methods/attributes [on 3x5 cards] for
a four-banger. I managed to persuade the team I was on that an RPN
calculator would be simpler to (potentially) implement... And THEN
persuaded them to NOT use the equivalent of an ALU class, but to put the
math work into each operation button... Justification when presented to
class: one could turn that four-banger into a scientific [or financial]
[and of course, buttons on screen] rather than having to rewrite an
"ALU" class to understand all functions.}
--
Wulfraed Dennis Lee Bieber AF6VN
HTTP://wlfraed.home.netcom.com/

Dennis Lee Bieber, Jul 12, 2012
10. ### Daniel FetchinsonGuest

>> You should not be using lambda in this case
>> .for x in [2, 3]:
>> . funcs = [x**ctr for ctr in range( 5 )]
>> . for p in range(5):
>> . print x, funcs[p]
>> . print

>
> If you change the requirements, it's always easy to solve problems. But
> it is the wrong problem that you have solved.
>
> The problem we have been asked to solve is to create a sequence of
> function objects, so that they can be called later, when needed, *not* to
> pre-calculate the results.
>
> In this case, the most obvious solution is to store a local variable in
> each function object with the value you want.
>
> funcs = [(lambda x, i=i: x**i) for i in range(5)]
>
> creates a list of five functions:
>
> lambda x, i=0: x**i
> lambda x, i=1: x**i
> lambda x, i=2: x**i
> and so on.
>
> In this case, each function has two local variables, x and i, with i
> having a default value. The function parameter i is bound to the value of
> i when the function was created.
>
> Because parameter defaults are calculated once, when the function is
> created, this causes the value of i to stick to the newly created
> function, and we get the result we need.
>
> What happens if you don't use a parameter with a default value?
>
> funcs = [(lambda x: x**i) for i in range(5)]
>
> In this case, each function body contains one local variable, x, and one
> non-local or global variable, i.
>
> Because i is a non-local, the function doesn't store a value for it.
> Instead, the function stores a lump of data pointing to just enough of
> the environment to fetch the current value of the non-local i when
> needed. Since all five functions are in the same environment, they all
> see the same value of i when you call them, regardless of what the value
> of i was when they were created.
>
> This is little different from doing this:
>
> i = 1
> def f1(x): return x**i
>
> i = 2
> def f2(x): return x**i
>
> i = 3
> def f3(x): return x**i
>
> Is there any surprise that all three functions return the same value?
> They all point to the same global variable i. I'm not sure what it is
> about lambda that fools people into thinking that it is different (I've
> even been fooled myself!) but it is not.

Thank you Steve!
Precise and clear, as always!

Cheers,
Daniel

--
Psss, psss, put it down! - http://www.cafepress.com/putitdown

Daniel Fetchinson, Jul 12, 2012
11. ### 88888 DihedralGuest

On Thursday, July 12, 2012 12:34:33 AM UTC+8, Ian wrote:
> On Wed, Jul 11, 2012 at 4:28 AM, Colin J. Williams &lt;&gt; wrote:
> &gt; I don't understand why you would expect 1, 2, 4.
>
> Because:
>
> funcs[0](2) == 2 ** 0 == 1
> funcs[1](2) == 2 ** 1 == 2
> funcs[2](2) == 2 ** 2 == 4
>
> &gt; Perhaps parentheses will help the order of evaluation:
> &gt;
> &gt; funcs = [(lambda x: x**i) for i in range( 5 )]
> &gt;
> &gt; This gives:
> &gt; 1
> &gt; 16
> &gt; 81
>
> No, that gives 16, 16, 16 just like the original. I don't understand
> why you would expect 1, 16, 81, unless you have misread the code.

I'll contribute my way of python programming:

def powerb(x, b): #
return x**b

One functor is enough!

88888 Dihedral, Jul 12, 2012
12. ### 88888 DihedralGuest

On Thursday, July 12, 2012 12:34:33 AM UTC+8, Ian wrote:
> On Wed, Jul 11, 2012 at 4:28 AM, Colin J. Williams &lt;&gt; wrote:
> &gt; I don't understand why you would expect 1, 2, 4.
>
> Because:
>
> funcs[0](2) == 2 ** 0 == 1
> funcs[1](2) == 2 ** 1 == 2
> funcs[2](2) == 2 ** 2 == 4
>
> &gt; Perhaps parentheses will help the order of evaluation:
> &gt;
> &gt; funcs = [(lambda x: x**i) for i in range( 5 )]
> &gt;
> &gt; This gives:
> &gt; 1
> &gt; 16
> &gt; 81
>
> No, that gives 16, 16, 16 just like the original. I don't understand
> why you would expect 1, 16, 81, unless you have misread the code.

I'll contribute my way of python programming:

def powerb(x, b): #
return x**b

One functor is enough!

88888 Dihedral, Jul 12, 2012
13. ### Steven D'ApranoGuest

On Wed, 11 Jul 2012 20:39:45 -0700, 88888 Dihedral wrote:

> I'll contribute my way of python programming:
>
> def powerb(x, b): #
> return x**b

Here's a shorter version:

py> pow
<built-in function pow>

> One functor is enough!

Nothing we have been discussing in this thread has been a functor, either
in the Haskell sense or the C++ sense.

--
Steven

Steven D'Aprano, Jul 12, 2012
14. ### Steven D'ApranoGuest

On Wed, 11 Jul 2012 21:05:30 -0400, Dennis Lee Bieber wrote:

> {non sequitur: I still recall my archaic C++ class with the OOAD
> assignment of designing said calculator -- we never had to implement
> one, just design the basic classes/methods/attributes [on 3x5 cards] for
> a four-banger. I managed to persuade the team I was on that an RPN
> calculator would be simpler to (potentially) implement... And THEN
> persuaded them to NOT use the equivalent of an ALU class, but to put the
> math work into each operation button...

"ALU class"?

Googling gives me no clue.

--
Steven

Steven D'Aprano, Jul 12, 2012
15. ### Steven D'ApranoGuest

On Wed, 11 Jul 2012 08:41:57 +0200, Daniel Fetchinson wrote:

> funcs = [ lambda x: x**i for i in range( 5 ) ]

Here's another solution:

from functools import partial
funcs = [partial(lambda i, x: x**i, i) for i in range(5)]

Notice that the arguments i and x are defined in the opposite order.
That's because partial only applies positional arguments from the left.
If there was a "right partial" that applies from the right, we could use

funcs = [rpartial(pow, i) for i in range(5)]

--
Steven

Steven D'Aprano, Jul 12, 2012
16. ### Terry ReedyGuest

On 7/11/2012 11:53 PM, Steven D'Aprano wrote:
> On Wed, 11 Jul 2012 21:05:30 -0400, Dennis Lee Bieber wrote:
>
>> {non sequitur: I still recall my archaic C++ class with the OOAD
>> assignment of designing said calculator -- we never had to implement
>> one, just design the basic classes/methods/attributes [on 3x5 cards] for
>> a four-banger. I managed to persuade the team I was on that an RPN
>> calculator would be simpler to (potentially) implement... And THEN
>> persuaded them to NOT use the equivalent of an ALU class, but to put the
>> math work into each operation button...

>
> "ALU class"?
>
> Googling gives me no clue.

4th hit: Arithmetic Logic Unit (hardware) -- Wikipedia
ALU class -- software version or simulation thereof.

The hardware version of Dennis's distributed design would be
interesting. Spreadsheets are based in part on the same idea of
distributed logic and arithmetic.

--
Terry Jan Reedy

Terry Reedy, Jul 12, 2012
17. ### Dennis Lee BieberGuest

On 12 Jul 2012 03:53:03 GMT, Steven D'Aprano
<> declaimed the following in
gmane.comp.python.general:

> "ALU class"?
>
> Googling gives me no clue.

Arithmetic/Logic Unit

http://en.wikipedia.org/wiki/Arithmetic_logic_unit
http://en.wikipedia.org/wiki/74181
{diversion: http://en.wikipedia.org/wiki/VAX-11/780 -- incredible...
that used to be considered a "super-mini" when I worked on them; and now
would be shamed by most laptops except for the ability to support so
many users concurrently (let me know when a Windows laptop supports 32
VT-100 class connections <G>)}

--
Wulfraed Dennis Lee Bieber AF6VN
HTTP://wlfraed.home.netcom.com/

Dennis Lee Bieber, Jul 12, 2012
18. ### 88888 DihedralGuest

On Thursday, July 12, 2012 11:51:04 AM UTC+8, Steven D'Aprano wrote:
> On Wed, 11 Jul 2012 20:39:45 -0700, 88888 Dihedral wrote:
>
> &gt; I'll contribute my way of python programming:
> &gt;
> &gt; def powerb(x, b): #
> &gt; return x**b
>
> Here's a shorter version:
>
> py&gt; pow
> &lt;built-in function pow&gt;
>
>
> &gt; One functor is enough!
>
> Nothing we have been discussing in this thread has been a functor, either
> in the Haskell sense or the C++ sense.
>
>
>
> --
> Steven

Well, I encountered this kind of problems before in OOP.

I have to make sure my functor to keep the state variable values
for different objects that call the same functor to behave correctly
in order to avoid passing extra parameters in various objects using the same functor.

88888 Dihedral, Jul 12, 2012
19. ### John O'HaganGuest

On Wed, 11 Jul 2012 13:21:34 -0700 (PDT)

> Exactly. It's threads like these which remind me why I never use lambda. I
> would rather give a function an explicit name and adhere to the familiar
> Python syntax, despite the two extra lines of code. I don't even like the
> name "lambda". It doesn't tell you what it is (unless you're John McCarthy),
> a function that you won't re-use and so you don't really need to give it a
> persistent name.
>
> I haven't seen any lambdas in any Python library code, or in any of the
> third-party modules I use (numpy, matplotlib, Biopython). Do they exist?
> Because I have not been forced to do so, I haven't retained a space in the
> top drawer of my programming brain for lambda.

[...]

+1 about the name, -1 about the usefulness. For example it's a clear and
concise way to pass a key argument to min, max, sort and friends:

sort(seq, key=lambda x: x.a)

The alternative is to have trivial function definitions floating around the
namespace which readers of the code have to search for.

--
John

John O'Hagan, Jul 12, 2012
20. ### Steven D'ApranoGuest

On Wed, 11 Jul 2012 22:04:51 -0700, 88888 Dihedral wrote:

> I have to make sure my functor to keep the state variable values for
> different objects that call the same functor to behave correctly in
> order to avoid passing extra parameters in various objects using the
> same functor.

Yo dawg, I heard you liked functors, so I put a functor in your functor
so you can train your markov chains with extra functor parameters to
functor objects that are functor factory decorator functors.

--
Steven

Steven D'Aprano, Jul 12, 2012