List comprehension + lambdas - strange behaviour

Discussion in 'Python' started by Artur Siekielski, May 6, 2010.

  1. Hello.
    I found this strange behaviour of lambdas, closures and list
    comprehensions:

    >>> funs = [lambda: x for x in range(5)]
    >>> [f() for f in funs]

    [4, 4, 4, 4, 4]

    Of course I was expecting the list [0, 1, 2, 3, 4] as the result. The
    'x' was bound to the final value of 'range(5)' expression for ALL
    defined functions. Can you explain this? Is this only counterintuitive
    example or an error in CPython?


    Regards,
    Artur
     
    Artur Siekielski, May 6, 2010
    #1
    1. Advertising

  2. On May 6, 9:34 pm, Artur Siekielski <>
    wrote:
    > Hello.
    > I found this strange behaviour of lambdas, closures and list
    > comprehensions:
    >
    > >>> funs = [lambda: x for x in range(5)]
    > >>> [f() for f in funs]

    >
    > [4, 4, 4, 4, 4]
    >
    > Of course I was expecting the list [0, 1, 2, 3, 4] as the result. The
    > 'x' was bound to the final value of 'range(5)' expression for ALL
    > defined functions. Can you explain this? Is this only counterintuitive
    > example or an error in CPython?


    Try binding the value of x for each of the inner functions:

    >>> funs = [lambda x=x: x for x in range(5)]
    >>> [f() for f in funs]

    [0, 1, 2, 3, 4]

    Otherwise, the 'x' is just a global value and the lambdas look it up
    at when the function is invoked. Really, not surprising at all:

    >>> x = 10
    >>> def f():

    .... return x
    ....
    >>> x = 20
    >>> f()

    20


    Raymond
     
    Raymond Hettinger, May 6, 2010
    #2
    1. Advertising

  3. On 5/6/2010 12:34 PM Artur Siekielski said...
    > Hello.
    > I found this strange behaviour of lambdas, closures and list
    > comprehensions:
    >
    >>>> funs = [lambda: x for x in range(5)]


    funs is now a list of lambda functions that return 'x' (whatever it
    currently is from whereever it's accessible when invoked)



    >>> [f() for f,x in zip(funs,range(5))]

    [0, 1, 2, 3, 4]
    >>> del x
    >>> [f() for f in funs]

    Traceback (most recent call last):
    File "<stdin>", line 1, in ?
    File "<stdin>", line 1, in <lambda>
    NameError: global name 'x' is not defined
    >>>


    Emile
     
    Emile van Sebille, May 6, 2010
    #3
  4. Artur Siekielski <artur.siekielski <at> gmail.com> writes:
    >
    > Of course I was expecting the list [0, 1, 2, 3, 4] as the result. The
    > 'x' was bound to the final value of 'range(5)' expression for ALL
    > defined functions. Can you explain this? Is this only counterintuitive
    > example or an error in CPython?


    The former. Closures are rebound in a loop.
     
    Benjamin Peterson, May 6, 2010
    #4
  5. Artur Siekielski

    Terry Reedy Guest

    On 5/6/2010 3:34 PM, Artur Siekielski wrote:
    > Hello.
    > I found this strange behaviour of lambdas, closures and list
    > comprehensions:
    >
    >>>> funs = [lambda: x for x in range(5)]
    >>>> [f() for f in funs]

    > [4, 4, 4, 4, 4]


    You succumbed to lambda hypnosis, a common malady ;-).
    The above will not work in 3.x, which does not leak comprehension
    iteration variables. It is equivalent to

    funs = [lambda: x for y in range(5)]
    del y # only for 2.x. y is already gone in 3.x
    x = 4
    [f() for f in funs]

    Now, I am sure, you would expect what you got.

    and nearly equivalent to

    def f(): return x
    x=8
    funs = [f for x in range(5)]
    [f() for f in funs]

    # [8,8,8,8,8] in 3.x

    Ditto

    Terry Jan Reedy
     
    Terry Reedy, May 7, 2010
    #5
  6. Artur Siekielski

    Neil Cerutti Guest

    On 2010-05-07, Terry Reedy <> wrote:
    > On 5/6/2010 3:34 PM, Artur Siekielski wrote:
    >> Hello.
    >> I found this strange behaviour of lambdas, closures and list
    >> comprehensions:
    >>
    >>>>> funs = [lambda: x for x in range(5)]
    >>>>> [f() for f in funs]

    >> [4, 4, 4, 4, 4]

    >
    > You succumbed to lambda hypnosis, a common malady ;-). The
    > above will not work in 3.x, which does not leak comprehension
    > iteration variables.


    It functions the same in 3.1.

    Python 3.1.1 (r311:74483, Aug 17 2009, 17:02:12) [MSC v.1500 32 bit (Intel)] on
    win32
    Type "help", "copyright", "credits" or "license" for more information.
    >>> funs = [lambda: x for x in range(5)]
    >>> [f() for f in funs]

    [4, 4, 4, 4, 4]

    --
    Neil Cerutti
    *** Your child was bitten by a Bat-Lizard. ***
     
    Neil Cerutti, May 7, 2010
    #6
  7. Artur Siekielski

    Terry Reedy Guest

    On 5/7/2010 8:31 AM, Neil Cerutti wrote:
    > On 2010-05-07, Terry Reedy<> wrote:
    >> On 5/6/2010 3:34 PM, Artur Siekielski wrote:
    >>> Hello.
    >>> I found this strange behaviour of lambdas, closures and list
    >>> comprehensions:
    >>>
    >>>>>> funs = [lambda: x for x in range(5)]
    >>>>>> [f() for f in funs]
    >>> [4, 4, 4, 4, 4]

    >>
    >> You succumbed to lambda hypnosis, a common malady ;-). The
    >> above will not work in 3.x, which does not leak comprehension
    >> iteration variables.

    >
    > It functions the same in 3.1.
    >
    > Python 3.1.1 (r311:74483, Aug 17 2009, 17:02:12) [MSC v.1500 32 bit (Intel)] on
    > win32
    > Type "help", "copyright", "credits" or "license" for more information.
    >>>> funs = [lambda: x for x in range(5)]
    >>>> [f() for f in funs]

    > [4, 4, 4, 4, 4]


    Ok.

    >>> x

    Traceback (most recent call last):
    File "<pyshell#1>", line 1, in <module>
    x
    NameError: name 'x' is not defined
    #only in 3.x

    But because the list comp is implemented in 3.x as an anonymous
    function, which is then called and discarded (an implementation that I
    believe is not guaranteed by the language ref), the lambda expression
    defines a nested function which captures the (final) value of x.

    >>> funs[0].__closure__[0].cell_contents

    4

    So it works (runs without exception), but somewhat accidentally and for
    a different reason than in 2.x, where 'x' is 4 at the global level.

    Terry Jan Reedy
     
    Terry Reedy, May 7, 2010
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. George Henry

    Strange (?) list comprehension behavior

    George Henry, Jul 19, 2003, in forum: Python
    Replies:
    3
    Views:
    433
    Peter Hansen
    Jul 20, 2003
  2. Debajit Adhikary
    Replies:
    17
    Views:
    720
    Debajit Adhikary
    Oct 18, 2007
  3. Ken Pu
    Replies:
    2
    Views:
    282
    Jeffrey Froman
    Mar 1, 2008
  4. Jerry Hill
    Replies:
    4
    Views:
    283
    Steve Holden
    Mar 31, 2008
  5. Vedran Furac(
    Replies:
    4
    Views:
    355
    Marc 'BlackJack' Rintsch
    Dec 19, 2008
Loading...

Share This Page