Whoa! Do Python and Lisp really have LAMBDA ?

[mike420]

Earlier Ed Schofield (thanks, man) warned us that

flist = []

for i in range(3)
f = lambda x: x + i
flist.append(f)

[f(1) for f in flist]

gives [3, 3, 3]. So much for the principle of minimum surprise!

Doing the same in Lisp (with lists instead of arrays),

(setf flist (loop for i from 0 to 2
collect (lambda (x) (+ x i))))

(loop for f in flist
collect (funcall f 1))

I got (4 4 4).

Lisp has many gotchas, I just wasn't ready for this one.
(Google for "lisp gotchas" - someone posted a comprehensive
list to c.l.l. in 1995. Every Lisper should read it)

I'm sure Haskell does this right. What about Scheme and ML?

 

[Rob Warnock]

+---------------
| Doing the same in Lisp (with lists instead of arrays),
|
| (setf flist (loop for i from 0 to 2
| collect (lambda (x) (+ x i))))
|
| (loop for f in flist
| collect (funcall f 1))
|
| I got (4 4 4).
|
| Lisp has many gotchas, I just wasn't ready for this one.
+---------------

Why should this be considered a "gotcha"? It's doing exactly what
you asked it to: all three lambdas are closed over the *same* variable
binding, which was left holding "3" when the loop finished. Try it
this way instead and you might get what you wanted/expected:

> (defparameter flist (loop for i from 0 to 2
collect (let ((u i))
(lambda (x) (+ x u)))))
FLIST
> (loop for f in flist
collect (funcall f 1))
(1 2 3)
>

In this case the lambdas are closed over *distinct* bindings.


-Rob

 

[Edi Weitz]

+---------------
| Lisp has many gotchas, I just wasn't ready for this one.
+---------------

Why should this be considered a "gotcha"?

Because he's a troll.

 

[Bradd W. Szonye]

(setf flist (loop for i from 0 to 2

collect (lambda (x) (+ x i))))
(loop for f in flist
collect (funcall f 1))

I got (4 4 4).

Yes, that is suprising, although it makes more sense once you realize
that they all bind to the same i, which is mutated during the loop.

I'm sure Haskell does this right. What about Scheme and ML?

The equivalent in Scheme (named let) introduces a new binding with each
iteration, so it does what you expect.

(define flist
(let loop ((i 0) (r '()))
(cond ((> i 2) (reverse r))
(else (loop (+ 1 i)
(cons (lambda (x) (+ x i)) r))))))

(let loop ((l flist) (r '()))
(cond ((null? l) (reverse r))
(else (loop (cdr l)
(cons ((car l) 1) r)))))

Unlike the Lisp version of flist, the Scheme loop binds a new i for each
iteration. Therefore, each closure has its own i.

My Scheme version is much wordier than the Lisp version above. Perhaps
the more experienced schemers can show you a less verbose version that
still does what you want. I've always been fond of functional languages,
but I've only recently had the chance to work with them extensively, so
I'm still learning.

 

[Marcin 'Qrczak' Kowalczyk]

On Sun, 26 Oct 2003 00:11:05 -0700, mike420 wrote:

[...]

I'm sure Haskell does this right. What about Scheme and ML?

Indeed Haskell does this right.

OCaml does this right.

SML doesn't have a for loop. If you emulate it with recursion idiomatic
to SML (passing the incremented argument, not using a mutable reference)
then it will work.

Scheme doesn't have a for loop either, I think it's like in SML - or would
it be more idiomatic to use "set!"? in which case it would not work.

Ruby does this wrong if you use "for i in 0..2 do ... end" but right if
you use "(0..2).each do |i| ... end".

Smalltalk does this right, unless you use some ancient implementations
which make block parameters local to the method in which they are written.
I'm not sure how widespread are such implementations.

Perl does this right if you remember to use "foreach my $i (...)" instead
of "foreach $i (...)" or "foreach (...)". In the latter cases a global
variable is used which is obviously wrong. I think Perl courses should
emphasize "my" more.

In Java I think you can't reference a mutable variable from a local class
but you can reference a final variable, so it detects the problem and
requires manual creation of an immutable binding to work around it.

I suspect that the newer C# which will have anonymous functions does this
wrong.

What about Dylan? Erlang? Mercury?

Moral 1: first class functions are better used with functional style
(immutable data). It's because they make the time when something is
evaluated harder to see, which is fine as long as data is immutable.
In this example it's easy to see that the lambda is evaluated later
but it's not as easy to notice that it matters that the dereferencing of
the variable happens when the function is called, not when it's created.
By taking away the possibility of mutation you take away some surprises.

Moral 2: if you design a language with closures, it's better not to use
a shared mutable variable in a "for" loop.

 

[=?ISO-8859-2?Q?Jens_Axel_S=3Fgaard?=]

Scheme doesn't have a for loop either, I think it's like in SML - or would
it be more idiomatic to use "set!"? in which case it would not work.

You forget do.
(And for-each and map)

 

[Edi Weitz]

On Sun, 26 Oct 2003 00:11:05 -0700, mike420 wrote:

[...]

I'm sure Haskell does this right. What about Scheme and ML?

Indeed Haskell does this right.

OCaml does this right.

Just for the record: Common Lisp also does it right. The fact that it
doesn't do what someone "expects" who hasn't read the spec doesn't
make its behaviour wrong.

As others have pointed out you can choose if you want all the closures
to capture the same binding or if you want each closure to capture a
new binding. This is a feature, not a bug.

Edi.

 

[Marco Antoniotti]

Earlier Ed Schofield (thanks, man) warned us that

flist = []

for i in range(3)
f = lambda x: x + i
flist.append(f)

[f(1) for f in flist]

gives [3, 3, 3]. So much for the principle of minimum surprise!

Doing the same in Lisp (with lists instead of arrays),

(setf flist (loop for i from 0 to 2
collect (lambda (x) (+ x i))))

(loop for f in flist
collect (funcall f 1))

I got (4 4 4).

Lisp has many gotchas, I just wasn't ready for this one.
(Google for "lisp gotchas" - someone posted a comprehensive
list to c.l.l. in 1995. Every Lisper should read it)

I'm sure Haskell does this right. What about Scheme and ML?

Common Lisp does it right.

(mapcar (lambda (f) (funcall f 1))
(mapcar (lambda (i)
(lambda (x) (+ x i)))
(list 1 2 3)))

.... This is what the Haskell code eventually boild down to.

It is Python that apparently cannot do this. But, in all fairness,
nowhere in Python there is a claim that lambda expressions are full fledged.

The LOOP based version of Common Lisp does not do what you think it does
because the LOOP semantics is not the one you think it is.

Of course, I can always come up with a nice set of macros that would
hide some of the syntactic messiness in CL (of course do not ask me to
change the evaluation rules for CL: CL is simply not lazy)

Cheers

 

[Karl A. Krueger]

Common Lisp does it right.

(mapcar (lambda (f) (funcall f 1))
(mapcar (lambda (i)
(lambda (x) (+ x i)))
(list 1 2 3)))

... This is what the Haskell code eventually boild down to.

It is Python that apparently cannot do this. But, in all fairness,
nowhere in Python there is a claim that lambda expressions are full fledged.

Python lambda isn't *that* limited. It's just that the equivalent is
rather ugly by Python standards:

[f(1) for f in
[(lambda i: lambda x: x + i)(y)
for y in [1, 2, 3]]]

This also works, but isn't any prettier:

map(lambda f: apply(f, (1,)),
map(lambda i:
lambda x: (x + i),
[1, 2, 3]))

(Bleah. All those colons and commas are giving me MPI flashbacks.)

 

[Juliusz Chroboczek]

| (loop for f in flist
| collect (funcall f 1))
|
| I got (4 4 4).

RW> Why should this be considered a "gotcha"?

Because it is? The loop/collect idiom has a mostly functional feel to
it, while it is implemented imperatively.

It's an issue that becomes transparent once you become used to Common
Lisp, just like the semantic difference between DO in CL and in Scheme.
But it's still an issue.

Followups restricted.

Juliusz

 

[Juliusz Chroboczek]

Q> Scheme doesn't have a for loop either,

It's called DO and it does the right thing.

Juliusz