Replacement for lambda - 'def' as an expression?

[talin at acm dot org]

I've been reading about how "lambda" is going away in Python 3000 (or
at least, that's the stated intent), and while I agree for the most
part with the reasoning, at the same time I'd be sad to see the notion
of "anonymous functions" go - partly because I use them all the time.

Of course, one can always create a named function. But there are a lot
of cases, such as multimethods / generics and other scenarios where
functions are treated as data, where you have a whole lot of functions
and it can be tedious to come up with a name for each one.

For example, my current hobby project is implementing pattern matching
similar to Prolog in Python. The dispatcher I am making allows you to
create "overloaded" versions of a function that take different patterns
as their input arguments, so that Simplify( (add, x, y) ) calls a
different method than Simplify( (log, x) ) -- in other words, the
choice of which code is executed is based on the structure of the tuple
that is passed into it. However, in order for this to work, I need to
be able to assign a block of Python code to a particular pattern, and
having to invent a named function for each pattern is a burden

Anyway, here's an example, then, of how 'def' could be used:

add = def( a, b ):
return a + b

The lack of a function name signals that this is an anonymous function.
The def keyword defines the function using the same syntax as always -
the arguments are in parentheses, and are unevaluated; The colon marks
the beginning of a suite.

In fact, it looks a lot like the existing lambda, with a couple of
differences:

1) It uses the familiar "def" keyword, which every Python beginner
understands, instead of the somewhat unfamiliar "lambda"
2) The arguments are enclosed in parentheses, instead of a bare tuple
followed by a colon, again reiterating the similarity to the normal
usage of "def".
3) The statements are a real suite instead of a pseudo-suite - they can
consist of multiple lines of statements.

Like all statements whose last argument is a suite, you can put the
body of the function on a single line:

add = def( a, b ): return a + b

(If this were Perl, you could also omit the "return", since in Perl the
last evaluated expression in the function body is what gets returned if
there's no explicit return statement.)

What about passing an anonymous function as an argument, which is the
most common case? This gets tricky, because you can't embed a suite
inside of an expression. Or can you?

The most powerful option would be to leverage the fact that you can
already do line breaks inside of parentheses. So the "def" keyword
would tell the parser to restart the normal indentation calculations,
which would terminate whenever an unmatched brace or paren was
encountered:

a = map(
(def( item ):
item = do_some_calculation( item )
return item
), list )

The one-liner version looks a lot prettier of course:

a = map( (def( item ): return item * item), list )

And it looks even nicer if we switch the order of the arguments around,
since you can now use the final paren of the enclosing function call to
terminate the def suite.

a = map( list, def( item ): return item * item )

Unfortunately, there's no other good way I can think of to signal the
end of the block of statements without introducing some radical new
language construct.

(Besides, if being an expression is good enough for 'yield', why
shouldn't def get the same privilege?

 

[Torsten Bronger]

Hallöchen!

[...]

Anyway, here's an example, then, of how 'def' could be used:

add = def( a, b ):
return a + b

I'm really not an expert in functional programming, so I wonder
what's the difference between "add = def" (assumed that it worked)
and "def add"?

Tschö,
Torsten.

 

[Guest]

I'm really not an expert in functional programming, so I wonder
what's the difference between "add = def" (assumed that it worked)
and "def add"?

In the former case one could write

self.add[0] = def(a, b)
# etc.

 

[Sybren Stuvel]

talin at acm dot org enlightened us with:

I'd be sad to see the notion of "anonymous functions" go

Same here. I think it's a beautyful concept, and very powerful. It
also allows for dynamic function creation in cases where a name would
not be available.

What about passing an anonymous function as an argument, which is
the most common case?

I don't really like that. The syntax is way too messy. Just the

funcref = def(args):
...

syntax would suffice for me.

Sybren

 

[Rocco Moretti]

I'm really not an expert in functional programming, so I wonder
what's the difference between "add = def" (assumed that it worked)
and "def add"?

In the former case one could write

self.add[0] = def(a, b)
# etc.

If that's the issue, it might make more sense to extend def to take any
lvalue.

def self.add[0](a, b):
return a + b

 

[Leif K-Brooks]

It also allows for dynamic function creation in cases where a name
would not be available.

What cases are those?

 

[Sybren Stuvel]

Leif K-Brooks enlightened us with:

What cases are those?

An example:

def generate_randomizer(n, m):
randomizer = def(x):
return x ** n % m

return randomizer

Sybren

 

[Paul Rubin]

An example:

def generate_randomizer(n, m):
randomizer = def(x):
return x ** n % m

return randomizer

You're a little bit confused; "name" doesn't necessarily mean "persistent
name". You could write the above as:

def generate_randomizer (n, m):
def randomizer(x):
return pow(x, n, m)
return randomizer

 

[D H]

I've been reading about how "lambda" is going away in Python 3000 (or

See the page built from earlier threads about this:
http://wiki.python.org/moin/AlternateLambdaSyntax

Your syntax is the same used in boo: http://boo.codehaus.org/Closures

 

[Terry Reedy]

Of course, one can always create a named function. But there are a lot
of cases, such as multimethods / generics and other scenarios where
functions are treated as data, where you have a whole lot of functions
and it can be tedious to come up with a name for each one.

Either reuse names or 'index' them: f0, f1, f2, ...

add = def( a, b ):
return a + b

The difference between this and def add(a,b): return a+b would be the

I need to be able to assign a block of Python code to a particular
pattern,

How about (untested -- I have never actually written a decorator, and am
following a remembered pattern of parameterized decorators) :

patcode = {}
def pat(pattern): # return decorator that registers f in patcode
def freg(f):
f.func_name = 'pat: <%s>' % pattern # optional but useful for debug
patcode[pattern] = f
# no return needed ? since def name is dummy
return freg

@pat('pattern1')
def f(): <code for pattern 1>

@pat('pattern2')
def f(): <code> for pattern 2>

etc

or define freg(f,pat) and call freg *after* each definition

having to invent a named function for each pattern is a burden

But you do not *have to* ;-)
or rather, you can replace func_name with a useful tag as suggested above.

Terry J. Reedy

 

[Terry Reedy]

talin at acm dot org enlightened us with:

Though it is as yet unclear as to what may come in compensation.

Same here. I think it's a beautyful concept

Are you claiming that including a reference to the more humanly readable
representation of a function (its source code) somehow detracts from the
beauty of the function concept? Or are you claiming that binding a
function to a name rather than some other access reference (like a list
slot) somehow detracts from its conceptual beauty? Is so, would you say
the same about numbers?

It seems to me that the beauty of the function concept is quite independent
of its definition syntax and post-definition access method.

, and very powerful.

If anything, adding a source pointer to a function object makes it more,
not less powerful.

I don't really like that. The syntax is way too messy.

I agree.

Just the
funcref = def(args):
...
syntax would suffice for me.

But this is deficient relative to def funcref(args): ... since the *only*
difference is to substitute a generic tag (like '<lambda>') for a specific
tag (like 'funcref') for the .func_name attribute.

Terry J. Reedy

 

[Sybren Stuvel]

Paul Rubin enlightened us with:

You're a little bit confused; "name" doesn't necessarily mean
"persistent name".

Wonderful. Another feature added to Python (that is: the Python
version in my mind ;-) without the need to add any features to Python
(that is: the real Python)

Thanks!

Sybren

 

[Sybren Stuvel]

Terry Reedy enlightened us with:

Are you claiming that including a reference to the more humanly readable
representation of a function (its source code) somehow detracts from the
beauty of the function concept?
Nope.

Or are you claiming that binding a function to a name rather than
some other access reference (like a list slot) somehow detracts from
its conceptual beauty?
Nope.

Is so, would you say the same about numbers?

Nope.

I was under the (apparently very wrong) impression (don't ask my why)
that something like the example that Paul Rubin gave wouldn't be
possible. Now that I've learned that, I take back what I've said. His
code is more beautyful IMO ;-)

Sybren

 

[talin at acm dot org]

I like the decorator idea. Unfortunately, the version of Python I am
using is pre-decorator, and there are various issues involved in
upgrading on Mac OS X (due to the built-in Python 2.3 being used by the
OS itself.) I'll have to look into how to upgrade without breaking too
much...

Some further examples of what I am trying to do. First let me state
what my general goal is: There are lots of inference engines out there,
from Prolog to Yacas, but most of them rely on a custom interpreter.
What I want to find out is if I can build a solver, not by creating a
new language on top of Python, but rather by giving solver-like
capabilities to a Python programmer. Needless to say, this involves a
number of interesting hacks, and part of the motivation for my
suggestion(s) is reducing the hack factor.

So, at the risk of being visited by Social Services for my abuse of
Python Operators, here's a sample of how the sovler works:

# Define a function with multiple arities
Simplify = Function()

# Define some arities. We overload __setitem__ to define an arity.
# Param is a class who'se metaclass defines __getattr__ to return a new
instance
# of Param with the given parameter name.
Simplify[ ( add, Param.x, 0 ) ] = lamba x: return Simplify( x ) # x
+ 0 = x
Simplify[ ( mul, Param.x, 1 ) ] = lamba x: return Simplify( x ) # x
* 1 = x
Simplify[ ( mul, Param.x, 0 ) ] = lamba x: return 0 #
x * 0 = 0
Simplify[ Param.x ] = lamba x: return x
# Fallback case

# Invoke the function. Should print the value of x
print Simplify( (add, x, 0) )

Of course, what I really want is not def or lambda, what I really want
is to be able to define functions that take suites as arguments. But
that would be crazy talk

Define( "Simplify", args ):
code

 

[Robert Kern]

I like the decorator idea. Unfortunately, the version of Python I am
using is pre-decorator, and there are various issues involved in
upgrading on Mac OS X (due to the built-in Python 2.3 being used by the
OS itself.) I'll have to look into how to upgrade without breaking too
much...

There really aren't any issues. The official 2.4.1 binary installs
alongside the built-in 2.3. The executables python{,w,2.4,w2.4} are
installed the /usr/local/bin . Under no circumstances should you have to
replace the built-in 2.3. Indeed, under no circumstances should you
replace it at all.

--
Robert Kern
(e-mail address removed)

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

 

[Paul Rubin]

Are you claiming that including a reference to the more humanly readable
representation of a function (its source code) somehow detracts from the
beauty of the function concept?

Huh? Anonymous functions mean you can use functions as values by
spelling out their source code directly, instead of having to make a
separate reference and then pass that. There are times when the
separate reference is just clutter. It's less readable, not more readable.

Or are you claiming that binding a
function to a name rather than some other access reference (like a list
slot) somehow detracts from its conceptual beauty? Is so, would you say
the same about numbers?

Yes, I would say the same about numbers; Python would suffer if users
were required to give a name to every number. I'd say

x = f(1, 3)

is much less ugly than

one = 1
three = 3
x = f(one, three)

I further don't see how the second example is more "readable" than the first.

 

[Simo Melenius]

You're a little bit confused; "name" doesn't necessarily mean "persistent
name". You could write the above as:

def generate_randomizer (n, m):
def randomizer(x):
return pow(x, n, m)
return randomizer

But if you could do anonymous blocks, you could just write something
like:

def generate_randomizer (n, m):
return def (x):
return pow (x, n, m)

Personally, I don't mind naming local functions in practice (and
Python syntax doesn't lend itself very well to anonymous blocks) but
rather, the nuisance is that I feel there's just plain something wrong
with it. It's less beautiful than it could be.

Luckily, the last time I followed the discussion on this topic in
c.l.p, some bright mind whose name escapes me now pointed out the
craziness of _having to_ name functions by comparing it to the
situation where you'd have to bind any literal objects to symbols
before you could use them. Like:

def return_fixed_number ():
res = 42
return res

or:

arg1 = "foo"
arg2 = 42
arg3 = baz ()
myfunction (arg1, arg2, arg3.xyzzy ())

Sure, you don't lose any expressiveness in that: if you had to name
any object before using it, you could write all the same programs that
you can in the current Python. But it's the expressiveness of your
mind that gets harpooned: you'll have to keep part of your focus on
these extraneous local variables instead of thinking only in terms
of values where only values matter.

 

[Paul Rubin]

But if you could do anonymous blocks, you could just write something
like:

def generate_randomizer (n, m):
return def (x):
return pow (x, n, m)

Yes, as it stands you can already say:

def generate_randomizer(n, m):
return lambda x: pow(x, n, m)

I was showing that it can also be done with a named internal function.

Sure, you don't lose any expressiveness in that: if you had to name
any object before using it, you could write all the same programs that
you can in the current Python. But it's the expressiveness of your
mind that gets harpooned: you'll have to keep part of your focus on
these extraneous local variables instead of thinking only in terms
of values where only values matter.

Yes, I agree with this.

 

[Tom Anderson]

add = def( a, b ):
return a + b

+1

This is so obviously the right syntax for closures in python that i really
can't believe we're still arguing about it.

What about passing an anonymous function as an argument, which is the
most common case? This gets tricky, because you can't embed a suite
inside of an expression. Or can you?

The most powerful option would be to leverage the fact that you can
already do line breaks inside of parentheses. So the "def" keyword
would tell the parser to restart the normal indentation calculations,
which would terminate whenever an unmatched brace or paren was
encountered:

a = map(
(def( item ):
item = do_some_calculation( item )
return item
), list )

Can't we just rely on indentation here:

a = map(
def(item):
item = do_some_calculation(item)
return item
, list)

?

A consequence of that is that you *must* end the suite on a line of its
own; with your scheme, you can in fact write:

a = map((def(item):
item = do_some_calculation(item)
return item), list)

Although i'm not convinced that this is something i want to be possible!

The one-liner version looks a lot prettier of course:

a = map( (def( item ): return item * item), list )

To do one-liners, which is absolutely essential, we can't rely on line
ends, of course, so we'd need your scheme to be in operation here. For
consistency, it should also apply to multi-line suites; it should be
possible to have both the bracket-based and line-based rules in effect at
the same time - changes in indent level are essentially treated as a kind
of bracket.

And it looks even nicer if we switch the order of the arguments around,
since you can now use the final paren of the enclosing function call to
terminate the def suite.

a = map( list, def( item ): return item * item )

Unfortunately, there's no other good way I can think of to signal the
end of the block of statements without introducing some radical new
language construct.

If there were no statements which ended with an expression list, it would
be possible to detect the end by the presence of a comma. The python
grammar would only need a few changes to meet that requirement, none of
them that disruptive (mostly, you replace the expression list with a tuple
- in many cases, making explicit what was previously implicit).

(Besides, if being an expression is good enough for 'yield', why
shouldn't def get the same privilege?

A fine point!

tom