Anonymus functions revisited

D

Duncan Booth

George said:
Yeap.. a simple one-liner can do the trick:

def makeVars(**nameVals):
sys._getframe(1).f_locals.update(nameVals)

try: b
except NameError: print "Before makeVars: NameError"
else: print "Before makeVars: Not NameError"
makeVars(b=2)
try: b
except NameError: print "After makeVars: NameError"
else: print "After makeVars: Not NameError"

Do I really need to mention that the whole concept here is broken. This
only works if you call it from global scope. If you call it from inside a
function it [usually] won't work:
sys._getframe(1).f_locals.update(nameVals)

try: b
except NameError: print "Before makeVars: NameError"
else: print "Before makeVars: Not NameError"
makeVars(b=2)
try: b
except NameError: print "After makeVars: NameError"
else: print "After makeVars: Not NameError"

Before makeVars: NameError
After makeVars: NameError
 
D

Diez B. Roggisch

Can someone who thinks this way, please explain why this is acceptable
[ x * x for x in some_iterator ]

But this is not

map(lambda x: x * x, some_iteraror)

and should be replaced with

def sqr(x): return x * x
map(sqr , some_iterator)

It shouldn't, it should be replaced with the listcomp. And IMHO it results
in better readable code.

AFAIK the reason people say you should use named functions is _not_ because
of these single expression replacements. The reason is that frequently
people complain about lambda being so restrictive so that you can't write

lambda x,y: if x > 100: x else: y

and want to _extend_ lambda beyond its current capabilities and make it full
featured but anonymous functions.

And these requests are rebuked with the argument that having to give a more
complex function a name instead of declaring it anonymously doesn't cost
you much - especially since you can declare them inside other functions so
the global namespace isn't cluttered.

I have to admit that rereading my statement

"""
You are right, but for lambda in its current limited form short, named
functions are a good replacement
"""

is not really what I wanted to say.

I should have written:

"""
for lambda in its current limited form, listcomprehensions often are a good
replacement.
"""

But as I've said various times before: I personally don't mind lambdas and
for example the reduce function has been useful for me quite a few times,
can't be replaced by listcomps, and frequently needs a callable consisting
of only a single expression. So I'll continue to use lambdas there.
 
K

Kay Schluehr

Duncan said:
Do I really need to mention that the whole concept here is broken. This
only works if you call it from global scope. If you call it from inside a
function it [usually] won't work:
sys._getframe(1).f_locals.update(nameVals)
try: b
except NameError: print "Before makeVars: NameError"
else: print "Before makeVars: Not NameError"
makeVars(b=2)
try: b
except NameError: print "After makeVars: NameError"
else: print "After makeVars: Not NameError"

Before makeVars: NameError
After makeVars: NameError

Yes.

To put it short:

def makeVars(**nameVals):
print "prev",sys._getframe(1).f_locals["z"]
sys._getframe(1).f_locals.update(nameVals)
print "post",sys._getframe(1).f_locals["z"]

def test():
z = 0
makeVars(z=2)
prev 0
post 3
prev 0
post 0


The Python runtime uses the opcodes STORE_FAST and LOAD_FAST to
store/access local variables of a function.

Take a closer look at the code:

case STORE_FAST:
v = POP();
SETLOCAL(oparg, v);
goto fast_next_opcode;

with

register PyObject **fastlocals

and macros

#define GETLOCAL(i) (fastlocals)
#define SETLOCAL(i, value) do { PyObject *tmp = GETLOCAL(i); \
GETLOCAL(i) = value; \
Py_XDECREF(tmp); } while (0)

The local variables will be stored in the pointer array fastlocals not
within a Python dict.

If a variable is created as a global it will be stored using the
STORE_NAME opcode instead:


case STORE_NAME:
w = GETITEM(names, oparg);
v = POP();
if ((x = f->f_locals) != NULL) {
if (PyDict_CheckExact(x))
err = PyDict_SetItem(x, w, v);
else
err = PyObject_SetItem(x, w, v);
Py_DECREF(v);
if (err == 0) continue;
break;
}


which clearly accesses a Python dict for storage.


Regards Kay
 
A

Antoon Pardon

Op 2005-03-23 said:
Can someone who thinks this way, please explain why this is acceptable

[ x * x for x in some_iterator ]

But this is not

map(lambda x: x * x, some_iteraror)

and should be replaced with

def sqr(x): return x * x
map(sqr , some_iterator)

It shouldn't, it should be replaced with the listcomp. And IMHO it results
in better readable code.

This misses the point. The fact that map and the listcomprehension do
the same stuff is just a detail. It is about giving a function an
expression as argument that is to be evaluated multiple times.
AFAIK the reason people say you should use named functions is _not_ because
of these single expression replacements. The reason is that frequently
people complain about lambda being so restrictive so that you can't write

lambda x,y: if x > 100: x else: y

and want to _extend_ lambda beyond its current capabilities and make it full
featured but anonymous functions.

This is one kind of people. An other kind of people seems to want to
get rid of lamda's altogether and asks whether it is so difficult
to make a named function each time you would otherwise use a lambda.
But as I've said various times before: I personally don't mind lambdas and
for example the reduce function has been useful for me quite a few times,
can't be replaced by listcomps, and frequently needs a callable consisting
of only a single expression. So I'll continue to use lambdas there.

Then my question was not meant for you.
 
R

Ron

Do I really need to mention that the whole concept here is broken. This
only works if you call it from global scope. If you call it from inside a
function it [usually] won't work:


That's only becuase it was asked to go up 1 frame, and not 2.


def makeVars(**nameVals):
sys._getframe(2).f_locals.update(nameVals) # <- get 2 frames up.


def test():
try: b
except NameError: print "Before makeVars: NameError"
else: print "Before makeVars: Not NameError"
makeVars(b=2)
try: b
except NameError: print "After makeVars: NameError"
else: print "After makeVars: Not NameError"


import sys
test()
Before makeVars: NameError
After makeVars: Not NameError
 
R

Ron

Do I really need to mention that the whole concept here is broken. This
only works if you call it from global scope. If you call it from inside a
function it [usually] won't work:

Ok... you can get globals this way if you know how many frames up it
is. Not terrable useful. :/


def frame0():
print 'frame0 (locals): ',sys._getframe(0).f_locals
print '\nframe1 (parent): ',sys._getframe(1).f_locals
print '\n(Global) : ',sys._getframe(2).f_locals

def frame1():
frame0()

import sys
frame1()

frame0 (locals): {}

frame1 (parent): {}

(Global) : {'__builtins__': <module '__builtin__' (built-in)>, 'sys':
<module 'sys' (built-in)>, '__name__': '__main__', 'frame1': <function
 
K

Kay Schluehr

Ron said:
Do I really need to mention that the whole concept here is broken. This
only works if you call it from global scope. If you call it from inside a
function it [usually] won't work:


That's only becuase it was asked to go up 1 frame, and not 2.

def makeVars(**nameVals):
sys._getframe(2).f_locals.update(nameVals) # <- get 2 frames up.

def test():
try: b
except NameError: print "Before makeVars: NameError"
else: print "Before makeVars: Not NameError"
makeVars(b=2)
try: b
except NameError: print "After makeVars: NameError"
else: print "After makeVars: Not NameError"


import sys
test()


Before makeVars: NameError
After makeVars: Not NameError

That's true only because the globals are updated by {"b":2}
two levels down.

If You nest the test() function You reproduce the error again:

def test():
def inner(): try: b
except NameError: print "Before makeVars: NameError"
else: print "Before makeVars: Not NameError"
makeVars(b=2)
try: b
except NameError: print "After makeVars: NameError"
else: print "After makeVars: Not NameError"
inner()


Before makeVars: NameError
After makeVars: NameError


A working makeVars seems not to be different from

def makeVars(**nameVals):
globals().update(nameVals)


Regards Kay
 
D

Duncan Booth

Kay said:
A working makeVars seems not to be different from

def makeVars(**nameVals):
globals().update(nameVals)

Not quite. If Ron can come up with a working makeVars it would update the
caller's globals whereas what you just posted updates makeVar's globals so
there is a difference (when the makeVars and the calling function are in
different modules), just not a very useful one.
 
J

Julian Smith

Can someone who thinks this way, please explain why this is acceptable

[ x * x for x in some_iterator ]

But this is not

map(lambda x: x * x, some_iteraror)

and should be replaced with

def sqr(x): return x * x
map(sqr , some_iterator)

I guess I have similar feelings.

I've only been using python for a year or two, so I'm still a relative
newcomer.

Like a lot of people, I found Python to be magically simple and intuitive.

But I've never really found list comprehensions particularly clear. I don't
know whether it's my backgroud (mainly C/C++), or my brain or what, but they
are probably the only thing in Python that hasn't seemed transparent and
obvious to me (apart from shadowing of variables in nested functions).

Of course, I'm sure I could get used to them, given time. But that rather
defeats one of the main reasons for using python in the first place - the
lack of a steep learning curve.

In contrast, I find lambdas, even with their significant restrictions, to be
much more obvious. In fact, the proposal (sorry, don't have a link to hand)
about extending lambda to allow things like `myfn = def <indented block>' was
something that I initially assumed would become part of the language in time.

Is there some definitive information that explains why python is moving away
from lambdas and towards things like list comprehensions?

- Julian
 
G

George Sakkis

bruno modulix said:
They are two different beasts. Note that you don't have anything like
list unpacking, now tuple unpacking is pretty common in Python (swap,
multiple return values, formatted strings and outputs, ...).

All the following are possible:
(x,y,z) = (1,2,3)
(x,y,z) = [1,2,3]
[x,y,z] = (1,2,3)
[x,y,z] = [1,2,3]

What exactly do you mean by "don't have anything like list unpacking" ?

George
 
R

Ron

Not quite. If Ron can come up with a working makeVars it would update the
caller's globals whereas what you just posted updates makeVar's globals so
there is a difference (when the makeVars and the calling function are in
different modules), just not a very useful one.

How about this one? The only reliable way I found to do it is to
pass locals() to the function.

def defvalue(**args):
args = args.items()
a1 = args[0]
a2 = args[1]
if type(a1[1]) == type({}):
vv, names = a2, a1[1]
else:
vv, names = a1, a2[1]
if names.has_key(vv[0]):
return names[vv[0]]
return vv[1]

f = defvalue(f=1, v=locals())
print f # 0

g = 19
g = defvalue(g=2, v=locals())
print g # 19

z = 6
def f1():
#z = 4
z = defvalue(z=3, v=locals())
print z
f1()
 
B

bruno modulix

George said:
(snip)
Note that you don't have anything like
list unpacking, now tuple unpacking is pretty common in Python (swap,
multiple return values, formatted strings and outputs, ...).


All the following are possible:

(x,y,z) = (1,2,3)
(x,y,z) = [1,2,3]
[x,y,z] = (1,2,3)
[x,y,z] = [1,2,3]


What exactly do you mean by "don't have anything like list unpacking" ?

A stupidity :(
 
D

Duncan Booth

Ron said:
How about this one? The only reliable way I found to do it is to
pass locals() to the function.

Yes, but you are still missing the fundamental point. The locals()
dictionary is not guaranteed to do anything useful if you update it. The
current C implementation will reflect changes in the locals dictionary if
you call locals() from global scope or in a few other circumstances, but
this is simply an implementation detail.

If you want to update global variables then use globals() or setattr on the
module. Only use locals() to access local variables indirectly, never to
try and set them.
 
R

Ron

Yes, but you are still missing the fundamental point. The locals()
dictionary is not guaranteed to do anything useful if you update it. The
current C implementation will reflect changes in the locals dictionary if
you call locals() from global scope or in a few other circumstances, but
this is simply an implementation detail.

Nope, Didn't miss the point. The functions return a value, not create
it from within.
If you want to update global variables then use globals() or setattr on the
module. Only use locals() to access local variables indirectly, never to
try and set them.

Good advise. :)

One of pythons weak points is it is sometimes difficult to 'monitor
and confirm' what is happening leading to confusing try/except
constructions.

Having the function is_defined() and if_not_defined() have the
advantage that they can be use in expressions where try/except can't.
And the source code could be more compact and more readable.

The disadvantage is the functions are quite a bit slower than
try/except, probably due to the function call over head.

If they were built in, they may be as fast as the try/except and there
wouldn't be issues with having to passing locals() or globals() name
dictionaries.


It's interesting that there is a whole is_"type"_() group of functions
in the inspect module, but not a is_defined(). Maybe I just haven't
found it yet.


#############

def if_not_defined(v, dv=None, lv=locals()):
if lv.has_key(v):
return lv[v]
return dv

def is_defined(v, lv=locals()):
if lv.has_key(v):
return True
False

# Shorten names and pass locals() with lambas for
# convenience. (This needs to be in the function
# where they are used or it will break.
# Another use for lamba! ;)

ifnd = lambda v, dv, lv=locals(): if_not_defined(v,dv,lv)
isa = lambda v, lv=locals(): is_defined(v, lv)

# Totally useless routine. ;)
import random
for n in range(10):

# Delete a random x,y,z coordinate to
# simulate an unreliable data source.
d = random.choice([1,2,3])
if d==1:
if isa('x'): del x
elif d==2:
if isa('y'): del y
else:
if isa('z'): del z

# Replace the missing variable with a random number.
r = int(random.random()*100)
x, y, z = ifnd('x',r), ifnd('y',r), ifnd('z',r)
print x, y, z

###########
 
D

Duncan Booth

Ron said:
It's interesting that there is a whole is_"type"_() group of functions
in the inspect module, but not a is_defined(). Maybe I just haven't
found it yet.

I've never found any need for an is_defined function. If in doubt I just
make sure and initialise all variables to a suitable value before use.
However, I'll assume you have a good use case.
#############

def if_not_defined(v, dv=None, lv=locals()):
if lv.has_key(v):
return lv[v]
return dv

I don't see the point of the default argument. locals() at the global level
simply returns globals, so you might as well use that. A more useful
default would be the caller's locals.
def is_defined(v, lv=locals()):
if lv.has_key(v):
return True
False

Same comments, plus the whole 'if on a boolean to return a boolean' is a
bit redundant.

Try something on these lines:
if lv is None:
lv = inspect.currentframe().f_back.f_locals
return name in lv
print is_defined('x')
print is_defined('y')
y = 0
print is_defined('y')

True
False
True


# Shorten names and pass locals() with lambas for
# convenience. (This needs to be in the function
# where they are used or it will break.
# Another use for lamba! ;)

ifnd = lambda v, dv, lv=locals(): if_not_defined(v,dv,lv)
isa = lambda v, lv=locals(): is_defined(v, lv)

There is no need for lambda here, it adds nothing. Use a 'def'.
 
R

Ron_Adam

I've never found any need for an is_defined function. If in doubt I just
make sure and initialise all variables to a suitable value before use.
However, I'll assume you have a good use case.

I admit that that is the better practice. George's example was the
conversion of data from one form to another where the data is mixed
with complete and incomplete items. And Kay is looking at tuple
unpacking.

It's hard to beat try/except for these situations though. :)

I cleaned it up some more and figured out the proper use of
_getframe(). So no lambdas, and no passing of locals needed., and it
checks for globals and builtins before defining the default value so
as not to over write a readable value.

I'm not sure what the best behavior should be. Maybe a routine to
tell where a name is, ie.. local, global, builtin, or a writable
global? maybe isa() return the location or None.? I think that would
be better.

The best purpose for utilities like these is for debugging and getting
feedback about the environment. So I'm thinking of putting them in a
module for that purpose. I have a subroutine to list all the names
attached to an object. I think I can add that a bit now too.


#---Here's the code---------------------

import sys

def isa(v):
"""
Check if a varable exists in the current
(parent to this function), global, or
builtin name spaces.

use: bool = isa( str )
returns True or False
"""
plocals = sys._getframe(1).f_locals
if plocals.has_key(v) or globals().has_key(v) or \
__builtins__.locals().has_key(v):
return True
return False


def ifno(v, obj=None):
"""
Check if a varable does not exists, return a
default value, otherwise return the varable obj.

use: obj = ifno( str [,obj=None] )
if str exist, returns str's object
if str does not exist, returns specified object
"""
plocals = sys._getframe(1).f_locals
if plocals.has_key(v):
return plocals[v]
if globals().has_key(v):
return globals()[v]
if __builtins__.locals().has_key(v):
return __builtins__.locals()[v]
return obj


def test():
"""
Test isa() and ifno() functions:
"""

# Totally useless routine. ;)
import random
for n in range(25):

# Delete a random x,y,z coordinate to
# simulate an unrealiabe data source.
d = random.choice([1,2,3])
if d==1:
if isa('x'): del x
elif d==2:
if isa('y'): del y
else:
if isa('z'): del z

# Replace the missing Varible with a random number.
r = int(random.random()*100)
x, y, z = ifno('x',r), ifno('y',r), ifno('z',r)
print x, y, z


if __name__ == '__main__':
test()

#-------------------------------------
 
G

George Sakkis

Ron_Adam said:
I admit that that is the better practice. George's example was the
conversion of data from one form to another where the data is mixed
with complete and incomplete items. And Kay is looking at tuple
unpacking.

It's hard to beat try/except for these situations though. :)

I posted a recipe in python cookbook
(http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/392768) for the subproblem I was interested
in initially (variable-length iterable unpacking), and I prefer it over explicit try/except (but of
course I'm biased :)). Kay is proposing something even more general and powerful, and it will be
interesting to see if all this brainstorming can be brought forward more 'formally', e.g. at a PEP
or pre-PEP level.

Regards,
George
 
R

Ron_Adam

I posted a recipe in python cookbook
(http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/392768) for the subproblem I was interested
in initially (variable-length iterable unpacking), and I prefer it over explicit try/except (but of
course I'm biased :)). Kay is proposing something even more general and powerful, and it will be
interesting to see if all this brainstorming can be brought forward more 'formally', e.g. at a PEP
or pre-PEP level.

Regards,
George

Looks good George. :)

I'm not sure what Kay is trying for, but it does look interesting. I'm
all for new features as long as they are consistent and easy to use,
and easy to remember as well. I keep finding ways to improve the
little routines I'm playing with. ;) I'll probably post them in the
cookbook also, and maybe put them together in a mod.

A few more pieces and I should be able to build a name space explorer
which I think will be good for debugging programs. I'm thinking you
could put it in the program where you are having problems and it will
open a tree type window where you can examine all the names and
objects at that point. When done, close it and the programs
continues. It will give you a little more info than sticking print
statements hear and there.

Ron
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,796
Messages
2,569,645
Members
45,371
Latest member
TroyHursey

Latest Threads

Top