Getting a set of lambda functions

[Martin Manns]

Hi,

I try to get a set of lambda functions that allows me executing each
function code exactly once. Therefore, I would like to modify the set
function to compare the func_code properties (or the lambda
functions to use this property for comparison).

(The reason is that the real function list is quite large (> 1E5), there
are only few functions with non-equal code and the order of execution
is not important.)

How can I achieve this?

func_strings=['x', 'x+1', 'x+2', 'x']
funclist = [eval('lambda x:' + func) for func in func_strings]
len(funclist) 4
len(set(funclist)) 4
funclist[0].func_code == funclist[3].func_code True
funclist[0] == funclist[3]

False


Thanks in advance

Martin

 

[Ivan Illarionov]

Hi,

I try to get a set of lambda functions that allows me executing each
function code exactly once. Therefore, I would like to modify the set
function to compare the func_code properties (or the lambda functions to
use this property for comparison).

(The reason is that the real function list is quite large (> 1E5), there
are only few functions with non-equal code and the order of execution is
not important.)

How can I achieve this?

func_strings=['x', 'x+1', 'x+2', 'x'] funclist = [eval('lambda x:' +
func) for func in func_strings] len(funclist) 4
len(set(funclist)) 4
funclist[0].func_code == funclist[3].func_code True
funclist[0] == funclist[3]

False


Thanks in advance

Martin

Maybe make a set of code objects?

func_code_set = set([f.func_code for f in funclist])

funclist = []
for fc in func_code_set:
f = lambda x: x
f.func_code = fc
funclist.append(f)

-- Ivan

 

[Ivan Illarionov]

Hi,

I try to get a set of lambda functions that allows me executing each
function code exactly once. Therefore, I would like to modify the set
function to compare the func_code properties (or the lambda functions to
use this property for comparison).

(The reason is that the real function list is quite large (> 1E5), there
are only few functions with non-equal code and the order of execution is
not important.)

How can I achieve this?

func_strings=['x', 'x+1', 'x+2', 'x'] funclist = [eval('lambda x:' +
func) for func in func_strings] len(funclist) 4
len(set(funclist)) 4
funclist[0].func_code == funclist[3].func_code True
funclist[0] == funclist[3]

False


Thanks in advance

Martin

Maybe make a set of code objects?

func_code_set = set([f.func_code for f in funclist])

funclist = []
for fc in func_code_set:
f = lambda x: x
f.func_code = fc
funclist.append(f)

-- Ivan

 

[Martin Manns]

On Sun, 25 May 2008 13:43:15 +0200, Martin Manns wrote:

Maybe make a set of code objects?

func_code_set = set([f.func_code for f in funclist])

funclist = []
for fc in func_code_set:
f = lambda x: x
f.func_code = fc
funclist.append(f)

Works for me.

Thanks

Martin

 

[I V]

I try to get a set of lambda functions that allows me executing each
function code exactly once. Therefore, I would like to modify the set
function to compare the func_code properties (or the lambda functions to
use this property for comparison).

With Ivan's approach, you lose access to the actual lambdas, so you need
to create a new function and then modify its code object to actually call
the code; this seems a little clunky to me. You might instead want to
wrap the lambdas in an object that will do the comparison you want:

class Code(object):
def __init__(self, func):
self._func = func

def __cmp__(self, other):
return cmp(self._func.func_code, other._func.func_code)

def __call__(self, *args, **kwargs):
return self._func(*args, **kwargs)

func_set = set(Code(f) for f in funclist)

 

[bearophileHUGS]

I V:

You might instead want to
wrap the lambdas in an object that will do the comparison you want:

This looks very nice, I haven't tried it yet, but if it works well
then it may deserve to be stored in the cookbook, or better, it may
become the built-in behavior of hashing functions.

Bye,
bearophile

 

[bearophileHUGS]

This may have some bugs left, but it looks a bit better:

from inspect import getargspec

class HashableFunction(object):
"""Class that can be used to wrap functions, to allow their
hashing,
for example to create a set of unique functions.

>>> func_strings = ['x', 'x+1', 'x+2', 'x']
>>> func_list1 = [eval('lambda x:' + func) for func in func_strings]
>>> func_set = set(HashableFunction(f) for f in func_list1)
>>> len(func_set) 3
>>> set(f(10) for f in func_set) == set([10, 11, 12])

True

>>> func_list2 = [lambda x=1: x, lambda x: x]
>>> len(set(HashableFunction(f) for f in func_list2))

2

>>> class Foo: ... def __eq__(self, other): raise NotImplementedError
>>> func_list3 = [lambda x=Foo(): x, lambda x=Foo(): x]
>>> set(HashableFunction(f) for f in func_list3)

Traceback (most recent call last):
...
NotImplementedError
"""
# Original code by I V <[email protected]>, 25 May 2008,
# http://groups.google.com/group/comp.lang.python/browse_thread/thread/a973de8f3562675c
# modified by bearophile
def __init__(self, func):
self._func = func
signature = getargspec(func)
self._hash = hash(self._func.func_code) ^ \
hash(tuple(signature[0]) + tuple(signature[1:3]))
def __eq__(self, other):
return self._func.func_code == other._func.func_code and \
getargspec(self._func) == getargspec(other._func)
def __hash__(self):
return self._hash
def __call__(self, *args, **kwargs):
return self._func(*args, **kwargs)

I haven't put a full hashing of getargspec(self._func) too into the
__init__() because it may contain too much nested unhashable
structures, but the __eq__() is able to tell them apart anyway, with
some collisions.

Bye,
bearophile

 

[Denis Kasak]

Hi,

I try to get a set of lambda functions that allows me executing each
function code exactly once. Therefore, I would like to modify the set
function to compare the func_code properties (or the lambda
functions to use this property for comparison).

(The reason is that the real function list is quite large (> 1E5), there
are only few functions with non-equal code and the order of execution
is not important.)

How can I achieve this?

func_strings=['x', 'x+1', 'x+2', 'x']
funclist = [eval('lambda x:' + func) for func in func_strings]
len(funclist) 4
len(set(funclist)) 4
funclist[0].func_code == funclist[3].func_code True
funclist[0] == funclist[3]

False

Isn't this a bug? Shouldn't it be possible to create a set of
different lambda functions via a loop? At first I thought it was just
a quirk of list comprehensions, but the following example also yields
incorrect (or at least unintuitive) results:

spam = []
for i in range(10): .... spam.append(lambda: i)
spam[0]() 9
spam[1]()

9

Manually creating the lambdas and appending them to a list works as
expected, naturally; I don't see a good reason why it wouldn't work
with a loop. Am I missing something?

 

[Denis Kasak]

> Hi,
>
> I try to get a set of lambda functions that allows me executing each
> function code exactly once. Therefore, I would like to modify the set
> function to compare the func_code properties (or the lambda
> functions to use this property for comparison).
>
> (The reason is that the real function list is quite large (> 1E5), there
> are only few functions with non-equal code and the order of execution
> is not important.)
>
> How can I achieve this?
>

> >>> func_strings=['x', 'x+1', 'x+2', 'x']
> >>> funclist = [eval('lambda x:' + func) for func in func_strings]
> >>> len(funclist) > 4
> >>> len(set(funclist)) > 4
> >>> funclist[0].func_code == funclist[3].func_code > True
> >>> funclist[0] == funclist[3]

> False

Isn't this a bug? Shouldn't it be possible to create a set of
different lambda functions via a loop? At first I thought it was just
a quirk of list comprehensions, but the following example also yields
incorrect (or at least unintuitive) results:

>>> spam = []
>>> for i in range(10): .... spam.append(lambda: i)
>>> spam[0]() 9
>>> spam[1]()

9

Manually creating the lambdas and appending them to a list works as
expected, naturally; I don't see a good reason why it wouldn't work
with a loop. Am I missing something?

 

[Denis Kasak]

Denis Kasak wrote:
...

spam = []
for i in range(10): ... spam.append(lambda: i)
spam[0]() 9
spam[1]()

9

Manually creating the lambdas and appending them to a list works as
expected, naturally; I don't see a good reason why it wouldn't work
with a loop. Am I missing something?

Yes, you are missing something: binding time. your anonymous function
returns the value of "i" in the enclosing scope _at_the_time_of_
_the function's_execution_. If you follow your previous example with
'del i' and then execute any of your spam functions, you'll get an
"NameError: global name 'i' is not defined".

Ah, the problem was in the subtle misunderstanding of the semantics of
lambda functions on my part. It's much clearer now. Thanks.

There are a couple of ways to slve your problem:
(1) use default args to do the binding at the function definition time.
for i in range(10):
spam.append(lambda arg=i: arg)
The lambda expression is normally spelled "lambda i=i: i)", but if you
don't know the idiom, I find the "i=i" part confuses people.

Indeed. It hasn't occured to me that lambdas could bind the variables
inside them by name. I guess my lambdas are still a little shaky.

 

[Martin Manns]

On Sun, 25 May 2008 14:39:28 -0700 (PDT)

This may have some bugs left, but it looks a bit better: [...]
self._hash = hash(self._func.func_code) ^ \
hash(tuple(signature[0]) + tuple(signature[1:3]))
def __eq__(self, other):
return self._func.func_code == other._func.func_code and \
getargspec(self._func) == getargspec(other._func)
def __hash__(self):
return self._hash
def __call__(self, *args, **kwargs):
return self._func(*args, **kwargs)

I haven't put a full hashing of getargspec(self._func) too into the
__init__() because it may contain too much nested unhashable
structures, but the __eq__() is able to tell them apart anyway, with
some collisions.

Looking at a function:
['__call__', '__class__', '__delattr__', '__dict__', '__doc__',
'__get__', '__getattribute__', '__hash__', '__init__', '__module__',
'__name__', '__new__', '__reduce__', '__reduce_ex__', '__repr__',
'__setattr__', '__str__', 'func_closure', 'func_code', 'func_defaults',
'func_dict', 'func_doc', 'func_globals', 'func_name']

Should func_globals and func_name also be taken into account for
__eq__()?

Best Regards

Martin

 

[Terry Reedy]

| On Sun, 25 May 2008 13:43:15 +0200, Martin Manns wrote:
| > I try to get a set of lambda functions that allows me executing each

I think it worth the reminder that Python has lambda *expressions* that
result in function objects that identical to the function objects produced
by def statements (except for the special name '<lambda>'). For the
purpose of this thread,comparing functions by comparing their code objects,
the name attribute is irrelevant. So this discussion has nothing to do
with the syntactic form of function definitions.

| > function code exactly once. Therefore, I would like to modify the set
| > function to compare the func_code properties (or the lambda functions
to
| > use this property for comparison).
|
| With Ivan's approach, you lose access to the actual lambdas,

I presume you mean function object rather than lambda expression

| so you need
| to create a new function and then modify its code object to actually call
| the code; this seems a little clunky to me. You might instead want to
| wrap the lambdas in an object that will do the comparison you want:
|
| class Code(object):
| def __init__(self, func):
| self._func = func
|
| def __cmp__(self, other):
| return cmp(self._func.func_code, other._func.func_code)
|
| def __call__(self, *args, **kwargs):
| return self._func(*args, **kwargs)

If one could subclass from function instead of object, this would be even
simpler (even with Bearophile's addition) since no __call__ would be
needed. But we cannot, partly because there did not seem to be much use
case when this was once discussed. But here is one ;-)

tjr

 

[Terry Reedy]

| On Sun, 25 May 2008 14:39:28 -0700 (PDT)
| (e-mail address removed) wrote:
|
| > This may have some bugs left, but it looks a bit better:
| [...]
| > self._hash = hash(self._func.func_code) ^ \
| > hash(tuple(signature[0]) + tuple(signature[1:3]))
| > def __eq__(self, other):
| > return self._func.func_code == other._func.func_code and \
| > getargspec(self._func) == getargspec(other._func)
| > def __hash__(self):
| > return self._hash
| > def __call__(self, *args, **kwargs):
| > return self._func(*args, **kwargs)
| >
| > I haven't put a full hashing of getargspec(self._func) too into the
| > __init__() because it may contain too much nested unhashable
| > structures, but the __eq__() is able to tell them apart anyway, with
| > some collisions.
|
| Looking at a function:
|
| >>> a=lambda x:x
| >>> dir(a)
| ['__call__', '__class__', '__delattr__', '__dict__', '__doc__',
| '__get__', '__getattribute__', '__hash__', '__init__', '__module__',
| '__name__', '__new__', '__reduce__', '__reduce_ex__', '__repr__',
| '__setattr__', '__str__', 'func_closure', 'func_code', 'func_defaults',
| 'func_dict', 'func_doc', 'func_globals', 'func_name']
|
| Should func_globals and func_name also be taken into account for
| __eq__()?

It depends on what *you* want == to mean.

tjr

 

[John Nagle]

Hi,

I try to get a set of lambda functions that allows me executing each
function code exactly once. Therefore, I would like to modify the set
function to compare the func_code properties (or the lambda
functions to use this property for comparison).

(The reason is that the real function list is quite large (> 1E5), there
are only few functions with non-equal code and the order of execution
is not important.)

How can I achieve this?

func_strings=['x', 'x+1', 'x+2', 'x']
funclist = [eval('lambda x:' + func) for func in func_strings]

What are you actually trying to do, anyway? What's the application?

You probably don't want to use "eval" that way. If you want function
objects out, use "compile".

John Nagle

 

[Martin Manns]

func_strings=['x', 'x+1', 'x+2', 'x']
funclist = [eval('lambda x:' + func) for func in func_strings]

What are you actually trying to do, anyway? What's the
application?

You probably don't want to use "eval" that way. If you want
function objects out, use "compile".

I am writing on an application that I call pyspread
(http://pyspread.sourceforge.net).
It provides a grid (mapped to a numpy.array) into which users can type
in strings that contain python expressions.
Each expression is transformed into a function object in a second
numpy.array of the same size, so that each function can be called by
accessing the respective cell of the grid.

eval seems to work quite fine and provides nice access to globals,
system functions, modules, etc. Therefore, one can do general
programming tasks within the grid without worrying about cell
precedence.

compile returns code objects instead of function objects. I can of
course evaluate these code objects. However, when I store the code
objects, it is an extra operation each time I want to call the
function. So what is the benefit?

More specific: Is there any benefit of calling:

eval(compile(mystring, '', 'eval'))

instead of

eval(mystring)

?

What are the drawbacks of my approach that I may be missing?
Any suggestions?

Best Regards

Martin

 

[Ivan Illarionov]

func_strings=['x', 'x+1', 'x+2', 'x']
funclist = [eval('lambda x:' + func) for func in func_strings]

   What are you actually trying to do, anyway?  What's the
application?

   You probably don't want to use "eval" that way.  If you want
function objects out, use "compile".

I am writing on an application that I call pyspread
(http://pyspread.sourceforge.net).
It provides a grid (mapped to a numpy.array) into which users can type
in strings that contain python expressions.
Each expression is transformed into a function object in a second
numpy.array of the same size, so that each function can be called by
accessing the respective cell of the grid.

eval seems to work quite fine and provides nice access to globals,
system functions, modules, etc. Therefore, one can do general
programming tasks within the grid without worrying about cell
precedence.

compile returns code objects instead of function objects. I can of
course evaluate these code objects. However, when I store the code
objects, it is an extra operation each time I want to call the
function. So what is the benefit?

More specific: Is there any benefit of calling:

eval(compile(mystring, '', 'eval'))

instead of

eval(mystring)

?

What are the drawbacks of my approach that I may be missing?
Any suggestions?

Best Regards

Martin

Yes, there is.
Your original example can be rewritten

func_strings=['x', 'x+1', 'x+2', 'x']
funclist = [compile('lambda x: %s' % func, '<string>', 'eval') for func in func_strings]
len(funclist) 4
len(set(funclist)) 3
eval(funclist[0])(1)

1

Ivan