Lambda alternative?

[mousemeat]

Hello all,

I really like the lambda function for a particular task i have: I am
writing some simulation software, working with many different
materials. Each material with many different properties, some are
temperature dependent, others aren't, some are defined by a function,
some are a lookup table, some are constant. Here is a simplified
version:

class material(object):
def __init__(self,density):
self.density=density

airdensity=lambda T:100000/(287*T)
air=material(airdensity)

steeldensity=lambda T:interp(T,[0,1000],[7856,7813])
steel=material(steeldensity)

rockdensity=lambda T:5000
rock=material(rockdensity)

I really like doing things this way, because i can then define other
properties within the class and i can always use the syntax:
specificgravity_at_273K=stone.specificgravity(273), whether
specificgravity is a function or a parameter. (not sure thats the
right nomenclature, but i think you'll know what i mean.)

But, lambda functions can't be pickled. I would like to pickle my
objects, and i would really like to use parallel python (which
requires pickling).

The best way i can see to do that is to make a new class which
inheirits the 'material' class for each physical material i want to
model, but surely i should be creating instances rather than classes.

Is this going to present some new problems that anyone can forsee? Can
anyone think of a better way to do this? I keep reading that lambda
functions aren't particularly pythonic and that there are always
alternatives, but they seem very elegant to me.

Thanks for your time,

Warren

 

[Paul Rubin]

But, lambda functions can't be pickled. I would like to pickle my
objects, and i would really like to use parallel python (which
requires pickling).

Maybe you could use marshal instead of pickle. Make sure to read the
documentation carefully--there are some pitfalls that can bite you.

 

[Paul Rubin]

What exactly is your objection to using a named function here?...
Surely that would solve your pickle problem *and* give you more meaningful
tracebacks in your exceptions?

I don't think it would help the pickle problem.

 

[Piet van Oostrum]

m> Hello all,
m> I really like the lambda function for a particular task i have: I am
m> writing some simulation software, working with many different
m> materials. Each material with many different properties, some are
m> temperature dependent, others aren't, some are defined by a function,
m> some are a lookup table, some are constant. Here is a simplified
m> version:

m> class material(object):
m> def __init__(self,density):
m> self.density=density

m> airdensity=lambda T:100000/(287*T)
m> air=material(airdensity)

m> steeldensity=lambda T:interp(T,[0,1000],[7856,7813])
m> steel=material(steeldensity)

m> rockdensity=lambda T:5000
m> rock=material(rockdensity)

The following isn't much different (only 6 chars more per def):

def airdensity(T): return 100000/(287*T)
air=material(airdensity)

def steeldensity(T): return interp(T,[0,1000],[7856,7813])
steel=material(steeldensity)

def rockdensity(T): return 5000
rock=material(rockdensity)

 

[mousemeat]

(e-mail address removed) (m) wrote:

m> Hello all,
m> I really like the lambda function for a particular task i have: I am
m> writing some simulation software, working with many different
m> materials. Each material with many different properties, some are
m> temperature dependent, others aren't, some are defined by a function,
m> some are a lookup table, some are constant. Here is a simplified
m> version:
m> class material(object):
m>     def __init__(self,density):
m>         self.density=density
m> airdensity=lambda T:100000/(287*T)
m> air=material(airdensity)
m> steeldensity=lambda T:interp(T,[0,1000],[7856,7813])
m> steel=material(steeldensity)
m> rockdensity=lambda T:5000
m> rock=material(rockdensity)

The following isn't much different (only 6 chars more per def):

def airdensity(T): return 100000/(287*T)
air=material(airdensity)

def steeldensity(T): return interp(T,[0,1000],[7856,7813])
steel=material(steeldensity)

def rockdensity(T): return 5000
rock=material(rockdensity)

--
Piet van Oostrum <[email protected]>
URL:http://pietvanoostrum.com[PGP 8DAE142BE17999C4]
Private email: (e-mail address removed)- Hide quoted text -

- Show quoted text -

Will i then be able to pickle 'rock', and any object that may contain
'rock' as one of it's fields? I'm not sure that i will, but i'll give
it a shot.

 

[Paul Rubin]

dumped = dumps(air)
t = loads(dumped)
print t # works fine

Hmm, well, that doesn't really pickle the function; it pickles a class
instance that records where the class definition was and (on
unpickling) imports that module. Maybe that is sufficient for this
purpose.

 

[mousemeat]

It pickles a class instance which contains a reference to a function and
on unpickling it recreates the class instance with a new reference to
the function of the same name.

That's how pickling works: it records where the class or function is
defined and creates a new reference to the class or function of the same
name when you unpickle. The classes and functions have to be defined at
module scope otherwise it won't be able to reference them. You don't
ever actually get either a class or function in the pickle so if you
want you can update the code and the unpickled object gets the new one.


        def __init__(self, fn):
                self.fn = fn
        def callfn(self):
                self.fn()


        print "Hello I'm foo"


        print "I'm a new foo"


Hello I'm foo

Thank you for everyone's explanations, help and interest on this one.
I have reworked my code as described and promised myself not to use
lambdas ever again (i still think they are an elegant idea, but if
they are incompatible with frequently used modules, then the
inelegance of reality quickly overshadows the elegance of the theory).

I think my problem now is that parallel python will want to unpickle
my objects but can't find the modules. I've asked the pp forum for
insight, but i think i'm going to have to plug away with this one.

Thanks again,

Warren

 

[Hrvoje Niksic]

Thank you for everyone's explanations, help and interest on this
one. I have reworked my code as described and promised myself not
to use lambdas ever again (i still think they are an elegant idea,
but if they are incompatible with frequently used modules, then the
inelegance of reality quickly overshadows the elegance of the
theory).

I think you're going too far concluding that lambdas are unusable.
lambdas are a problem only when they are stored as data attributes
that need to be pickled, but that's far from being the only use case.
You can still use them for what they're meant to be used: tiny anonymous
function-expressions, typically passed as parameters to functions that
expect a callback.

Avoiding lambdas because they're unpicklable is like avoiding bound
methods because they're just as unpicklable.

 

[mousemeat]

I think you're going too far concluding that lambdas are unusable.
lambdas are a problem only when they are stored as data attributes
that need to be pickled, but that's far from being the only use case.
You can still use them for what they're meant to be used: tiny anonymous
function-expressions, typically passed as parameters to functions that
expect a callback.

Avoiding lambdas because they're unpicklable is like avoiding bound
methods because they're just as unpicklable.

Correct me if i am wrong, but i can pickle an object that contains a
bound method (it's own bound method). I cannot pickle an object that
contains a lambda function. Doesn't that make lambda's less
pickleable? (I don't mean to be argumentative, i'm trying to
understand python's syntax a little better.)

 

[Aaron Brady]

Correct me if i am wrong, but i can pickle an object that contains a
bound method (it's own bound method).  I cannot pickle an object that
contains a lambda function.  Doesn't that make lambda's less
pickleable?  (I don't mean to be argumentative, i'm trying to
understand python's syntax a little better.)

When you pickle an instance, you get its data and the name of its
class. If you don't have the class later, you can't unpickle it.

Pickle is intended for persistence, and the only real kinds of
persistent code are a self-contained library file, and the source,
especially given access to function-external variables, i.e.
closures. Pickling a function is like taking an excerpt from a binary
executable. It can only be used in highly limited contexts, and as
such not good for a standard module.

It is most likely you want to transmit the source-- either the entire
module, which you can then import dynamically, or an excerpt from it,
which is known to have no closures, etc., and carefully call 'eval' on
it.

I disagree that Andres Tremols & Peter Cogolo's 'Recipe 7.6. Pickling
Code Objects' is applicable. It's highly specialized, delicate, and
brittle.

 

[Hrvoje Niksic]

Correct me if i am wrong, but i can pickle an object that contains a
bound method (it's own bound method).

No, you can't:

import cPickle as p
p.dumps([]) '(l.'
p.dumps([].append)

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: expected string or Unicode object, NoneType found

(I don't mean to be argumentative, i'm trying to understand python's
syntax a little better.)

Note that this has nothing to do with syntax, but with the
(intentional) limitations of the "pickle" module.

 

[J. Cliff Dyer]

Correct me if i am wrong, but i can pickle an object that contains a
bound method (it's own bound method).

No, you can't:

import cPickle as p
p.dumps([]) '(l.'
p.dumps([].append)

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: expected string or Unicode object, NoneType found

Yes he can. mousemeat stated that he could pickle an object that
*contains* a bound method, not that he could pickle the method itself.

That said, you can make an instance method out of a lambda, just as well
as any named function, and you can pickle that object, too:

Python 2.5.2 (r252:60911, Oct 5 2008, 19:29:17)
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.'ccopy_reg\n_reconstructor\np1\n(c__main__\nFoo\np2\nc__builtin__
\nobject\np3\nNtRp4\n.'

Cheers,
Cliff

 

[Aaron Brady]

No, you can't:

import cPickle as p
p.dumps([]) '(l.'
p.dumps([].append)

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: expected string or Unicode object, NoneType found

Yes he can.  mousemeat stated that he could pickle an object that
*contains* a bound method, not that he could pickle the method itself.

That said, you can make an instance method out of a lambda, just as well
as any named function, and you can pickle that object, too:

snip

'Contains' here is ambiguous. If the object contains a bound method,
that is, if a bound method is in its dictionary, you can't.
.... def f( self ): print( 'f' )
....Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Programs\Python30\lib\pickle.py", line 1329, in dumps
Pickler(f, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <class 'method'>: attribute
lookup
builtins.method failed

In this example, 'b' contains a bound method, 'a.f'. However, for
other definitions of 'contains', such as if 'b' is an instance of a
class that contains methods, you can. But in that case, the method is
not in 'b.__dict__'.
{'f': <bound method A.f of <__main__.A object at 0x00B54B90>>}

 

[Hrvoje Niksic]

import cPickle as p
p.dumps([]) '(l.'
p.dumps([].append)

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: expected string or Unicode object, NoneType found

Not the best of examples: [].append is a built-in method rather than a
bound method.

Sorry about the confusion. By bound method I meant the callable
obtained by binding a method (built-in or written in python) to a
concrete instance, as opposed to an unbound method, such as
list.append. The reprs are different for built-in methods and for
python methods, but I consider that difference an artifact of the
CPython implementation, not a distinction mandated by the language.