Why are functions atomic?

M

Michael

Why are functions atomic? (I.e. they are not copied.)

For example, I would like to make a copy of a function so I can change
the default values:
(2, 2)

I would like the following behaviour:
(1,2)

I know I could use a 'functor' defining __call__ and using member
variables, but this is more complicated and quite a bit slower. (I
also know that I can use new.function to create a new copy, but I
would like to know the rational behind the decision to make functions
atomic before I shoot myself in the foot;-)

Thanks,
Michael.
 
L

Laurent Pointal

Michael said:
Why are functions atomic? (I.e. they are not copied.)

For example, I would like to make a copy of a function so I can change
the default values:

(2, 2)

I would like the following behaviour:

(1,2)

I know I could use a 'functor' defining __call__ and using member
variables, but this is more complicated and quite a bit slower. (I
also know that I can use new.function to create a new copy, but I
would like to know the rational behind the decision to make functions
atomic before I shoot myself in the foot;-)

Thanks,
Michael.

This dont make functions copiable but may resolve your default arguments
problem. Under Python 2.5, see functools.partial().

http://docs.python.org/lib/partial-objects.html
 
7

7stud

Why are functions atomic? (I.e. they are not copied.)

For example, I would like to make a copy of a function so I can change
the default values:


(2, 2)

I would like the following behaviour:


(1,2)

I know I could use a 'functor' defining __call__ and using member
variables, but this is more complicated and quite a bit slower. (I
also know that I can use new.function to create a new copy, but I
would like to know the rational behind the decision to make functions
atomic before I shoot myself in the foot;-)

Thanks,
Michael.

Does deepcopy work?
 
D

Duncan Booth

7stud said:
Does deepcopy work?

It doesn't copy a function.

The easiest way to make a modified copy of a function is to use the 'new'
module.
x= 2
 
C

Chris Mellon

It doesn't copy a function.

The easiest way to make a modified copy of a function is to use the 'new'
module.

x= 2

The copy module considers functions to be immutable and just returns
the object. This seems pretty clearly wrong to me - functions are
clearly not immutable and it's easy to copy a function using new, as
shown above.
From copy.py:

def _copy_immutable(x):
return x
for t in (type(None), int, long, float, bool, str, tuple,
frozenset, type, xrange, types.ClassType,
types.BuiltinFunctionType,
types.FunctionType):
d[t] = _copy_immutable
 
J

John Nagle

Michael said:
Why are functions atomic? (I.e. they are not copied.)

Because Python has objects for when you need to associate
state with a function.

John Nagle
 
?

=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=

I know I could use a 'functor' defining __call__ and using member
variables, but this is more complicated and quite a bit slower. (I
also know that I can use new.function to create a new copy, but I
would like to know the rational behind the decision to make functions
atomic before I shoot myself in the foot;-)

Function objects were not copyable because they were immutable. For
an immutable object, a copy cannot reasonably be distinguished from
the original object. See copy.py, around line 105.

Regards,
Martin
 
M

Michael

Because Python has objects for when you need to associate
state with a function.

John Nagle

Then why are functions mutable?

I can understand to some extent why functions are not picklable,
because the bytecode may not be the same across python implementations
(is that true?), but I do not understand why copying functions is a
problem. The patch that allows copy to pass-through functions just
emulates pickle, but I can find no discussion or justification for not
allowing functions to be copied:

http://thread.gmane.org/gmane.comp.python.devel/76636

Michael.
 
M

Michael

From TFM

"Function objects also support getting and setting arbitrary
attributes, which can be used, for example, to attach metadata to
functions. Regular attribute dot-notation is used to get and set such
attributes. Note that the current implementation only supports
function attributes on user-defined functions. Function attributes on
built-in functions may be supported in the future."

http://docs.python.org/ref/types.html

Again, rather inconsitent with the copy sematics.
 
M

Michael

A bit more info, but still no clear picture about why functions are
mutable but have immutable copy symantics. There are arguments why
functions should be immutable, but the decision was to make user-
defined functions mutable. My question is still: why the present
ummutable copy symantics?

http://www.python.org/dev/peps/pep-0232/
 
C

Carsten Haese

A bit more info, but still no clear picture about why functions are
mutable but have immutable copy symantics. There are arguments why
functions should be immutable, but the decision was to make user-
defined functions mutable. My question is still: why the present
ummutable copy symantics?

One could make a case that this is a bug, a leftover from when functions
were mostly immutable. However, one can also make a case that correcting
this bug is not worth the effort. Your use case appears to be that you
want to make multiple copies of the same function, and those copies
should be almost, but not quite, the same.

The Pythonic solution is to produce the copies by a factory function
along these lines:
.... def inner(x):
.... return x**exponent
.... return inner
.... 27

This approach makes copying functions unnecessary, and as you have
pointed out yourself, if you find yourself needing to make a copy of an
existing function you can work around the unexpected copy semantics with
new.function.

-Carsten
 
?

=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=

Michael said:
A bit more info, but still no clear picture about why functions are
mutable but have immutable copy symantics. There are arguments why
functions should be immutable, but the decision was to make user-
defined functions mutable. My question is still: why the present
ummutable copy symantics?

The answer is really really simple. The implementation of copy predates
mutability. When the copy code was written, functions *were* immutable.
When functions became mutable, the copy code was not changed, and
nobody noticed or complained.

Regards,
Martin
 
J

John Nagle

Martin said:
The answer is really really simple. The implementation of copy predates
mutability. When the copy code was written, functions *were* immutable.
When functions became mutable, the copy code was not changed, and
nobody noticed or complained.

That's probably an indication that mutable functions don't
get used all that much. Are there any instances of them in the
standard Python libraries?

John Nagle
 
M

Michael

Your use case appears to be that you
want to make multiple copies of the same function, and those copies
should be almost, but not quite, the same.

The Pythonic solution is to produce the copies by a factory function...

... def inner(x):
... return x**exponent
... return inner

Is there a reason for using the closure here? Using function defaults
seems to give better performance:.... def inner(x,exponent=exponent):
.... return x**exponent
.... return inner

This is definitely one viable solution and is essentially what I had
in mind, but I did not want to have to carry the generator arround
with me: Instead, I wanted to use it once as a decorator and then
carry only the function around.
34

The decorator declare_options behaves like the generator above, but
adds some methods (set_options) etc. to allow me to manipulate the
options without generating a new function each time.

I have functions with many options that may be called in the core of
loops, and found that the most efficient solution was to provide all
of the options through func_defaults.
.... return x*(opt1 + opt2*opt3)

The cleanest (and fastest) solution I found was to set the options in
the defaults:
Then f can be passed to the inner loops and f(x) is very quick.

Other options include using lists and dict's:7

but then I have to pass f and opt around. This also appears to be
somewhat slower than the defaults method. Dictionaries have the
advantage of associating the names with the values
7

but this is much slower. Wrapping the function with a generator as
you suggest also works and packages everything together, but again
suffers in performance. It also complicates my code.

The result of my declare_options decorator is that the result is a
regular function, complete with docstring etc. but with added
annotations that allow the options to be set. In addition, the
performance optimal. I though this was a very clean solution until I
realized that I could not make copies of the functions to allow for
different option values with the usual python copy symantics (for
example, a __copy__ method is ignored). I can easily get around this
by adding a custom copy() method, but wondered if there was anything
inherently dangerous with this approach that would justify the added
complications of more complicated wrappings and the performance hit.

Pickling is an obvious issue, but it seems like there is nothing wrong
with the copy semantics and that the limitations are artificial and
out of place. (It is also easily fixed: if the object has a __copy__
method, use it. Truely immutable objects will never have one. There
may be subtle issues here, but I don't know what they are.)

Thanks for all of the suggestions,
Michael.
 
G

Gabriel Genellina

The answer is really really simple. The implementation of copy predates
mutability. When the copy code was written, functions *were* immutable.
When functions became mutable, the copy code was not changed, and
nobody noticed or complained.

Likely scenario, but not true. Support for copying user functions was
added on 2.5 (see
http://svn.python.org/view/python/trunk/Lib/copy.py?rev=42573&r1=38995&r2=42573)
and functions were mutable since a long time ago. On previous versions,
functions could be pickled but not copied.
The same thing happens for classes: they are mutable too, but copy
considers them immutable and returns the same object. This is clearly
stated on the documentation (but the module docstring is still outdated).
 
C

Carsten Haese

Is there a reason for using the closure here? Using function defaults
seems to give better performance:[...]

It does? Not as far as I can measure it to any significant degree on my
computer.
This is definitely one viable solution and is essentially what I had
in mind, but I did not want to have to carry the generator arround
with me:

I don't know what you mean by "carry it around." Just put it in a module
and import it where you need it.

An overriding theme in this thread is that you are greatly concerned
with the speed of your solution rather than the structure and
readability of your code. How often is your function going to get called
and how much of a performance benefit are you expecting?

-Carsten
 
A

Alex Martelli

Michael said:
Is there a reason for using the closure here? Using function defaults
seems to give better performance:

What measurements show you that...?

brain:~ alex$ cat powi.py
def powerfactory1(exponent):
def inner(x):
return x**exponent
return inner

def powerfactory2(exponent):
def inner(x, exponent=exponent):
return x**exponent
return inner

brain:~ alex$ python -mtimeit -s'import powi; p=powi.powerfactory1(3)'
'p(27)'
1000000 loops, best of 3: 0.485 usec per loop

brain:~ alex$ python -mtimeit -s'import powi; p=powi.powerfactory2(3)'
'p(27)'
1000000 loops, best of 3: 0.482 usec per loop


Alex
 
?

=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=

The answer is really really simple. The implementation of copy predates
That's probably an indication that mutable functions don't
get used all that much. Are there any instances of them in the
standard Python libraries?

All user-defined functions are mutable. To me, this issue is rather that
copying of functions isn't used all that much.

In addition, it is certainly true that functions are rarely modified,
even though all of them are mutable. See this for an example

py> def foo():pass
....
py> foo
<function foo at 0xb7db5f44>
py> foo.__name__='bar'
py> foo
<function bar at 0xb7db5f44>
py> foo.value
Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: 'function' object has no attribute 'value'
py> foo.value=100
py> foo.value
100

Regards,
Martin
 
?

=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=

The answer is really really simple. The implementation of copy predates
Likely scenario, but not true.

Interesting. I (clearly) missed that part of the defaultdict discussion
(that was a long thread).

Regards,
Martin
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
474,432
Messages
2,571,680
Members
48,796
Latest member
Greg L.

Latest Threads

Top