Once-only evaluation of default parameter values function definitions

F

Fred Ma

Hello,

I'm looking at Python for the 1st time today, motivated by its clarity
of syntax. I've looked at the tutoarial at
http://www.python.org/doc/current/tut/tut.html, particularly at the
section about default arguments. I've got some C++/STL under my belt,
so I'm familiar with passing/returning by reference, by value, and by
pointers. I've also used containers of pointers, as well as c++
default parameter values. I am aware of the general idea of reference
counting, but have never used it.

From the python tutorial, it's clear that to avoid sharing default
values between function calls, one has to avoid:

Example#1
---------
def f(a, L=[]):
L.append(a)
return L

Instead, one should use:

Example#2
---------
def f(a, L=None):
if L is None:
L = []
L.append(a)
return L

None has been described a analogous to the NULL value in c++. I'm
been looking and googling this newsgroup to get a clearer explanation
of why this works. The tutorial says that the default value is only
evaluated once. By that, I assume that there is some unnamed object
that is given the value of the 1st and only evaluation of the default
expression. Every time the function is called without an L parameter,
L gets the value of this unnamed object. But that is not consistent
with the Example#1 i.e.

print f(1)
print f(2)
print f(3)

prints

[1]
[1, 2]
[1, 2, 3]

The unnamed object that I imagined should have the value [], and L
should get set to [] every time that f() is called without a value
specified for L.

The alternative explanation that I could think of is that L is bound
to the unnamed object [], and the object itself changes values to
reflect changes to L i.e. L is now a persistent variable, retaining
its value between function calls unless a value is provided for L in
the function call's argument list. In the latter case, one can
imagine L simply being overwritten with the value provided.

The problem with this picture is that Example#2 should fail for the
same reasons as Example#1. That is, L will not get the value of None
on the 2nd call to f() without a value specified for L. Hence, L will
not be reset to [].

I searched through the tutorials for programmers. The same idiom is
presented, but I can't seem to find a more detailed explanation. Some
threads here presented None as something that gets treated differently
from a value, so I confirmed from the language reference that None is
not just a value, but its own type. The thread "Default Parameters to
function!" quotes the language reference in saying that the object
that L is bound to actually gets changed, when the function body makes
changes to the variable L. Again, if this was the case, it seems like
that should cause Example#2 to fail as well (obviously something I'm
missing here).

I'd appreciate it if someone could provide an explanation that's
suitable for my background (some C++ experience, but not a
practitioner of the more advanced concepts). Thanks.

Fred
 
M

Michael Geary

Fred said:
From the python tutorial, it's clear that to avoid sharing
default values between function calls, one has to avoid:

Example#1
---------
def f(a, L=[]):
L.append(a)
return L

Instead, one should use:

Example#2
---------
def f(a, L=None):
if L is None:
L = []
L.append(a)
return L

None has been described a analogous to the NULL value in
c++. I'm been looking and googling this newsgroup to get
a clearer explanation of why this works. The tutorial says
that the default value is only evaluated once. By that, I
assume that there is some unnamed object that is given the
value of the 1st and only evaluation of the default expression.
Every time the function is called without an L parameter, L
gets the value of this unnamed object.

This is where you went wrong. It looks like you're thinking that the =
operator copies a value into a variable as it might in C++. It doesn't.
Instead, = binds a name to an object.

In C++ terms, think of L as a *reference* to that list (array) object.
The unnamed object that I imagined should have the value
[], and L should get set to [] every time that f() is called
without a value specified for L.

The alternative explanation that I could think of is that L is
bound to the unnamed object [], and the object itself
changes values to reflect changes to L i.e. L is now a
persistent variable, retaining its value between function
calls unless a value is provided for L in the function call's
argument list. In the latter case, one can imagine L simply
being overwritten with the value provided.

Python doesn't have variables. It has names and objects. The = operator
binds a name to an object (i.e. makes the name a reference to an object).

When the function is defined, the [] is evaluated and an empty list object
created. Then, every time the function is called, the local name L is bound
to that object. But, it's bound to the *same* object every time. L doesn't
get a copy of that object; the name L refers directly to that object.

That's why the append() call keeps appending to the list. It's the same list
every time you call the function.

Here's a simpler example:
a = []
b = a
a []
b []
a.append(1)
a [1]
b [1]

See what happened? The statement 'b = a' didn't make a copy of a and store
it in b. Instead, it made the name b refer to the same object as a. So, when
I said b.append(1), it meant exactly the same thing as if I'd said
a.append(1).

-Mike
 
F

Fred Ma

Michael said:
Example#1
---------
def f(a, L=[]):
L.append(a)
return L

Example#2
---------
def f(a, L=None):
if L is None:
L = []
L.append(a)
return L

Here's a simpler example:
a = []
b = a
a []
b []
a.append(1)
a [1]
b
[1]

See what happened? The statement 'b = a' didn't make a copy of a and
store it in b. Instead, it made the name b refer to the same object
as a. So, when I said b.append(1), it meant exactly the same thing
as if I'd said a.append(1).

Thanks, Mike. That's alot clearer. The default value looks like it's
a persistent object. It is similar to a static local variable in C++.
In Example#1, that object actually got changed. In contrast, in
Example#2, the code prevents L from remaining bound to None beyond the
first line, so the None object never gets changed. In fact, my 2nd
explanation was close: that there is an unnamed persistent object that
holds the default value to be bound to L for invocations that don't
explicity supply an argument for L. The place I went wrong was to say
that L never gets bound to None again, after it has been bound to a
caller supplied object on a previous invocation of the function.

Fred
 
B

Brian Gough

Fred Ma said:
I'm looking at Python for the 1st time today, motivated by its clarity
of syntax. I've looked at the tutoarial at
http://www.python.org/doc/current/tut/tut.html, particularly at the
section about default arguments. I've got some C++/STL under my belt,
so I'm familiar with passing/returning by reference, by value, and by
pointers.....

The problem with this picture is that Example#2 should fail for the
same reasons as Example#1. That is, L will not get the value of None
on the 2nd call to f() without a value specified for L. Hence, L will
not be reset to [].

Your explanation is correct up to this point, but the behavior of the
two cases is different. When L=[] is in the body of the fuction, it
is like a local "automatic" variable in C/C++ and discarded at the
end of the function. When L=[] is used in the parameter list, it is
like a "static" variable in C/C++ and persists on future calls.

HTH
 
S

Shalabh Chaturvedi

Fred said:
Michael Geary wrote:

Example#1
---------
def f(a, L=[]):
L.append(a)
return L

Example#2
---------
def f(a, L=None):
if L is None:
L = []
L.append(a)
return L

Here's a simpler example:

a = []
b = a
a
[]

[]
a.append(1)
a

[1]

b

[1]

See what happened? The statement 'b = a' didn't make a copy of a and
store it in b. Instead, it made the name b refer to the same object
as a. So, when I said b.append(1), it meant exactly the same thing
as if I'd said a.append(1).


Thanks, Mike. That's alot clearer. The default value looks like it's
a persistent object. It is similar to a static local variable in C++.
In Example#1, that object actually got changed. In contrast, in
Example#2, the code prevents L from remaining bound to None beyond the
first line, so the None object never gets changed. In fact, my 2nd
explanation was close: that there is an unnamed persistent object that
holds the default value to be bound to L for invocations that don't
explicity supply an argument for L. The place I went wrong was to say
that L never gets bound to None again, after it has been bound to a
caller supplied object on a previous invocation of the function.

Fred

That is correct, L will be bound to None whevever there is no argument
supplied. Also note that the None object is immutable, and cannot be
changed, so this mechanism is safe.

The default value is as 'persistent' as the function itself. Functions
are objects and spring into existance only after Python encounters and
executes the 'def' statement. Python also evaluates the default values
at this point and keeps these objects around, along with the function
definition.

If at a later point you do 'del f', you will delete the function and the
default values (since there is no more use for them, the function cannot
be called). Assuming, of course, that f was the only reference to the
function.

Since the 'name-binding' concept in Python is not exactly the same as
references in C++, I suggest you also read 'How to think like a Pythonista':

http://starship.python.net/crew/mwh/hacks/objectthink.html

for an excellent coverage of one of Python's basic concepts.
 
S

Sean Ross

Fred Ma said:
From the python tutorial, it's clear that to avoid sharing default
values between function calls, one has to avoid:

Example#1
---------
def f(a, L=[]):
L.append(a)
return L

Instead, one should use:

Example#2
---------
def f(a, L=None):
if L is None:
L = []
L.append(a)
return L


For fun, here's a little hack to work around that:

from copy import deepcopy

def fresh_defaults(f):
fdefaults = f.func_defaults # get f's defaults
def freshener(*args, **kwds):
f.func_defaults = deepcopy(fdefaults) # and keep fresh
return f(*args, **kwds) # between calls
return freshener

def f(a, L=[]):
L.append(a)
return L
f = fresh_defaults(f) # <= transform f


[snip]
print f(1)
print f(2)
print f(3)

prints

[1]
[1, 2]
[1, 2, 3]

now prints

[1]
[2]
[3]


Neat.

Of course, it adds indirection and slows down the code... oh well ;)

Welcome to Python,
Sean
 
M

Michael Geary

Fred said:
Thanks, Mike. That's alot clearer. The default value looks like it's
a persistent object. It is similar to a static local variable in C++.
In Example#1, that object actually got changed. In contrast, in
Example#2, the code prevents L from remaining bound to None
beyond the first line, so the None object never gets changed. In
fact, my 2nd explanation was close: that there is an unnamed
persistent object that holds the default value to be bound to L for
invocations that don't explicity supply an argument for L. The
place I went wrong was to say that L never gets bound to None
again, after it has been bound to a caller supplied object on a
previous invocation of the function.

That's just about right, Fred. More precisely, the default value is as
persistent as the function definition itself (as opposed to a particular
invocation of the function).

Shalabh's message has some more details and tips.

BTW, I'd say you're doing great for one day with the language!

-Mike
 
F

Fred Ma

Mike, Shalabh,

Thanks for the confirmations and pointer to more info on
the python paradigm. Regarding the progress in one day,
I was actually looking at another data munging language
on the first half of that day, but was a bit concerned
about the intricacies of the syntax, and the medium term
impact on development time. Programs start to get more
involved when I'm trying to "reconstitute" textual
circuit design info that crosses several physical lines,
spanning several files, so I would like the code to be
very clear about exactly what is happening without having
several info terminals open. I'm actually surprised that I
haven't gotten working code yet, as I thought I would just
"fall into" this cleaner syntax. Kind of worried, too,
since it seems there's more conceptual infrastructure to
establish before creating useful code, though I'm sure
not all of it is needed to do the data munging I need.
Maybe the documentation is "top-down" rather than "bottom-
up". Actually, that's not fair either, there's lots of
simple, clean examples of small tasks. Knowing what is
necessary to know is just always clearer in hindsight.

Fred
 
F

Fred Ma

Brian, Sean,

Thanks for the clarification and example. I'll need a bit more
time to get my head around the example, but I'm pretty clear about
what's happening in the situation of default values for function
arguments, as well as the idiom to avoid sharing default values
across function calls.

Fred
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top