Pre-PEP: Dynamically evaluating default function arguments

[Daniel Ehrenberg]

One of the most common bugs that people have on the Python Tutor list
are caused by the fact the default arguments for functions are
evaluated only once, not when the function is called. The solution to
this is usually to use the following idiom:

def write_stuff(stuff=None):
"""Function to write its argument.

Defaults to whatever is in the variable "things"
"""
if stuff is None:
stuff = things
print stuff

However, it would be much more intuitive and concise if the following
could be done instead, but with the guarantee that changes in the
variable 'things' will be noticed:

def write_stuff(stuff=things):
print stuff

This would instead print whatever is in things when the function is
defined.

The goal of this pre-PEP is to make it so that the default arguments
are evaluated dynamically instead of when the function is created. As
this is a pre-PEP, the details have not been finalized yet. One of the
first details is whether or not the default argument should be checked
to make sure it exists before the function is called. I think it would
make the most sense if things were evaluated when the function was
defined but the result was not saved and was reevaluated whenever the
function is called. This might not be the best way to do it, though,
and it might be better to treat it just like a regular equals sign.

As with any additional dynamic properties added, there is the
possibility of some errors to go uncaught and some. Here is an example
of some hypothetical (but unlikely) code that would cause an error as
a result of this change:
Traceback (most recent call last):
File "<input>", line 1, in ?
NameError: name 'x' is not defined

To me, this seems like the logical behavior, but there is still the
possiblility of some programs being broken. A way to preserve
backwards compatability could be to store the initial evaluation of
the default and use it to fall back on if the variable needed isn't
around anymore, but this seems overly complicated.

Daniel Ehrenberg

 

[Jeff Epler]

That proposal gets "-All" from me. (or, at least, I think that's what I
mean. Maybe I just mean "None", I couldn't stand that other thread)

Reams of code depend on default arguments working as they do today. For
instance:
l = []
for i in range(10):
l.append(lambda x, y=i: x+y)
or
def fib(x, y={0: 1, 1: 1}):
assert x >= 0
if not y.has_key(x):
y[x] = fib(x-1) + fib(x-2)
return y[x]
or
def stack():
def push(x, l=[]):
l.append(x)
def pop(l=push.func_defaults[0]):
return l.pop()
return push, pop

It's not clear exactly how this feature would work. What additional
state will you store with function objects to make it work? Right now,
it works like this:
def f(l=x):
pass
is equivalent to
_magic = x
def f(*args):
assert len(args) < 2
if args: l = args[0]
else: l = _magic
you're proposing something more like
_magic = lambda: x
def f(*args):
assert len(args) < 2
if args: l = args[0]
else: l = _magic()
you've just added one additional function call overhead for each
defaulted argument, plus the time to evaluate the expression 'x', even
in cases where it makes no difference. Here's a case where the
programmer has performed a micro-optimization that will probably be
slower after your change (even though the program's meaning is not
changed):
def log10(x, l=math.log, l10=math.log(10)):
return l(x)/l10

That leaves the case where you actually want the default argument value
to depend on the current program's state, as below:
def log(s, when=time.time()):
print >>stderr, time.asctime(when), s
I think it's better to write
def log(s, when=None):
if when is None: when = time.time()
print >>stderr, time.asctime(when), s
because it makes calling code easier, too:
def log_with_prefix(p, s, when=None):
log("%s: %s" % (p, s), when)
instead of repeating the default argument everwhere you may call it both ways
def log_with_prefix(p, s, when=time.time()):
log("%s: %s" % (p, s), when)
or having to test at each place you call:
def log_with_prefix(p, s, when=None):
if when is None:
log("%s: %s" % (p, s))
else:
log("%s: %s" % (p, s), when)

Jeff

 

[John Roth]

One of the most common bugs that people have on the Python Tutor list
are caused by the fact the default arguments for functions are
evaluated only once, not when the function is called. The solution to
this is usually to use the following idiom:

[snip]

It's an interesting proposal, but outside of the possible
breakage, there is one really major problem:

Where is the variable going to come from on the function
call? If it comes from the original context, it's static so why
bother, and if it comes from the caller's context, you're
causing a *huge* amount of coupling, as well as a substantial
slowdown to do a search of the calling chain and the global
context.

You can't even make it optional: use the original
value if it's not defined in the caller's context. If you
do that, you're still exposing the names to the caller
as names he has to avoid when writing the calling
routine.

To put a somewhat positive spin on it though,
it is a significant novice problem. Given that you
don't want to change the language semantics, what's
the next possible way to fix it?

Compare PEP's 215 and 292. Both of these
do dynamic variable fills, but they have one
critical difference: the template is visible, and not
off in a function or method definition somewhere.
Even so, there are significant issues that have to
be resolved.

John Roth

 

[Daniel Ehrenberg]

I'm sorry I wasted your time, everyone.