Default arguments, object sharing, name lookup and others

[Maciej Sobczak]

Hi,

Playing around with the Python Tutorial I found the following definition:

def f(a,L=[]):
L.append(a)
return L

then:

f(1)
f(2)
f(3)

will accumulate the values appending them to the *same* list.

Now:

def f(a,s=''):
s = s + a
return s

f('hello')
f('world')

This will not cause value accumulation.
Interestingly, this will neither:

def f(a,L=[]):
L = L + [a]
return L

which is most confusing for me.

I do not understand how this works (the first one).
I would like to ask you for some explanation, especially:
- where is the object stored if it is shared between subsequent calls?
how it is found?
- why does it work for lists and not for strings?
- why does it work for lists only when the append method is used?

My "native" language is C++. Feel free to use analogies, where appropriate.

Thank you very much for any light,

 

[anton muhin]

Hi,

Playing around with the Python Tutorial I found the following definition:

def f(a,L=[]):
L.append(a)
return L

then:

f(1)
f(2)
f(3)

will accumulate the values appending them to the *same* list.

Now:

def f(a,s=''):
s = s + a
return s

f('hello')
f('world')

This will not cause value accumulation.
Interestingly, this will neither:

def f(a,L=[]):
L = L + [a]
return L

which is most confusing for me.

I do not understand how this works (the first one).
I would like to ask you for some explanation, especially:
- where is the object stored if it is shared between subsequent calls?
how it is found?
- why does it work for lists and not for strings?
- why does it work for lists only when the append method is used?

My "native" language is C++. Feel free to use analogies, where appropriate.

Thank you very much for any light,

It' one of most famous Python's traps. Consider the following:

def foo1(a, L = []):
print id(L)
L = L + [a]
print id(L)
return L

def foo2(a, L = []):
print id(L)
L.append(a)
print id(L)
return L

def foo3(a, L = ''):
print id(L)
L += a
print id(L)
return L

print 'foo1'
foo1('a')
print

print 'foo2'
foo2('a')
print

print 'foo3'
foo3('a')
print

id returns an object's id, usually it's an address.
The code produces:

foo1
8276816
8276848

foo2
8276144
8276144

foo3
7774264
7919296

You can note, that only foo2 doesn't change L's id. That's the
problem---each call populates the same L.

In more details:

L = L + [a] creates a new list
L.append(a) modifies a list in-place
s = s + a creates a new string. BTW, strings in Python are immutable.

How to cope with it. First of all, immutable default parameters
prefered. If you need a mutable one, usual trick is somehting like this:

def foo(a, L = None):
if L is None:
return [a]
else:
return L + [a]

Or, in most of the cases, shorter:

def foo(a, L = None):
return (L or []) + [a]

For more information you can search the group---this question is really
popular one

regards,
anton.

 

[anton muhin]

anton muhin wrote:
I forgot: default parameters are created when the function is defined,
if I'm right.

anton.

 

[Skip Montanaro]

Maciej> Playing around with the Python Tutorial I found the following
Maciej> definition:

Maciej> def f(a,L=[]):
Maciej> L.append(a)
Maciej> return L

Maciej> then:

Maciej> f(1)
Maciej> f(2)
Maciej> f(3)

Maciej> will accumulate the values appending them to the *same* list.

This works because default arguments are evaluated when the function is
defined and stored with the function. When the function is called, the
default is retrieved. In this case, the default is a mutable object (a
list), so the same list keeps getting modified.

Maciej> Now:

Maciej> def f(a,s=''):
Maciej> s = s + a
Maciej> return s

Maciej> f('hello')
Maciej> f('world')

Maciej> This will not cause value accumulation.

Nope. You're rebinding the local variable s. Try this slight modification:

def f(a, s=''):
print id(s)
s = s + a
print id(s)
return s

id() prints the address of the object. You should see the two print
statements display different addresses.

Maciej> Interestingly, this will neither:

Maciej> def f(a,L=[]):
Maciej> L = L + [a]
Maciej> return L

Maciej> which is most confusing for me.

Same reason.

Maciej> - where is the object stored if it is shared between subsequent
Maciej> calls?

Take a look at f.func_defaults.

Maciej> how it is found?

Index into f.func_defaults based on position in the argument list. (L is
the zeroth arg with a default, so if missing it gets assigned
f.func_defaults[0].)

Maciej> - why does it work for lists and not for strings?

Your functions are coded so that the difference is because in one
(L.append(a)) you're modifying the object to which the variable refers,
while in the other (L = L + a or s = s + a) you're rebinding the local
variable. I think you're asking a slightly wrong question here. If you
defined:

def f(a, s=''):
s += a
return s

vs

def f(a, L=[]):
L += a
return a

the apparent difference in behavior would be because lists are mutable while
strings are not.

Maciej> - why does it work for lists only when the append method is used?

It doesn't. Try L.insert() instead. You notice similar effects.

Skip

 

[Christos TZOTZIOY Georgiou]

Hi,

Playing around with the Python Tutorial I found the following definition:

[snip of confusion about names and binding]

This might help:

http://groups.google.com/[email protected]

Then again, it might not

 

[Maciej Sobczak]

Hi,

Playing around with the Python Tutorial I found the following definition:

def f(a,L=[]):
L.append(a)
return L

[...]

Thank you very much for your replies.
The most enlightening was the one mentioning func_defaults.


For your *amusement only*, here is the C++ analogy I devised for myself
to better understand what is going on:

The Python function definition is like definition *and* creation of C++
function object, initialized with what should be the default value for
further function calls:

class Fun
{
public:
Fun(shared_ptr<Any> default_value)
: def_val_(default_value) {}
Any operator()(shared_ptr<Any> arg)
{
// here goes the body of the function
// ...
}
Any operator()()
{
return operator()(def_val_);
}
private:
shared_ptr<Any> def_val_;
};

(of course, this applies to *one-argument* function with default
argument value)

Once the function object is created, it stores the default value for
future. When it is called without arguments, it retrieves the default
value from where it is stored.
Now, the hard core:
- If I do some mutating operation on the default value (like list append
or insert operation), I can see it the next time I call the function.
- If I reassign the arg shared pointer to something else (for example,
to the new value like in "L = L + a" Python expression), it does not
influence the object pointed by def_val_ pointer and is visible only to
the end of the current flow in operator(), when the local arg variable
goes out of scope.

Regards to all,