Something in the function tutorial confused me.

L

Lee Fleming

Hello,
I have a simple question. Say you have the following function:

def f(x, y = []):
y.append(x)
return y

print f(23) # prints [23]
print f(42) # prints [23, 42]

As far as I understand, the default value y, an empty list, is created
when the def statement evaluates. With this thought in mind, the above
calls
to f make sense.

But this, the code that "fixes" the list accumulation confounds me:
def f(x, y=None):
if y is None: y = []
y.append(x)
return y

print f(23) # prints [23]
print f(42) # prints [42]

Why didn't the second call to f, f(42) return [23, 42]?
As I understand it, y is only None at the beginning of f(23).
Then y changes from None to 23. When f ends, doesn't y still have 23
in it,
just as it did in the first function I discussed?
And if y has 23 in it, won't the second call to f not execute what's
in the if statement?

In other words, what's going on here? How is it that y accumulates
argument values between function calls in the first function, but
doesn't in the second one?
 
M

Marc 'BlackJack' Rintsch

But this, the code that "fixes" the list accumulation confounds me:
def f(x, y=None):
if y is None: y = []
y.append(x)
return y

print f(23) # prints [23]
print f(42) # prints [42]

Why didn't the second call to f, f(42) return [23, 42]?
As I understand it, y is only None at the beginning of f(23).
Then y changes from None to 23. When f ends, doesn't y still have 23
in it, just as it did in the first function I discussed?
Click to expand...

After the function's end, the name local name `y` doesn't exist anymore.
If you enter a function, the names of the arguments with default values
are bound to those default values. So in the second call `y` is bound to
`None` just like in the first example `y` is bound to the list at every
call.
In other words, what's going on here? How is it that y accumulates
argument values between function calls in the first function, but
doesn't in the second one?
Click to expand...

Not `y` is accumulating but the list object does.

Ciao,
Marc 'BlackJack' Rintsch
 
S

Stargaming

Hello,
I have a simple question. Say you have the following function:

def f(x, y = []):
y.append(x)
return y

print f(23) # prints [23]
print f(42) # prints [23, 42]

As far as I understand, the default value y, an empty list, is created
when the def statement evaluates. With this thought in mind, the above
calls
to f make sense.

But this, the code that "fixes" the list accumulation confounds me: def
f(x, y=None):
if y is None: y = []
y.append(x)
return y

print f(23) # prints [23]
print f(42) # prints [42]

Why didn't the second call to f, f(42) return [23, 42]? As I understand
it, y is only None at the beginning of f(23). Then y changes from None
to 23. When f ends, doesn't y still have 23 in it,
just as it did in the first function I discussed? And if y has 23 in it,
won't the second call to f not execute what's in the if statement?

In other words, what's going on here? How is it that y accumulates
argument values between function calls in the first function, but
doesn't in the second one?
Click to expand...

You're just unluckily shadowing the name `y` in the local scope of your
function. Your code snippet could be rewritten as::

def f(x, y=None):
if y is None: my_y = []
else: my_y = y
my_y.append(x)
return my_y

HTH,
Stargaming
 
A

Alex Popescu

Hello,
I have a simple question. Say you have the following function:

def f(x, y = []):
y.append(x)
return y

print f(23) # prints [23]
print f(42) # prints [23, 42]

As far as I understand, the default value y, an empty list, is created
when the def statement evaluates. With this thought in mind, the above
calls
to f make sense.

But this, the code that "fixes" the list accumulation confounds me: def
f(x, y=None):
if y is None: y = []
y.append(x)
return y

print f(23) # prints [23]
print f(42) # prints [42]

Why didn't the second call to f, f(42) return [23, 42]? As I understand
it, y is only None at the beginning of f(23). Then y changes from None
to 23. When f ends, doesn't y still have 23 in it,
just as it did in the first function I discussed? And if y has 23 in it,
won't the second call to f not execute what's in the if statement?

In other words, what's going on here? How is it that y accumulates
argument values between function calls in the first function, but
doesn't in the second one?
Click to expand...

You're just unluckily shadowing the name `y` in the local scope of your
function. Your code snippet could be rewritten as::

def f(x, y=None):
if y is None: my_y = []
else: my_y = y
my_y.append(x)
return my_y

HTH,
Stargaming
Click to expand...

For the given example this will continue to print:
print f(23) # prints [23]
print f(42) # prints [42]
Click to expand...

so this doesn't solve/explain OP's initial question. A previous post has
already clarified the reasons for seeing this normal behavior.

bests,
../alex
 
N

Neil Cerutti

I have a simple question. Say you have the following function:

def f(x, y = []):
y.append(x)
return y

print f(23) # prints [23]
print f(42) # prints [23, 42]

def f(x, y=None):
if y is None: y = []
y.append(x)
return y

print f(23) # prints [23]
print f(42) # prints [42]
Click to expand...

Default argument expressions are evaluated only once,
specifically, at the time the function definition is executed.

Statements inside the function are executed every time the
function is called.
Why didn't the second call to f, f(42) return [23, 42]?
Click to expand...

Because when the function is called, the line
if y is None: y = []
Click to expand...

is executed, binding a brand new empty list to y. This
"rebinding" happens every time the function is called, unless you
provide an argument for y that is not None.

For example, with the second definition...
[23, 42]

See:

http://www.ferg.org/projects/python_gotchas.html#contents_item_6

This has been called a gotcha for a good reason. The sequence of
events for the binding of default arguments isn't obvious, and
for expressions that result in mutable objects, it's not
intuitive.
 
S

Stargaming

[email protected]:
[snip]
You're just unluckily shadowing the name `y` in the local scope of your
function. Your code snippet could be rewritten as::

def f(x, y=None):
if y is None: my_y = []
else: my_y = y
my_y.append(x)
return my_y

HTH,
Stargaming
Click to expand...

For the given example this will continue to print:
print f(23) # prints [23]
print f(42) # prints [42]
Click to expand...

so this doesn't solve/explain OP's initial question. A previous post has
already clarified the reasons for seeing this normal behavior.

bests,
./alex
Click to expand...

If it keeps normal behaviour, that's exactly what it ought to explain. In
his local scope, there is an `y`, having the value of f.func_defaults.
Because it's harder to see "hey, in some cases, y vanishes, in some it
survives", I invented the new local reference `my_y` -- which should be
clear to go away after completion of the function body.

Regards,
Stargaming
 
L

Lee Fleming

Because when the function is called, the line

if y is None: y = []
Click to expand...


is executed, binding a brand new empty list to y. This
"rebinding" happens every time the function is called, unless you
provide an argument for y that is not None.

Thanks for the prompt replies. But I am still confused. This is what
confuses me....
The first time you call the function, say with f(23), after the
function ends,
y no longer equals None. Therefore, calling f again, this time like
this f(24),
should make y equal [23,24], because the 'if y == None' test fails, or
at least I think it
fails, because y.append(x) added something that was not equal to None
during the previous call.

Please help me!
 
L

Lee Fleming

When you call f(23), the variable y within it gets created and points at
None. When f(23) exits, the y that it created gets destroyed. (Well,
goes out of scope, but even if it's not garbage collected it won't ever
come back into scope.) When you then call f(24), a new y is created
that also points to None, and disappears forever when f(24) exits.

The values in a def statement are created when the def is executed, but
the variables are only created when the function is actually called, and
new ones are created every time the function is called.
Click to expand...

why isn't the y in def f (x, y = []): something
garbage-collected?
 
N

Neil Cerutti

Because when the function is called, the line

if y is None: y = []
Click to expand...


is executed, binding a brand new empty list to y. This
"rebinding" happens every time the function is called, unless
you provide an argument for y that is not None.

Thanks for the prompt replies. But I am still confused. This is
what confuses me.... The first time you call the function, say
with f(23), after the function ends, y no longer equals None.
Therefore, calling f again, this time like this f(24), should
make y equal [23,24], because the 'if y == None' test fails, or
at least I think it fails, because y.append(x) added something
that was not equal to None during the previous call.

Please help me!
Click to expand...

Let's back up a little, and examine a model of the sequence of events.
.... if y is None:
.... y = []
.... y.append(x)
.... return y
f(2, f(1)) [1, 2]
f(0)
Click to expand...
Click to expand...
Click to expand...
[0]

First, Python reads and executes the function definition. This is
the only time that the default argument expressions of f are
evaluated. None is evaluated, resulting in None, and this is
stored somewhere conveniently inside the function representing f
(see f.func_defaults, for example).

Next, Python evaluates the function call "f(2, f(1))". To do
this, it must first evaluate the function call's arguments. 2
evaluates to 2 (that was easy!).

Next, the expression "f(1)" is evaluated. The name x is bound to
1, and, since y's positional argument was not provided, the name
y is bound to the previously stored default argument.

Now the statements in f are executed. The if statement checks to
see if y is None. It is, so y is rebound to a new empty list
object ("y = []"). Then, the object bound to x (1) is appended to
the list bound to y, and finally that list is returned. It
becomes the second argument of the "outer" call of f.

For this call to f, x is bound to 2, and y is bound to the list
object returned by the previous call to f. Since y is not None
this time, the function simply appends 2 to [1] and returns the
resultant list: [1, 2].

Finally, "f(0)" is evaluated. This calls f, binding x to 0. y is
again bound to the stored default argument, None. As before, this
results in the statement "y = []" binding a new empty list to the
name y. Then 0 is appened to that list, and finally the list is
returned: [0].
 
N

Neil Cerutti

When you call f(23), the variable y within it gets created and points at
None. When f(23) exits, the y that it created gets destroyed. (Well,
goes out of scope, but even if it's not garbage collected it won't ever
come back into scope.) When you then call f(24), a new y is created
that also points to None, and disappears forever when f(24) exits.

The values in a def statement are created when the def is executed, but
the variables are only created when the function is actually called, and
new ones are created every time the function is called.
Click to expand...

why isn't the y in def f (x, y = []): something
garbage-collected?
Click to expand...

Because the result of evaluating [] is stored somewhere in f as a
default argument. The result of evaluating [] is a new empty
list. It will never be garbage collected, because f maintains a
reference to it. Note that while [] and None may appear to be
similar expressions, they are not. None evaluated to itself,
while [] evaluates to a new empty list.
None is None True
[] is []
Click to expand...
Click to expand...
Click to expand...
False
 
M

Marc 'BlackJack' Rintsch

why isn't the y in def f (x, y = []): something
garbage-collected?
Click to expand...

`y` is a name. Only objects are garbage collected. There is no `y` in
that ``def`` in the sense that a local name `y` exists when the ``def`` is
executed. The line just says there will be a local name `y` if
the function `f()` is executed and that local name will be bound to the
given object. Which happen to be a list. This list is referenced by the
function object, so it won't get garbage collected, and it is bound to a
local name `y` every time the function is called. It is always the
very same list object. And if you mutate it, this will be visible to
other calls to the function.

Ciao,
Marc 'BlackJack' Rintsch
 
L

Lee Fleming

Thanks for all the help, everyone. I guess I was confused with default
arguments that were mutable and immutable. I will continue to look
over these posts until I understand what is happening.

I cannot believe the number of helpful responses I got!
 
S

Steve Holden

Lee said:
Because when the function is called, the line

if y is None: y = []
Click to expand...


is executed, binding a brand new empty list to y. This
"rebinding" happens every time the function is called, unless you
provide an argument for y that is not None.

Thanks for the prompt replies. But I am still confused. This is what
confuses me....
The first time you call the function, say with f(23), after the
function ends,
y no longer equals None. Therefore, calling f again, this time like
this f(24),
should make y equal [23,24], because the 'if y == None' test fails, or
at least I think it
fails, because y.append(x) added something that was not equal to None
during the previous call.

Please help me!
Click to expand...
The thing that doesn't seem to have been stated explicitly is that each
function call creates a brand-new namespace. This namespace is
initialized from the function definition, which includes assignment of
default values to absent keyword arguments. The function definition,
then, contains a reference to each default argument value.

In the case that tripped you up, however, the default argument you
provided was a mutable object (a list) that can be changed in place by
operations like .append(), whereas when you provide a None default any
further assignment to the argument name binds it to a new value. In the
first case, changes to the object are naturally reflected in later
calls, since it is the object referenced in the function definition that
is being mutated.

It's only really the same difference as

a = [1, 2]
b = a
a.append("three")

which modifies a single list to which both names, a and b, are bound, and

a = 5
b = a
a += 3

which rebinds the name a to a new object, meaning that a and b are no
longer bound to the same object.

regards
Steve

--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
--------------- Asciimercial ------------------
Get on the web: Blog, lens and tag the Internet
Many services currently offer free registration
----------- Thank You for Reading -------------
 
L

Lee Fleming

Lee said:
Because when the function is called, the line
if y is None: y = []
Click to expand...
Click to expand...
is executed, binding a brand new empty list to y. This
"rebinding" happens every time the function is called, unless you
provide an argument for y that is not None.
Click to expand...
Thanks for the prompt replies. But I am still confused. This is what
confuses me....
The first time you call the function, say with f(23), after the
function ends,
y no longer equals None. Therefore, calling f again, this time like
this f(24),
should make y equal [23,24], because the 'if y == None' test fails, or
at least I think it
fails, because y.append(x) added something that was not equal to None
during the previous call.
Click to expand...
Please help me!
Click to expand...

The thing that doesn't seem to have been stated explicitly is that each
function call creates a brand-new namespace. This namespace is
initialized from the function definition, which includes assignment of
default values to absent keyword arguments. The function definition,
then, contains a reference to each default argument value.

In the case that tripped you up, however, the default argument you
provided was a mutable object (a list) that can be changed in place by
operations like .append(), whereas when you provide a None default any
further assignment to the argument name binds it to a new value. In the
first case, changes to the object are naturally reflected in later
calls, since it is the object referenced in the function definition that
is being mutated.

It's only really the same difference as

a = [1, 2]
b = a
a.append("three")

which modifies a single list to which both names, a and b, are bound, and

a = 5
b = a
a += 3

which rebinds the name a to a new object, meaning that a and b are no
longer bound to the same object.

regards
Steve

--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
--------------- Asciimercial ------------------
Get on the web: Blog, lens and tag the Internet
Many services currently offer free registration
----------- Thank You for Reading -------------- Hide quoted text -

- Show quoted text -
Click to expand...

Thanks! I'm glad you all took the time out to help me. :)
 
S

Steve Holden

Marc said:
why isn't the y in def f (x, y = []): something
garbage-collected?
Click to expand...

`y` is a name. Only objects are garbage collected. There is no `y` in
that ``def`` in the sense that a local name `y` exists when the ``def`` is
executed. The line just says there will be a local name `y` if
the function `f()` is executed and that local name will be bound to the
given object.
Click to expand...

Unless the user provides a value for the argument in the call.
Which happen to be a list. This list is referenced by the
function object, so it won't get garbage collected, and it is bound to a
local name `y` every time the function is called. It is always the
very same list object. And if you mutate it, this will be visible to
other calls to the function.
Click to expand...
regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
--------------- Asciimercial ------------------
Get on the web: Blog, lens and tag the Internet
Many services currently offer free registration
----------- Thank You for Reading -------------
 
?

=?ISO-8859-1?Q?Ricardo_Ar=E1oz?=

Lee said:
Thanks for all the help, everyone. I guess I was confused with default
arguments that were mutable and immutable. I will continue to look
over these posts until I understand what is happening.

I cannot believe the number of helpful responses I got!
Click to expand...

Apparently he didn't understand.
Neither did I.

Either (i)y's initial value (None or []) is saved somewhere to be
retrieved every time the function is called without 2nd argument, or
(ii) y keeps the value it has when last exiting the function (if there
is a third option, please mention it).

(i) (a) whenever the function is called without 2nd argument the value
None is retrieved and assigned to y, thus causing [] to be assigned to y
by the 'if' statement.
(i) (b) But then if it is "def f(x, y = [])" the list [] should be ALSO
saved somewhere and when the function is called without 2nd argument it
should be retrieved and assigned to y, thus y would always be [] when
you enter the function without 2nd arg.

(ii) (b) if y keeps the value it has when last exiting the function that
would explain that the second time you call it the list returned will
have two members.
(ii) (a) But then if it is "def f(x, Y = None)" when the "if" is
evaluated the empty list is assigned to y. So y will NO LONGER be None.
The second time the function is called y is NOT None (remember it keeps
it's last value) and the function returns a list with two members.

So, as I see it, the behavior is not coherent, the function treats y
differently according to the value it has assigned.

Please correct me!
 
S

Steve Holden

Ricardo said:
Lee said:
Thanks for all the help, everyone. I guess I was confused with default
arguments that were mutable and immutable. I will continue to look
over these posts until I understand what is happening.

I cannot believe the number of helpful responses I got!
Click to expand...

Apparently he didn't understand.
Neither did I.

Either (i)y's initial value (None or []) is saved somewhere to be
retrieved every time the function is called without 2nd argument, or
(ii) y keeps the value it has when last exiting the function (if there
is a third option, please mention it).

(i) (a) whenever the function is called without 2nd argument the value
None is retrieved and assigned to y, thus causing [] to be assigned to y
by the 'if' statement.
(i) (b) But then if it is "def f(x, y = [])" the list [] should be ALSO
saved somewhere and when the function is called without 2nd argument it
should be retrieved and assigned to y, thus y would always be [] when
you enter the function without 2nd arg.

(ii) (b) if y keeps the value it has when last exiting the function that
would explain that the second time you call it the list returned will
have two members.
(ii) (a) But then if it is "def f(x, Y = None)" when the "if" is
evaluated the empty list is assigned to y. So y will NO LONGER be None.
The second time the function is called y is NOT None (remember it keeps
it's last value) and the function returns a list with two members.

So, as I see it, the behavior is not coherent, the function treats y
differently according to the value it has assigned.

Please correct me!
Click to expand...

OK. The difference is that [] is a mutable value, while None is
immutable. So when the function starts out with [] as y's value, and the
value gets changed, the same value is used the next time the function is
called and the second call sees the mutated value.

When the function starts out with None as y's value any assignment to y
causes it to reference a different value (since None is immutable). So
the next time the function is called y is pointing at the None that it
was pointing at the last time the function was called (since None cannot
be mutated).

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
--------------- Asciimercial ------------------
Get on the web: Blog, lens and tag the Internet
Many services currently offer free registration
----------- Thank You for Reading -------------
 
G

greg

Steve said:
OK. The difference is that [] is a mutable value, while None is
immutable.
Click to expand...

No, it's not. It has nothing to do with mutability
vs immutability.

The difference is that in the first version the
expression [] is evaluated only *once*, when the
function is defined. Therefore there is just one
list object getting re-used.

In the second version, the expression [] gets
evaluated *each* time the function is called,
creating a new list object each time.
When the function starts out with None as y's value any assignment to y
causes it to reference a different value (since None is immutable).
Click to expand...

You're confused. Assigning to y simply causes it
to reference whatever object is being assigned.
This is always true, regardless of what y was
referencing before.

To see that the immutability of None has nothing
to do with it, try the following version:

def f(x, y = []):
y = []
y.append(x)
print y

f(17)
f(42)

Try to work out what it will do, then try it, and
see if you understand why it does what it does.
 
S

Steve Holden

greg said:
Steve said:
OK. The difference is that [] is a mutable value, while None is
immutable.
Click to expand...

No, it's not. It has nothing to do with mutability
vs immutability.
Click to expand...
I beg to differ. That has everything to do with it.
The difference is that in the first version the
expression [] is evaluated only *once*, when the
function is defined. Therefore there is just one
list object getting re-used.
Click to expand...
In exactly the same way that with a default value of None just one None
value is being re-used. The difference is that you have no way to nutate
that value in place, whereas with a list object you can indeed do that.
In the second version, the expression [] gets
evaluated *each* time the function is called,
creating a new list object each time.
Correct.
When the function starts out with None as y's value any assignment to y
causes it to reference a different value (since None is immutable).
Click to expand...

You're confused. Assigning to y simply causes it
to reference whatever object is being assigned.
This is always true, regardless of what y was
referencing before.
Click to expand...
While it's correct that rebinding y will usually cause it to reference a
different object, this need not be true of assignment. The augmented
assignment operations do no necessarily rebind their left-hand operand -
that depends on the implementation of the relevant method in whatever
type is the subject of the augmented assignment.

An *assignment* to y therefore *usually* rebinds the name y to point to
a different value. A *mutation* of the object *referenced* by y leaves y
pointing to the same value. This is why appends to a default list
argument are visible in later calls: because each call initializes y to
reference the single object that was provided as the default value,
which has been changed by previous calls.
To see that the immutability of None has nothing
to do with it, try the following version:

def f(x, y = []):
y = []
y.append(x)
print y

f(17)
f(42)

Try to work out what it will do, then try it, and
see if you understand why it does what it does.
Click to expand...
I know exactly what it will do without running it, thank you. Each call
will return a single-valued list. It's hardly relevant, though since the
value of the default argument isn't used anywhere in the function body.
You are correct in stating that the [] construct always creates a new
list. However, your explanation of why a list as a default argument
retains mutations in further calls seems to me to omit some important
details.

For some reason your reply got right up my nose, so I have had to
restrain the impulse to be unpleasant. I'm left wondering if that's
because of something in your reply (such as your assumption that your
understanding of this situation is superior to mine) or because I got
out of the wrong side of bed this morning. I'm currently giving you the
benefit of the doubt, and going for another cup of coffee.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
--------------- Asciimercial ------------------
Get on the web: Blog, lens and tag the Internet
Many services currently offer free registration
----------- Thank You for Reading -------------
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads

Trouble with prediction code, for the life of me I can't figure out why it isnt running properly. Help would be appreciated. 0
Why is Python telling me variable is local not global? 3
[C language] Issue in the Lotka-Volterra model. 0
Minimising chi square to fit two parameters 1
Confused compare function :) 2
How to Use OOPS instead of displaying to the window in a list 0
Function is not worked in C 2
Modifying the default argument of function 6

Members online

No members online now.
Total: 35 (members: 0, guests: 35)
Robots: 443

Forum statistics

Threads
473,768
Messages
2,569,574
Members
45,048
Latest member
verona

Latest Threads

Top