Default mutable parameters in functions

[fbicknel]

Hi all,

So I was reading about default values for functions and spied this:

Important warning: The default value is evaluated only once. This makes a difference when the default is a mutable object such as a list, dictionary, or instances of most classes.

That's ok - guess I can get used to that. So I did some experimenting and sure enough, immutables sort of behave "normally" in that they get the default value every time you call the function and don't specify the value for the parameter.

But mutables behave very strangely (imho). Take this example:

def foo(bar=[ 42 ]):
print "It's a parrot, not a cheese: {value}".format(value=bar)
bar[0] += 1

Now call it like this:
foo()
foo()
foo()
foo()

and based on the "Important warning" above, you get something expected:
It's a parrot, not a cheese: [42]
It's a parrot, not a cheese: [43]
It's a parrot, not a cheese: [44]
It's a parrot, not a cheese: [45]

Now call it with a value:
foo([ 3 ])

as you might expect:
It's a parrot, not a cheese: [3]

But now go back to no parameter in the call:
foo()
foo()
foo()

It's a parrot, not a cheese: [46]
It's a parrot, not a cheese: [47]
It's a parrot, not a cheese: [48]

it picks up where it left off.

I was rather expecting it to start with 4!

I put this into pythontutor.com's code visualization tool (http://goo.gl/XOmMjR) and it makes more sense what's going on.

I thought this was interesting; thought I would share.

 

[Ian Kelly]

Now call it with a value:
foo([ 3 ])

as you might expect:
It's a parrot, not a cheese: [3]

But now go back to no parameter in the call:
foo()
foo()
foo()

It's a parrot, not a cheese: [46]
It's a parrot, not a cheese: [47]
It's a parrot, not a cheese: [48]

it picks up where it left off.

I was rather expecting it to start with 4!

You haven't replaced the default value; you've only substituted a
different value for that call. In this function:

def foo(x=3):
print(x)

You wouldn't expect a call of foo(27) to change the default value to
27, would you?

 

[Ethan Furman]

I put this into pythontutor.com's code visualization tool (http://goo.gl/XOmMjR) and it makes more sense what's going on.

I thought this was interesting; thought I would share.

That visualization tool is certainly neat, thanks!

 

[Dennis Lee Bieber]

But mutables behave very strangely (imho). Take this example:

No, they don't...

it picks up where it left off.

As it should...

I was rather expecting it to start with 4!

def foo(bar = [42]):

.... print "I'm %s with %s" % (id(bar), bar[0])
.... bar[0] += 1
....

foo() I'm 70238984 with 42
foo() I'm 70238984 with 43
foo() I'm 70238984 with 44
foo([3]) I'm 70242184 with 3
foo([3]) I'm 70668424 with 3
foo() I'm 70238984 with 45

The default was evaluated once -- it is a list with ID 70238984 (in
this case), and that ID will not change regardless of what goes on inside
the list.

When called with [3], the function acts on a different list created
just for that call (since it was a list literal).

 

[Mark Lawrence]

Hi all,

So I was reading about default values for functions and spied this:
[snip]


I was rather expecting it to start with 4!

I just wish I had a quid for every time somebody expects something out
of Python, that way I'd have retired years ago. At least here it's not
accompanied by "as that's how it works in <some other language>".

 

[random832]

I just wish I had a quid for every time somebody expects something out
of Python, that way I'd have retired years ago. At least here it's not
accompanied by "as that's how it works in <some other language>".

I can't imagine a language that would work that way. For one, it would
also imply that passing a value would change the default for future
calls even for non-mutable types.

 

[Steven D'Aprano]

I can't imagine a language that would work that way.

That seems like a failure of imagination to me. At least, I can't imagine
anyone unable to imagine a language like that

For one, it would
also imply that passing a value would change the default for future
calls even for non-mutable types.

Not all programming languages distinguish between mutable and non-mutable
types. Or for that matter even have types.

But it's not hard to get that effect in Python, mutable or immutable
doesn't matter:


py> def spam(count, food="spam"):
.... spam.__defaults__ = (food,)
.... return food*count
....
py> spam(5)
'spamspamspamspamspam'
py> spam(3, 'eggs')
'eggseggseggs'
py> spam(5)
'eggseggseggseggseggs'
py> spam(5, 3)
15
py> spam(4)
12


Is it so unusual for a function to want to store persistent state which
survives from one call to another but may also vary from time to time?
Managing situations like that is one of the reasons OOP was invented!

 

[Chris Angelico]

But it's not hard to get that effect in Python, mutable or immutable
doesn't matter:


py> def spam(count, food="spam"):
... spam.__defaults__ = (food,)
... return food*count
...
py> spam(5)
'spamspamspamspamspam'
py> spam(3, 'eggs')
'eggseggseggs'
py> spam(5)
'eggseggseggseggseggs'
py> spam(5, 3)
15
py> spam(4)
12


Is it so unusual for a function to want to store persistent state which
survives from one call to another but may also vary from time to time?
Managing situations like that is one of the reasons OOP was invented!

Maybe it'd be clearer if we change the names around.

def signal(sig, handler=None):
ret = signals[sig]
if handler:
signals[sig] = handler
return ret

This neatly merges the definitions of signal.signal() and
signal.getsignal(); per the docs:

"""
signal.getsignal(signalnum)

Return the current signal handler for the signal signalnum. The
returned value may be a callable Python object, or one of the special
values signal.SIG_IGN, signal.SIG_DFL or None. Here, signal.SIG_IGN
means that the signal was previously ignored, signal.SIG_DFL means
that the default way of handling the signal was previously in use, and
None means that the previous signal handler was not installed from
Python.

signal.signal(signalnum, handler)

Set the handler for signal signalnum to the function handler. handler
can be a callable Python object taking two arguments (see below), or
one of the special values signal.SIG_IGN or signal.SIG_DFL. The
previous signal handler will be returned (see the description of
getsignal() above). (See the Unix man page signal(2).)
"""

(Incidentally, SIG_IGN and SIG_DFL are just integers. Are they targets
for enumification?)
other_ctrl_c_handler


I think that's a perfectly reasonable API... it just doesn't happen to
be how Python works by default.

ChrisA

 

[Dennis Lee Bieber]

I can't imagine a language that would work that way. For one, it would
also imply that passing a value would change the default for future
calls even for non-mutable types.

Some early FORTRAN compilers purportedly had problems with, for
example:

X = 1
call mutant(1)
Y = 1

where

subroutine mutant(y)

y = y + 1
return

meant that Y now held the value of 2 -- that is, literals were stored in
mutable memory, and since FORTRAN passes by reference, the address of the
literal is passed, and the assignment changed the "constant".

 

[Roy Smith]

Some early FORTRAN compilers purportedly had problems with, for
example:

X = 1
call mutant(1)
Y = 1

where

subroutine mutant(y)

y = y + 1
return

meant that Y now held the value of 2 -- that is, literals were stored in
mutable memory, and since FORTRAN passes by reference, the address of the
literal is passed, and the assignment changed the "constant".

Problem? I always assumed it was a feature

 

[Dave Angel]

Some early FORTRAN compilers purportedly had problems with, for
example:

X = 1
call mutant(1)
Y = 1

where

subroutine mutant(y)

y = y + 1
return

meant that Y now held the value of 2 -- that is, literals were stored in
mutable memory, and since FORTRAN passes by reference, the address of the
literal is passed, and the assignment changed the "constant".

I can confirm that, first hand.

In late 60's, CDC 6400, I deliberately wrote code to exploit that.
Not for production of course.