Once-only evaluation of default parameter values function definitions

Discussion in 'Python' started by Fred Ma, Apr 13, 2004.

  1. Fred Ma

    Fred Ma Guest

    Hello,

    I'm looking at Python for the 1st time today, motivated by its clarity
    of syntax. I've looked at the tutoarial at
    http://www.python.org/doc/current/tut/tut.html, particularly at the
    section about default arguments. I've got some C++/STL under my belt,
    so I'm familiar with passing/returning by reference, by value, and by
    pointers. I've also used containers of pointers, as well as c++
    default parameter values. I am aware of the general idea of reference
    counting, but have never used it.

    From the python tutorial, it's clear that to avoid sharing default
    values between function calls, one has to avoid:

    Example#1
    ---------
    def f(a, L=[]):
    L.append(a)
    return L

    Instead, one should use:

    Example#2
    ---------
    def f(a, L=None):
    if L is None:
    L = []
    L.append(a)
    return L

    None has been described a analogous to the NULL value in c++. I'm
    been looking and googling this newsgroup to get a clearer explanation
    of why this works. The tutorial says that the default value is only
    evaluated once. By that, I assume that there is some unnamed object
    that is given the value of the 1st and only evaluation of the default
    expression. Every time the function is called without an L parameter,
    L gets the value of this unnamed object. But that is not consistent
    with the Example#1 i.e.

    print f(1)
    print f(2)
    print f(3)

    prints

    [1]
    [1, 2]
    [1, 2, 3]

    The unnamed object that I imagined should have the value [], and L
    should get set to [] every time that f() is called without a value
    specified for L.

    The alternative explanation that I could think of is that L is bound
    to the unnamed object [], and the object itself changes values to
    reflect changes to L i.e. L is now a persistent variable, retaining
    its value between function calls unless a value is provided for L in
    the function call's argument list. In the latter case, one can
    imagine L simply being overwritten with the value provided.

    The problem with this picture is that Example#2 should fail for the
    same reasons as Example#1. That is, L will not get the value of None
    on the 2nd call to f() without a value specified for L. Hence, L will
    not be reset to [].

    I searched through the tutorials for programmers. The same idiom is
    presented, but I can't seem to find a more detailed explanation. Some
    threads here presented None as something that gets treated differently
    from a value, so I confirmed from the language reference that None is
    not just a value, but its own type. The thread "Default Parameters to
    function!" quotes the language reference in saying that the object
    that L is bound to actually gets changed, when the function body makes
    changes to the variable L. Again, if this was the case, it seems like
    that should cause Example#2 to fail as well (obviously something I'm
    missing here).

    I'd appreciate it if someone could provide an explanation that's
    suitable for my background (some C++ experience, but not a
    practitioner of the more advanced concepts). Thanks.

    Fred
    Fred Ma, Apr 13, 2004
    #1
    1. Advertising

  2. Fred Ma wrote:
    > From the python tutorial, it's clear that to avoid sharing
    > default values between function calls, one has to avoid:
    >
    > Example#1
    > ---------
    > def f(a, L=[]):
    > L.append(a)
    > return L
    >
    > Instead, one should use:
    >
    > Example#2
    > ---------
    > def f(a, L=None):
    > if L is None:
    > L = []
    > L.append(a)
    > return L
    >
    > None has been described a analogous to the NULL value in
    > c++. I'm been looking and googling this newsgroup to get
    > a clearer explanation of why this works. The tutorial says
    > that the default value is only evaluated once. By that, I
    > assume that there is some unnamed object that is given the
    > value of the 1st and only evaluation of the default expression.
    > Every time the function is called without an L parameter, L
    > gets the value of this unnamed object.


    This is where you went wrong. It looks like you're thinking that the =
    operator copies a value into a variable as it might in C++. It doesn't.
    Instead, = binds a name to an object.

    In C++ terms, think of L as a *reference* to that list (array) object.

    > The unnamed object that I imagined should have the value
    > [], and L should get set to [] every time that f() is called
    > without a value specified for L.
    >
    > The alternative explanation that I could think of is that L is
    > bound to the unnamed object [], and the object itself
    > changes values to reflect changes to L i.e. L is now a
    > persistent variable, retaining its value between function
    > calls unless a value is provided for L in the function call's
    > argument list. In the latter case, one can imagine L simply
    > being overwritten with the value provided.


    Python doesn't have variables. It has names and objects. The = operator
    binds a name to an object (i.e. makes the name a reference to an object).

    When the function is defined, the [] is evaluated and an empty list object
    created. Then, every time the function is called, the local name L is bound
    to that object. But, it's bound to the *same* object every time. L doesn't
    get a copy of that object; the name L refers directly to that object.

    That's why the append() call keeps appending to the list. It's the same list
    every time you call the function.

    Here's a simpler example:

    >>> a = []
    >>> b = a
    >>> a

    []
    >>> b

    []
    >>> a.append(1)
    >>> a

    [1]
    >>> b

    [1]
    >>>


    See what happened? The statement 'b = a' didn't make a copy of a and store
    it in b. Instead, it made the name b refer to the same object as a. So, when
    I said b.append(1), it meant exactly the same thing as if I'd said
    a.append(1).

    -Mike
    Michael Geary, Apr 13, 2004
    #2
    1. Advertising

  3. Fred Ma

    Fred Ma Guest

    Re: Once-only evaluation of default parameter values functiondefinitions

    Michael Geary wrote:

    > > Example#1
    > > ---------
    > > def f(a, L=[]):
    > > L.append(a)
    > > return L
    > >
    > > Example#2
    > > ---------
    > > def f(a, L=None):
    > > if L is None:
    > > L = []
    > > L.append(a)
    > > return L

    >
    > Here's a simpler example:
    >
    > >>> a = []
    > >>> b = a
    > >>> a

    > []
    > >>> b

    > []
    > >>> a.append(1)
    > >>> a

    > [1]
    > >>> b

    > [1]
    >
    > See what happened? The statement 'b = a' didn't make a copy of a and
    > store it in b. Instead, it made the name b refer to the same object
    > as a. So, when I said b.append(1), it meant exactly the same thing
    > as if I'd said a.append(1).


    Thanks, Mike. That's alot clearer. The default value looks like it's
    a persistent object. It is similar to a static local variable in C++.
    In Example#1, that object actually got changed. In contrast, in
    Example#2, the code prevents L from remaining bound to None beyond the
    first line, so the None object never gets changed. In fact, my 2nd
    explanation was close: that there is an unnamed persistent object that
    holds the default value to be bound to L for invocations that don't
    explicity supply an argument for L. The place I went wrong was to say
    that L never gets bound to None again, after it has been bound to a
    caller supplied object on a previous invocation of the function.

    Fred
    Fred Ma, Apr 13, 2004
    #3
  4. Fred Ma

    Brian Gough Guest

    Fred Ma <> writes:

    > I'm looking at Python for the 1st time today, motivated by its clarity
    > of syntax. I've looked at the tutoarial at
    > http://www.python.org/doc/current/tut/tut.html, particularly at the
    > section about default arguments. I've got some C++/STL under my belt,
    > so I'm familiar with passing/returning by reference, by value, and by
    > pointers.....
    >
    > The problem with this picture is that Example#2 should fail for the
    > same reasons as Example#1. That is, L will not get the value of None
    > on the 2nd call to f() without a value specified for L. Hence, L will
    > not be reset to [].


    Your explanation is correct up to this point, but the behavior of the
    two cases is different. When L=[] is in the body of the fuction, it
    is like a local "automatic" variable in C/C++ and discarded at the
    end of the function. When L=[] is used in the parameter list, it is
    like a "static" variable in C/C++ and persists on future calls.

    HTH

    --
    Brian Gough

    Network Theory Ltd,
    Publishing Free Software Manuals --- http://www.network-theory.co.uk/
    Brian Gough, Apr 13, 2004
    #4
  5. Re: Once-only evaluation of default parameter values functiondefinitions

    Fred Ma wrote:
    > Michael Geary wrote:
    >
    >
    >>> Example#1
    >>> ---------
    >>> def f(a, L=[]):
    >>> L.append(a)
    >>> return L
    >>>
    >>> Example#2
    >>> ---------
    >>> def f(a, L=None):
    >>> if L is None:
    >>> L = []
    >>> L.append(a)
    >>> return L

    >>
    >>Here's a simpler example:
    >>
    >>
    >>>>>a = []
    >>>>>b = a
    >>>>>a

    >>
    >>[]
    >>
    >>>>>b

    >>
    >>[]
    >>
    >>>>>a.append(1)
    >>>>>a

    >>
    >>[1]
    >>
    >>>>>b

    >>
    >>[1]
    >>
    >>See what happened? The statement 'b = a' didn't make a copy of a and
    >>store it in b. Instead, it made the name b refer to the same object
    >>as a. So, when I said b.append(1), it meant exactly the same thing
    >>as if I'd said a.append(1).

    >
    >
    > Thanks, Mike. That's alot clearer. The default value looks like it's
    > a persistent object. It is similar to a static local variable in C++.
    > In Example#1, that object actually got changed. In contrast, in
    > Example#2, the code prevents L from remaining bound to None beyond the
    > first line, so the None object never gets changed. In fact, my 2nd
    > explanation was close: that there is an unnamed persistent object that
    > holds the default value to be bound to L for invocations that don't
    > explicity supply an argument for L. The place I went wrong was to say
    > that L never gets bound to None again, after it has been bound to a
    > caller supplied object on a previous invocation of the function.
    >
    > Fred


    That is correct, L will be bound to None whevever there is no argument
    supplied. Also note that the None object is immutable, and cannot be
    changed, so this mechanism is safe.

    The default value is as 'persistent' as the function itself. Functions
    are objects and spring into existance only after Python encounters and
    executes the 'def' statement. Python also evaluates the default values
    at this point and keeps these objects around, along with the function
    definition.

    If at a later point you do 'del f', you will delete the function and the
    default values (since there is no more use for them, the function cannot
    be called). Assuming, of course, that f was the only reference to the
    function.

    Since the 'name-binding' concept in Python is not exactly the same as
    references in C++, I suggest you also read 'How to think like a Pythonista':

    http://starship.python.net/crew/mwh/hacks/objectthink.html

    for an excellent coverage of one of Python's basic concepts.

    --
    Shalabh
    Shalabh Chaturvedi, Apr 13, 2004
    #5
  6. Fred Ma

    Sean Ross Guest

    "Fred Ma" <> wrote in message
    news:...
    > From the python tutorial, it's clear that to avoid sharing default
    > values between function calls, one has to avoid:
    >
    > Example#1
    > ---------
    > def f(a, L=[]):
    > L.append(a)
    > return L
    >
    > Instead, one should use:
    >
    > Example#2
    > ---------
    > def f(a, L=None):
    > if L is None:
    > L = []
    > L.append(a)
    > return L
    >



    For fun, here's a little hack to work around that:

    from copy import deepcopy

    def fresh_defaults(f):
    fdefaults = f.func_defaults # get f's defaults
    def freshener(*args, **kwds):
    f.func_defaults = deepcopy(fdefaults) # and keep fresh
    return f(*args, **kwds) # between calls
    return freshener

    def f(a, L=[]):
    L.append(a)
    return L
    f = fresh_defaults(f) # <= transform f


    [snip]
    > print f(1)
    > print f(2)
    > print f(3)
    >
    > prints
    >
    > [1]
    > [1, 2]
    > [1, 2, 3]
    >


    now prints

    [1]
    [2]
    [3]


    Neat.

    Of course, it adds indirection and slows down the code... oh well ;)

    Welcome to Python,
    Sean
    Sean Ross, Apr 13, 2004
    #6
  7. Re: Once-only evaluation of default parameter values function definitions

    Fred Ma wrote:
    > Thanks, Mike. That's alot clearer. The default value looks like it's
    > a persistent object. It is similar to a static local variable in C++.
    > In Example#1, that object actually got changed. In contrast, in
    > Example#2, the code prevents L from remaining bound to None
    > beyond the first line, so the None object never gets changed. In
    > fact, my 2nd explanation was close: that there is an unnamed
    > persistent object that holds the default value to be bound to L for
    > invocations that don't explicity supply an argument for L. The
    > place I went wrong was to say that L never gets bound to None
    > again, after it has been bound to a caller supplied object on a
    > previous invocation of the function.


    That's just about right, Fred. More precisely, the default value is as
    persistent as the function definition itself (as opposed to a particular
    invocation of the function).

    Shalabh's message has some more details and tips.

    BTW, I'd say you're doing great for one day with the language!

    -Mike
    Michael Geary, Apr 13, 2004
    #7
  8. Fred Ma

    Fred Ma Guest

    Re: Once-only evaluation of default parameter values functiondefinitions

    Mike, Shalabh,

    Thanks for the confirmations and pointer to more info on
    the python paradigm. Regarding the progress in one day,
    I was actually looking at another data munging language
    on the first half of that day, but was a bit concerned
    about the intricacies of the syntax, and the medium term
    impact on development time. Programs start to get more
    involved when I'm trying to "reconstitute" textual
    circuit design info that crosses several physical lines,
    spanning several files, so I would like the code to be
    very clear about exactly what is happening without having
    several info terminals open. I'm actually surprised that I
    haven't gotten working code yet, as I thought I would just
    "fall into" this cleaner syntax. Kind of worried, too,
    since it seems there's more conceptual infrastructure to
    establish before creating useful code, though I'm sure
    not all of it is needed to do the data munging I need.
    Maybe the documentation is "top-down" rather than "bottom-
    up". Actually, that's not fair either, there's lots of
    simple, clean examples of small tasks. Knowing what is
    necessary to know is just always clearer in hindsight.

    Fred
    --
    Fred Ma
    Dept. of Electronics, Carleton University
    1125 Colonel By Drive, Ottawa, Ontario
    Canada, K1S 5B6
    Fred Ma, Apr 13, 2004
    #8
  9. Fred Ma

    Fred Ma Guest

    Re: Once-only evaluation of default parameter values functiondefinitions

    Brian, Sean,

    Thanks for the clarification and example. I'll need a bit more
    time to get my head around the example, but I'm pretty clear about
    what's happening in the situation of default values for function
    arguments, as well as the idiom to avoid sharing default values
    across function calls.

    Fred
    --
    Fred Ma
    Dept. of Electronics, Carleton University
    1125 Colonel By Drive, Ottawa, Ontario
    Canada, K1S 5B6
    Fred Ma, Apr 13, 2004
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Ilias Lazaridis
    Replies:
    2
    Views:
    390
    Ilias Lazaridis
    Apr 24, 2005
  2. Tim Frink
    Replies:
    4
    Views:
    365
    Tim Frink
    May 9, 2008
  3. Jeremy Banks
    Replies:
    12
    Views:
    435
    Steven D'Aprano
    Apr 24, 2009
  4. Ilias Lazaridis
    Replies:
    74
    Views:
    752
    Ilias Lazaridis
    Apr 4, 2005
  5. Ilias Lazaridis
    Replies:
    18
    Views:
    332
    Bill Guindon
    Apr 9, 2005
Loading...

Share This Page