Re: functions, list, default parameters

Discussion in 'Python' started by Chris Rebert, Oct 21, 2010.

  1. Chris Rebert

    Chris Rebert Guest

    On Thu, Oct 21, 2010 at 2:36 PM, Sean Choi <> wrote:
    > I found two similar questions in the mailing list, but I didn't understand
    > the explanations.
    > I ran this code on Ubuntu 10.04 with Python 2.6.5.
    > Why do the functions g and gggg behave differently? If calls gggg(3) and
    > g(3) both exit their functions in the same state, why do they not enter in
    > the same state when I call gggg(4) and g(4)?
    >
    > # ---------------------------------------------------------------------- my
    > code:
    > def gggg(a, L=[]):


    This is a common newbie stumbling-block: Don't use lists (or anything
    mutable) as default argument values (in this case, for L); a new list
    is *not* created for every function invocation, they'll all share the
    *exact same list object*. Use None and then create a fresh list (or
    what have you) in the function body. See
    http://effbot.org/pyfaq/why-are-default-values-shared-between-objects.htm

    >     print "enter function"
    >     print "a = ", a, "and L = ", L
    >     if L == []:
    >         print "hey, L is empty"
    >         L = []


    The previous line is why the two functions' behaviors differ (g()
    lacks this line). Read the above FAQ, and then carefully trace through
    the execution of the functions; the difference will then be clear.

    Cheers,
    Chris
    --
    http://blog.rebertia.com

    >     L.append(a)
    >     print "after append, L = ", L
    >     return L


    > def g(a, L=[]):
    >     print "enter function"
    >     print "a = ", a, "and L = ", L
    >     if L == []:
    >         print "hey, L is empty"
    >     L.append(a)
    >     print "after append, L = ", L
    >     return L
     
    Chris Rebert, Oct 21, 2010
    #1
    1. Advertising

  2. Chris Rebert

    John Nagle Guest

    On 10/21/2010 2:51 PM, Chris Rebert wrote:
    > On Thu, Oct 21, 2010 at 2:36 PM, Sean Choi<> wrote:
    >> I found two similar questions in the mailing list, but I didn't understand
    >> the explanations.
    >> I ran this code on Ubuntu 10.04 with Python 2.6.5.
    >> Why do the functions g and gggg behave differently? If calls gggg(3) and
    >> g(3) both exit their functions in the same state, why do they not enter in
    >> the same state when I call gggg(4) and g(4)?
    >>
    >> # ---------------------------------------------------------------------- my
    >> code:
    >> def gggg(a, L=[]):

    >
    > This is a common newbie stumbling-block: Don't use lists (or anything
    > mutable) as default argument values


    That really should be an error.

    John Nagle
     
    John Nagle, Oct 22, 2010
    #2
    1. Advertising

  3. On Thu, 21 Oct 2010 19:53:53 -0700 John Nagle <> wrote:

    > On 10/21/2010 2:51 PM, Chris Rebert wrote:
    > > On Thu, Oct 21, 2010 at 2:36 PM, Sean Choi<> wrote:
    > >> I found two similar questions in the mailing list, but I didn't
    > >> understand the explanations.
    > >> I ran this code on Ubuntu 10.04 with Python 2.6.5.
    > >> Why do the functions g and gggg behave differently? If calls
    > >> gggg(3) and g(3) both exit their functions in the same state, why
    > >> do they not enter in the same state when I call gggg(4) and g(4)?
    > >>
    > >> #
    > >> ----------------------------------------------------------------------
    > >> my code: def gggg(a, L=[]):

    > >
    > > This is a common newbie stumbling-block: Don't use lists (or
    > > anything mutable) as default argument values

    >
    > That really should be an error.
    >

    What do you mean? That using a list as default arguments should throw
    an error?

    While not very commonly needed, why should a shared default argument be
    forbidden?

    /W

    --
    To reach me via email, replace INVALID with the country code of my home
    country. But if you spam me, I'll be one sour Kraut.
     
    Andreas Waldenburger, Oct 22, 2010
    #3
  4. On Thu, 21 Oct 2010 19:53:53 -0700, John Nagle wrote:

    >> This is a common newbie stumbling-block: Don't use lists (or anything
    >> mutable) as default argument values

    >
    > That really should be an error.


    No it shouldn't. Punishing everybody for a newbie mistake that nobody
    makes twice would be the error.

    Default mutable arguments have their place, and there is an easy work-
    around for anyone who wants default arguments to be created on function
    invocation rather than function definition.



    --
    Steven
     
    Steven D'Aprano, Oct 22, 2010
    #4
  5. John Nagle wrote:
    > On 10/21/2010 2:51 PM, Chris Rebert wrote:


    >> This is a common newbie stumbling-block: Don't use lists (or anything
    >> mutable) as default argument values


    > That really should be an error.


    No, it shouldn't. The criterion isn't whether the object is
    mutable, but whether you actually mutate it. If you treat it
    as read-only, there's no problem.

    Since not all Python data types have immutable variants,
    sometimes it's necessary to do this.

    --
    Greg
     
    Gregory Ewing, Oct 29, 2010
    #5
  6. On Fri, 29 Oct 2010 21:24:23 +1300, Gregory Ewing wrote:

    > John Nagle wrote:
    >> On 10/21/2010 2:51 PM, Chris Rebert wrote:

    >
    >>> This is a common newbie stumbling-block: Don't use lists (or anything
    >>> mutable) as default argument values

    >
    >> That really should be an error.

    >
    > No, it shouldn't. The criterion isn't whether the object is mutable, but
    > whether you actually mutate it. If you treat it as read-only, there's no
    > problem.


    And how does Python know whether some arbitrary default object is mutable
    or not?

    > Since not all Python data types have immutable variants, sometimes it's
    > necessary to do this.



    And sometimes you want the behaviour as it is.


    --
    Steven
     
    Steven D'Aprano, Oct 29, 2010
    #6
  7. Steven D'Aprano wrote:

    > And how does Python know whether some arbitrary default object is mutable
    > or not?


    It doesn't, that's the whole point.

    --
    Greg
     
    Gregory Ewing, Oct 30, 2010
    #7
  8. On Sat, 30 Oct 2010 19:31:53 +1300, Gregory Ewing wrote:

    > Steven D'Aprano wrote:
    >
    >> And how does Python know whether some arbitrary default object is
    >> mutable or not?

    >
    > It doesn't, that's the whole point.


    I think we're in violent agreement :)



    --
    Steven
     
    Steven D'Aprano, Oct 30, 2010
    #8
  9. In message <>, Andreas Waldenburger
    wrote:

    > While not very commonly needed, why should a shared default argument be
    > forbidden?


    Because it’s safer to disallow it than to allow it.
     
    Lawrence D'Oliveiro, Nov 2, 2010
    #9
  10. In message <>, Gregory Ewing wrote:

    > Steven D'Aprano wrote:
    >
    >> And how does Python know whether some arbitrary default object is mutable
    >> or not?

    >
    > It doesn't, that's the whole point.


    Of course it knows. It is the one defining the concept in the first place,
    after all.
     
    Lawrence D'Oliveiro, Nov 2, 2010
    #10
  11. Chris Rebert

    Robert Kern Guest

    On 2010-11-01 22:31 , Lawrence D'Oliveiro wrote:
    > In message<>, Gregory Ewing wrote:
    >
    >> Steven D'Aprano wrote:
    >>
    >>> And how does Python know whether some arbitrary default object is mutable
    >>> or not?

    >>
    >> It doesn't, that's the whole point.

    >
    > Of course it knows. It is the one defining the concept in the first place,
    > after all.


    No, the Python interpreter doesn't define the concept. The Python language
    developers did. "Immutable objects" are just those without an obvious API for
    modifying them. With various trickeries, I can mutate any immutable object. The
    Python interpreter doesn't know what's an "obvious API" and what isn't. It's a
    reasonably vague concept that doesn't have an algorithmic formulation.

    --
    Robert Kern

    "I have come to believe that the whole world is an enigma, a harmless enigma
    that is made terrible by our own mad attempt to interpret it as though it had
    an underlying truth."
    -- Umberto Eco
     
    Robert Kern, Nov 2, 2010
    #11
  12. In message <>, Robert Kern
    wrote:

    > On 2010-11-01 22:31 , Lawrence D'Oliveiro wrote:
    >
    >> In message<>, Gregory Ewing wrote:
    >>
    >>> Steven D'Aprano wrote:
    >>>
    >>>> And how does Python know whether some arbitrary default object is
    >>>> mutable or not?
    >>>
    >>> It doesn't, that's the whole point.

    >>
    >> Of course it knows. It is the one defining the concept in the first
    >> place, after all.

    >
    > No, the Python interpreter doesn't define the concept. The Python language
    > developers did.


    The Python language developers said “let there be lightâ€, and the Python
    interpreter gave forth light. So which one defines the concept of “light�

    > "Immutable objects" are just those without an obvious API for modifying
    > them.


    They are ones with NO legal language constructs for modifying them. Hint: if
    a selector of some part of such an object were to occur on the LHS of an
    assignment, and that would raise an error, then the object is immutable. The
    interpreter already knows all this.

    > With various trickeries, I can mutate any immutable object.


    None within the Python language itself. Which is what we’re talking about
    here: a language construct which is probably one of the top 3 sources of
    grief to Python newbies. And not-so-newbies.
     
    Lawrence D'Oliveiro, Nov 2, 2010
    #12
  13. Chris Rebert

    Chris Rebert Guest

    On Fri, Oct 22, 2010 at 12:36 AM, Steven D'Aprano
    <> wrote:
    > On Thu, 21 Oct 2010 19:53:53 -0700, John Nagle wrote:
    >>> This is a common newbie stumbling-block: Don't use lists (or anything
    >>> mutable) as default argument values

    >>
    >>      That really should be an error.

    >
    > No it shouldn't. Punishing everybody for a newbie mistake that nobody
    > makes twice would be the error.
    >
    > Default mutable arguments have their place


    But it's a rather obscure one where it is almost never strictly
    necessary to venture.

    Cheers,
    Chris
     
    Chris Rebert, Nov 2, 2010
    #13
  14. In message <>, Chris
    Rebert wrote:

    > On Fri, Oct 22, 2010 at 12:36 AM, Steven D'Aprano
    > <> wrote:
    >>
    >> Default mutable arguments have their place

    >
    > But it's a rather obscure one where it is almost never strictly
    > necessary to venture.


    Mediocre programmers with a hankering towards cleverness latch onto it as an
    ingenious way of maintaing persistent context in-between calls to a
    function, completely overlooking the fact that Python offers much more
    straightforward, comprehensible, flexible, and above all maintainable ways
    of doing that sort of thing.
     
    Lawrence D'Oliveiro, Nov 2, 2010
    #14
  15. On Tue, 02 Nov 2010 22:06:40 +1300, Lawrence D'Oliveiro wrote:

    > In message <>, Chris
    > Rebert wrote:
    >
    >> On Fri, Oct 22, 2010 at 12:36 AM, Steven D'Aprano
    >> <> wrote:
    >>>
    >>> Default mutable arguments have their place

    >>
    >> But it's a rather obscure one where it is almost never strictly
    >> necessary to venture.

    >
    > Mediocre programmers with a hankering towards cleverness latch onto it
    > as an ingenious way of maintaing persistent context in-between calls to
    > a function, completely overlooking the fact that Python offers much more
    > straightforward, comprehensible, flexible, and above all maintainable
    > ways of doing that sort of thing.


    Perhaps.

    Or maybe it's the mediocre programmers who find mutable defaults
    confusing, incomprehensible and unmaintainable.

    Certainly it's the mediocre programmers who seem to be incapable of
    understanding that Python has no way of telling whether arbitrary objects
    are mutable or not.

    def foo(x, y=list()):
    pass

    Is y a mutable default or not?

    For the benefit of any mediocre programmers out there, be careful before
    you answer. This *is* a trick question.


    --
    Steven
     
    Steven D'Aprano, Nov 2, 2010
    #15
  16. On Tue, 02 Nov 2010 20:12:49 +1300, Lawrence D'Oliveiro wrote about
    mutable defaults:

    > Which is what we’re talking about
    > here: a language construct which is probably one of the top 3 sources of
    > grief to Python newbies. And not-so-newbies.



    I call bullshit. Maybe you should spend some time on the
    mailing list for a while to see what the common problems newbies actually
    have. Mutable defaults are *way* down the list.

    In any case, Python isn't written solely for newbies. Most people will
    make this mistake once, or twice if they're particularly slow learning.
    If newbies have a problem with mutable defaults, oh well, they'll learn,
    or they'll give up and go back to griefing their friends on Facebook.
    Either way, problem solved.


    --
    Steven
     
    Steven D'Aprano, Nov 2, 2010
    #16
  17. On Tue, 02 Nov 2010 00:40:17 -0700, Chris Rebert wrote:

    > On Fri, Oct 22, 2010 at 12:36 AM, Steven D'Aprano
    > <> wrote:
    >> On Thu, 21 Oct 2010 19:53:53 -0700, John Nagle wrote:
    >>>> This is a common newbie stumbling-block: Don't use lists (or anything
    >>>> mutable) as default argument values
    >>>
    >>>      That really should be an error.

    >>
    >> No it shouldn't. Punishing everybody for a newbie mistake that nobody
    >> makes twice would be the error.
    >>
    >> Default mutable arguments have their place

    >
    > But it's a rather obscure one where it is almost never strictly
    > necessary to venture.


    Very few language features are *strictly* necessary. I mean, all you
    really need is the ability to set the current memory location, a way to
    read and write to it, and a way to branch. Everything else is just gravy.

    Not that I'm suggesting we should all use Turing machines, but there are
    many things which aren't strictly necessary but are nice to have.

    In any case, this discussion is pointless. There is code out there that
    uses this feature, whether you (generic you) like it or not, and changing
    the behaviour *can't* happen until the moratorium ends. Even if there was
    consensus to make this change -- and there won't be -- it almost
    certainly won't happen before Python 4000.

    Even if it did happen, adding extra type-checks to every def statement
    with a default value, or adding extra complexity to the parser, won't and
    can't catch every mutable default because there is no way of telling
    whether an arbitrary object is mutable. It will just slow down the
    language for bugger-all benefit.


    --
    Steven
     
    Steven D'Aprano, Nov 2, 2010
    #17
  18. Chris Rebert

    Terry Reedy Guest

    On 11/2/2010 3:12 AM, Lawrence D'Oliveiro wrote:

    >> "Immutable objects" are just those without an obvious API for modifying
    >> them.


    After initial creation ;-)/

    > They are ones with NO legal language constructs for modifying them.


    Suppose I write an nasty C extension that mutates tuples. What then
    would be illegal about

    import tuple_mutator
    t = (1,2)
    tuple_mutator.inc(t)
    t
    # (2,3)

    > Hint: if
    > a selector of some part of such an object were to occur on the LHS of an
    > assignment, and that would raise an error, then the object is immutable.


    I am not sure what you are saying here, and how it applies to

    >>> lt = [(0,)]
    >>> lt[0][0] = 1

    Traceback (most recent call last):
    File "<pyshell#1>", line 1, in <module>
    lt[0][0] = 1
    TypeError: 'tuple' object does not support item assignment
    >>> tl = ([0],)
    >>> tl[0][0] = 1
    >>> tl

    ([1],)


    --
    Terry Jan Reedy
     
    Terry Reedy, Nov 2, 2010
    #18
  19. Chris Rebert

    Robert Kern Guest

    On 11/2/10 2:12 AM, Lawrence D'Oliveiro wrote:
    > In message<>, Robert Kern
    > wrote:
    >
    >> On 2010-11-01 22:31 , Lawrence D'Oliveiro wrote:
    >>
    >>> In message<>, Gregory Ewing wrote:
    >>>
    >>>> Steven D'Aprano wrote:
    >>>>
    >>>>> And how does Python know whether some arbitrary default object is
    >>>>> mutable or not?
    >>>>
    >>>> It doesn't, that's the whole point.
    >>>
    >>> Of course it knows. It is the one defining the concept in the first
    >>> place, after all.

    >>
    >> No, the Python interpreter doesn't define the concept. The Python language
    >> developers did.

    >
    > The Python language developers said “let there be lightâ€, and the Python
    > interpreter gave forth light. So which one defines the concept of “light�


    I'm sorry, but that's not even a meaningful reply. They said no such thing and
    the interpreter does no such thing. Let's talk about things the developers did
    say and the things the interpreter does do, shall we?

    The Python language developers defined the concept of mutable objects. However,
    they defined it in human terms, not algorithmic ones. They did not imbue the
    interpreter with a way of determining immutability.

    >> "Immutable objects" are just those without an obvious API for modifying
    >> them.

    >
    > They are ones with NO legal language constructs for modifying them. Hint: if
    > a selector of some part of such an object were to occur on the LHS of an
    > assignment, and that would raise an error, then the object is immutable. The
    > interpreter already knows all this.


    Incorrect. RHS method calls can often modify objects. set.add() mutates the set
    even though it does not have the usual LHS mutation methods like .__setitem__().
    And even among the LHS APIs, raising an error does not determine immutability.
    int.__iadd__() does not raise an error, but ints are still immutable.
    list.__iadd__() does not raise an error, and lists are mutable. And even if it
    were reliable, how would the interpreter know what arguments to pass?
    list.__iadd__() doesn't work for any object. And why do you think that testing
    mutability in such a way would be safe since attempting those would necessarily
    alter the object if it is mutable?

    Show me the algorithm that the interpreter can use to determine whether or not
    an object is mutable. Or better, since you think the interpreter already knows
    this, show me the implementation in the interpreter's source code.

    >> With various trickeries, I can mutate any immutable object.

    >
    > None within the Python language itself. Which is what we’re talking about
    > here: a language construct which is probably one of the top 3 sources of
    > grief to Python newbies. And not-so-newbies.


    "import ctypes" is within the Python language.

    --
    Robert Kern

    "I have come to believe that the whole world is an enigma, a harmless enigma
    that is made terrible by our own mad attempt to interpret it as though it had
    an underlying truth."
    -- Umberto Eco
     
    Robert Kern, Nov 2, 2010
    #19
  20. Chris Rebert

    Ian Guest

    On Nov 2, 5:59 am, Steven D'Aprano <st...@REMOVE-THIS-
    cybersource.com.au> wrote:
    > Certainly it's the mediocre programmers who seem to be incapable of
    > understanding that Python has no way of telling whether arbitrary objects
    > are mutable or not.
    >
    > def foo(x, y=list()):
    >     pass
    >
    > Is y a mutabledefaultor not?
    >
    > For the benefit of any mediocre programmers out there, be careful before
    > you answer. This *is* a trick question.


    I fail to see your point. You might as well argue that Python has no
    way of knowing whether it should raise a TypeError in the following
    example:

    my_tuple = (1, 2, 3) + list(xrange(4, 7))

    Dynamic typing means that these sorts of checks must be delayed until
    runtime, but that doesn't make them useless or impossible.

    It seems to me that there is a rather simple case to be made for
    allowing mutable default arguments: instances of user-defined classes
    are fundamentally mutable. Disallowing mutable default arguments
    would mean disallowing instances of user-defined classes entirely.
    That would be excessive and would violate Python's general rule of
    staying out of the programmer's way.

    Cheers,
    Ian
     
    Ian, Nov 2, 2010
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Dave
    Replies:
    2
    Views:
    365
  2. Xiangliang Meng
    Replies:
    1
    Views:
    1,631
    Victor Bazarov
    Jun 21, 2004
  3. Chris Jefferson
    Replies:
    0
    Views:
    286
    Chris Jefferson
    Mar 7, 2005
  4. desktop
    Replies:
    2
    Views:
    469
    Victor Bazarov
    Jun 6, 2007
  5. Replies:
    10
    Views:
    85
    Dave Angel
    Apr 5, 2014
Loading...

Share This Page