bug or feature?

Discussion in 'Python' started by beza1e1, Oct 5, 2005.

  1. beza1e1

    beza1e1 Guest

    Coming back from a bug hunt, i am not sure what to think of this python
    behaviour. Here is a demo program:

    class A:
    def __init__(self, lst=[]):
    self.lst = lst

    a = A()
    b = A()
    b.lst.append("hallo")
    print a.lst # output: ["hallo"]

    The point seems to be, that lst=[] creates a class attribute (correct
    name?), which is shared by all instances of A. So a.lst ist the same
    object as b.lst, despite the fact, that object a is different to object
    b.
     
    beza1e1, Oct 5, 2005
    #1
    1. Advertising

  2. beza1e1 wrote:
    > Coming back from a bug hunt, i am not sure what to think of this python
    > behaviour. Here is a demo program:
    >
    > class A:
    > def __init__(self, lst=[]):
    > self.lst = lst
    >
    > a = A()
    > b = A()
    > b.lst.append("hallo")
    > print a.lst # output: ["hallo"]
    >
    > The point seems to be, that lst=[] creates a class attribute (correct
    > name?), which is shared by all instances of A. So a.lst ist the same
    > object as b.lst, despite the fact, that object a is different to object
    > b.
    >


    It is an *instance attribute* by nature, since it does not reside in the
    class object, but only in its instances. The truth is, that a.lst and
    b.lst point to the same memory object, so it seems to behave much like
    the class attribute :)

    It is no more different from the simple fact, that two variables
    (attributes) may point to the same memory object, like you see below:

    a = b = []
    a.append("hallo")
    print b #output: ["hallo"]

    In fact, it is usually a bad practice to assign instance attributes a
    reference to the compound variable, existing in an external scope. Example:

    aList = []

    class A:
    def __init__(self, lst): #no default attribute!
    self.lst = lst

    a = A(aList)
    aList.append("hallo")
    print a.lst #output: ["hallo"]

    and your default value (, lst=[]) IS such an variable! The bad thing is,
    that the value of the instance attribute 'lst' (example above) depends
    on the external variable, which may be independently modified, thus
    modifying unexpectedly the instance attribute. The safer approach, of
    course is to write:

    class A:
    def __init__(self, lst): #no default attribute!
    self.lst = lst[:] #take a copy

    Summing up, is it an error, or a feature? I would say - a feature.
    Everyone should be aware, that the argument default values are evaluated
    once, and the same value (memory object) is reused at each instance
    creation.

    Best regards,
    Tomasz Lisowski
     
    Tomasz Lisowski, Oct 5, 2005
    #2
    1. Advertising

  3. Simon Percivall, Oct 5, 2005
    #3
  4. beza1e1

    Guest

    beza1e1> class A:
    beza1e1> def __init__(self, lst=[]):
    beza1e1> self.lst = lst

    Lists are mutable and default args are only evaluated once, at function
    definition. If you want independent default args use:

    class A:
    def __init__(self, lst=None):
    if lst is None:
    lst = []
    self.lst = lst

    The same scheme would work for other mutable types (dicts, sets, etc).

    This same question gets asked once a month or so. I'm sure this is in the
    Python FAQ (check the website), but it was faster to reply than to look it
    up...

    Skip
     
    , Oct 5, 2005
    #4
  5. beza1e1

    Steve Holden Guest

    beza1e1 wrote:
    > Coming back from a bug hunt, i am not sure what to think of this python
    > behaviour. Here is a demo program:
    >
    > class A:
    > def __init__(self, lst=[]):
    > self.lst = lst
    >
    > a = A()
    > b = A()
    > b.lst.append("hallo")
    > print a.lst # output: ["hallo"]
    >
    > The point seems to be, that lst=[] creates a class attribute (correct
    > name?), which is shared by all instances of A. So a.lst ist the same
    > object as b.lst, despite the fact, that object a is different to object
    > b.
    >

    Interestingly I couldn't find this in the FAQ, though it *is* a
    frequently-asked question [note: my not finding it doesn't guarantee
    it's not there]. The nearest I could get was in


    http://www.python.org/doc/faq/programming.html#my-program-is-too-slow-how-do-i-speed-it-up

    which says:

    """Default arguments can be used to determine values once, at compile
    time instead of at run time."""

    The point is that the value of the keyword argument is determined when
    the def statement is executed (which is to say when the function body is
    being bound to its name).

    If the default argument is (a reference to) a mutable object (such as a
    list instance) then if one call to the function modifies that mutable
    object, subsequent calls see the mutated instance as the default value.

    regards
    Steve
    --
    Steve Holden +44 150 684 7255 +1 800 494 3119
    Holden Web LLC www.holdenweb.com
    PyCon TX 2006 www.python.org/pycon/
     
    Steve Holden, Oct 5, 2005
    #5
  6. Steve Holden wrote:

    > Interestingly I couldn't find this in the FAQ, though it *is* a
    > frequently-asked question [note: my not finding it doesn't guarantee
    > it's not there].


    it's there:

    http://www.python.org/doc/faq/general.html#why-are-default-values-shared-between-objects

    (maybe "default values" should be changed to "default argument values")

    it's also mentioned in chapter 4 of the tutorial:

    http://docs.python.org/tut/node6.html#SECTION006710000000000000000

    "*Important warning*: The default value is evaluated only once. This
    makes a difference when the default is a mutable object such as a list,
    dictionary, or instances of most classes. "

    (the text then illustrates this with examples, and shows how to do things
    instead)

    and in the description of "def" in the language reference:

    http://docs.python.org/ref/function.html

    "*Default parameter values are evaluated when the function definition
    is executed*. This means that the expression is evaluated once, when the
    function is defined, and that that same "pre-computed" value is used for
    each call. This is especially important to understand when a default para-
    meter is a mutable object, such as a list or a dictionary: if the function
    modifies the object (e.g. by appending an item to a list), the default
    value is in effect modified."

    (the text then shows how to do things instead)

    </F>
     
    Fredrik Lundh, Oct 5, 2005
    #6
  7. beza1e1

    beza1e1 Guest

    Thanks for you answer! This copy trick is the most elegant solution i
    think.
     
    beza1e1, Oct 5, 2005
    #7
  8. beza1e1

    Ben Sizer Guest

    Fredrik Lundh wrote:

    > it's also mentioned in chapter 4 of the tutorial:
    >
    > http://docs.python.org/tut/node6.html#SECTION006710000000000000000
    >
    > "*Important warning*: The default value is evaluated only once. This
    > makes a difference when the default is a mutable object such as a list,
    > dictionary, or instances of most classes. "


    Perhaps it would be a good idea if Python actually raised a warning
    (SyntaxWarning?) if you use an unnamed list or dict as a default
    argument. This would doubtless help quite a few beginners. And for
    people who really do want that behaviour, working around the warning
    should involve minimal extra code, with extra clarity thrown in for
    free.

    --
    Ben Sizer
     
    Ben Sizer, Oct 5, 2005
    #8
  9. On Wed, 05 Oct 2005 03:39:30 -0700, beza1e1 wrote:

    > Coming back from a bug hunt, i am not sure what to think of this python
    > behaviour.


    [snip code]

    > The point seems to be, that lst=[] creates a class attribute (correct
    > name?), which is shared by all instances of A. So a.lst ist the same
    > object as b.lst, despite the fact, that object a is different to object
    > b.


    Not a bug, but not really a feature as such, it is a side-effect of
    the way Python works. I guess that makes it a gotcha.

    Argument defaults are set at compile time. You set the argument default to
    a mutable object, an empty list. Every time you call the function,
    you are appending to the same list.

    This is not a problem if your argument default is a string, or a number,
    or None, since these are all immutable objects that can't be changed.

    I suppose someone might be able to come up with code that deliberately
    uses this feature for good use, but in general it is something you want to
    avoid. Instead of:

    def spam(obj, L=[]):
    L.append(obj)

    do this:

    def spam(obj, L=None):
    if L is None: L = []
    L.append(obj)


    --
    Steven.
     
    Steven D'Aprano, Oct 5, 2005
    #9
  10. beza1e1

    Steve Holden Guest

    beza1e1:

    > Coming back from a bug hunt, i am not sure what to think of this python
    > behaviour. Here is a demo program:
    >
    > class A:
    > def __init__(self, lst=[]):
    > self.lst = lst
    >
    > a = A()
    > b = A()
    > b.lst.append("hallo")
    > print a.lst # output: ["hallo"]
    >
    > The point seems to be, that lst=[] creates a class attribute (correct
    > name?), which is shared by all instances of A. So a.lst ist the same
    > object as b.lst, despite the fact, that object a is different to object
    > b.
    >

    Fredrik Lundh wrote:
    > Steve Holden wrote:
    >
    >
    >>Interestingly I couldn't find this in the FAQ, though it *is* a
    >>frequently-asked question [note: my not finding it doesn't guarantee
    >>it's not there].

    >
    >
    > it's there:
    >
    > http://www.python.org/doc/faq/general.html#why-are-default-values-shared-between-objects
    >
    > (maybe "default values" should be changed to "default argument values")
    >

    I couldn't believe it wasn't, but you're right: it should be easier to
    find, and a change of wording may do that.

    regards
    Steve
    --
    Steve Holden +44 150 684 7255 +1 800 494 3119
    Holden Web LLC www.holdenweb.com
    PyCon TX 2006 www.python.org/pycon/
     
    Steve Holden, Oct 5, 2005
    #10
  11. beza1e1

    Steve Holden Guest

    Ben Sizer wrote:
    > Fredrik Lundh wrote:
    >
    >
    >>it's also mentioned in chapter 4 of the tutorial:
    >>
    >> http://docs.python.org/tut/node6.html#SECTION006710000000000000000
    >>
    >> "*Important warning*: The default value is evaluated only once. This
    >> makes a difference when the default is a mutable object such as a list,
    >> dictionary, or instances of most classes. "

    >
    >
    > Perhaps it would be a good idea if Python actually raised a warning
    > (SyntaxWarning?) if you use an unnamed list or dict as a default
    > argument. This would doubtless help quite a few beginners. And for
    > people who really do want that behaviour, working around the warning
    > should involve minimal extra code, with extra clarity thrown in for
    > free.
    >

    This would have to be extended to any mutable object. How does the
    compiler know which objects are mutable?

    This would not be a good change.

    regards
    Steve
    --
    Steve Holden +44 150 684 7255 +1 800 494 3119
    Holden Web LLC www.holdenweb.com
    PyCon TX 2006 www.python.org/pycon/
     
    Steve Holden, Oct 5, 2005
    #11
  12. Steven D'Aprano wrote:

    > I suppose someone might be able to come up with code that deliberately
    > uses this feature for good use


    argument binding is commonly used for optimization, and to give simple
    functions persistent storage (e.g. memoization caches).

    more importantly, it's the standard pydiom for passing object *values* (of
    any kind) into an inner scope:

    x = something

    def myfunc(arg, x=x):
    # myfunc needs the current value, not whatever x
    # happens to be when the function is called

    here's a typical gotcha:

    for i in range(10):
    def cb():
    print "slot", i, "fired"
    register_callback(slot=i, callback=cb)

    to make this work as expected, you have to do

    for i in range(10):
    def cb(i=i):
    print "slot", i, "fired"
    register_callback(slot=i, callback=cb)

    </F>
     
    Fredrik Lundh, Oct 6, 2005
    #12
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Shel Blauman [MSFT]

    Re: feature or bug?

    Shel Blauman [MSFT], Aug 11, 2003, in forum: ASP .Net
    Replies:
    0
    Views:
    412
    Shel Blauman [MSFT]
    Aug 11, 2003
  2. Replies:
    14
    Views:
    1,502
  3. Replies:
    6
    Views:
    1,979
  4. Meelis Lilbok

    Bug or Feature ;)

    Meelis Lilbok, Mar 30, 2006, in forum: ASP .Net
    Replies:
    4
    Views:
    430
    Juan T. Llibre
    Mar 30, 2006
  5. robic0
    Replies:
    7
    Views:
    111
    Samwyse
    Jan 24, 2006
Loading...

Share This Page