Trying to understand += better

Discussion in 'Python' started by Roy Smith, Nov 23, 2009.

  1. Roy Smith

    Roy Smith Guest

    If I've got an object foo, and I execute:

    foo.bar += baz

    exactly what happens if foo does not have a 'bar' attribute? It's
    pretty clear that foo.__getattr__('bar') gets called first, but it's a
    little murky after that. Assume for the moment that foo.__getattr__
    ('bar') returns an object x. I think the complete sequence of calls
    is:

    foo.__getattr__('bar') ==> x
    x.__add__(baz) ==> y
    foo.__setattr__('bar', y)

    but I'm not 100% sure. It would be nice if it was, because that would
    let me do some very neat magic in a system I'm working on :)

    How would things change if X defined __iadd__()?
     
    Roy Smith, Nov 23, 2009
    #1
    1. Advertising

  2. Roy Smith

    Lie Ryan Guest

    Roy Smith wrote:
    > If I've got an object foo, and I execute:
    >
    > foo.bar += baz
    >
    > exactly what happens if foo does not have a 'bar' attribute? It's
    > pretty clear that foo.__getattr__('bar') gets called first, but it's a
    > little murky after that. Assume for the moment that foo.__getattr__
    > ('bar') returns an object x. I think the complete sequence of calls
    > is:
    >
    > foo.__getattr__('bar') ==> x
    > x.__add__(baz) ==> y
    > foo.__setattr__('bar', y)
    >
    > but I'm not 100% sure. It would be nice if it was, because that would
    > let me do some very neat magic in a system I'm working on :)
    >
    > How would things change if X defined __iadd__()?



    The semantic of the in-place operator is something like:
    x += y
    becomes
    x = x.__iadd__(y)

    thus
    foo.bar += baz
    becomes
    foo.bar = foo.bar.__iadd__(baz)

    So the call sequence is,
    foo.__getattr__('bar') ==> x
    x.__iadd__(baz) ==> y
    foo.__setattr__('bar', y)

    the default definition of object.__iadd__ is something like this:
    def __iadd__(self, other):
    # this calls self.__add__ or other.__radd__ according to the
    # operator call rule, may call __coerce__ or any other magics
    # in operator calling
    return self + other
     
    Lie Ryan, Nov 23, 2009
    #2
    1. Advertising

  3. Roy Smith

    Steve Howell Guest

    On Nov 22, 7:28 pm, Lie Ryan <> wrote:
    > Roy Smith wrote:
    > > If I've got an object foo, and I execute:

    >
    > > foo.bar += baz

    >
    > > exactly what happens if foo does not have a 'bar' attribute?  It's
    > > pretty clear that foo.__getattr__('bar') gets called first, but it's a
    > > little murky after that.  Assume for the moment that foo.__getattr__
    > > ('bar') returns an object x.  I think the complete sequence of calls
    > > is:

    >
    > > foo.__getattr__('bar')  ==> x
    > > x.__add__(baz)  ==> y
    > > foo.__setattr__('bar', y)

    >
    > > but I'm not 100% sure.  It would be nice if it was, because that would
    > > let me do some very neat magic in a system I'm working on :)

    >
    > > How would things change if X defined __iadd__()?

    >
    > The semantic of the in-place operator is something like:
    > x += y
    > becomes
    > x = x.__iadd__(y)
    >
    > thus
    > foo.bar += baz
    > becomes
    > foo.bar = foo.bar.__iadd__(baz)
    >
    > So the call sequence is,
    > foo.__getattr__('bar') ==> x
    > x.__iadd__(baz) ==> y
    > foo.__setattr__('bar', y)
    >
    > the default definition of object.__iadd__ is something like this:
    > def __iadd__(self, other):
    >      # this calls self.__add__ or other.__radd__ according to the
    >      # operator call rule, may call __coerce__ or any other magics
    >      # in operator calling
    >      return self + other


    The __iadd__ method will often return self for mutable types. So, for
    example, these two statements are NOT identical where lst is a list:

    lst = lst + [3]
    lst += [3]

    The first statement is creating a whole new list; the second one
    isn't.


    http://docs.python.org/reference/datamodel.html
     
    Steve Howell, Nov 23, 2009
    #3
  4. Roy Smith

    Roy Smith Guest

    In article <4b0a01aa$>, Lie Ryan <>
    wrote:

    > The semantic of the in-place operator is something like:
    > x += y
    > becomes
    > x = x.__iadd__(y)
    >
    > thus
    > foo.bar += baz
    > becomes
    > foo.bar = foo.bar.__iadd__(baz)
    >
    > So the call sequence is,
    > foo.__getattr__('bar') ==> x
    > x.__iadd__(baz) ==> y
    > foo.__setattr__('bar', y)


    I don't get where the __setattr__() call comes from in this situation. I
    thought the whole idea of __iadd__(self, other) is that it's supposed to
    mutate self. So, why is there another assignment happening after the
    __iadd__() call?
     
    Roy Smith, Nov 23, 2009
    #4
  5. Roy Smith

    n00m Guest

    > The first statement is creating a whole new list;

    Yes but *imo* not quite exactly so.
    We can't think of 2 lists as of absolutely independent
    things.

    .... x = [[0]]
    .... id(x)
    19330632
    .... id(x[0])
    19316608
    .... z = x + [3]
    .... id(z)
    19330312
    .... id(z[0])
    19316608
    ....
    .... z[0] is x[0] # ?
    True
    .... x[0][0] = 1
    ....
    .... z[0][0]
    1
     
    n00m, Nov 23, 2009
    #5
  6. Roy Smith

    Steve Howell Guest

    On Nov 22, 9:11 pm, n00m <> wrote:
    > > The first statement is creating a whole new list;

    >
    > Yes but *imo* not quite exactly so.
    > We can't think of 2 lists as of absolutely independent
    > things.
    > [...]


    You are correct that two lists can both have the same mutable object
    as items, and if you mutate that object, both lists will see that
    mutation. Changing the innards of an item doesn't change the list, if
    you think of the list as just a collection of ids, but your point is
    well taken. It is probably easiest to understand all this with a more
    elaborate example.

    >>> mutable = {'foo': 'bar'}
    >>> list1 = [mutable, 'bla']
    >>> list2 = list1 + ['another element']
    >>> list1

    [{'foo': 'bar'}, 'bla']
    >>> list2

    [{'foo': 'bar'}, 'bla', 'another element']

    So list2 and list1 are no longer the same list, but...

    >>> mutable['foo'] = 'new value for mutable'
    >>> list1

    [{'foo': 'new value for mutable'}, 'bla']
    >>> list2

    [{'foo': 'new value for mutable'}, 'bla', 'another element']

    They do still share a common element, as shown above.

    But you can reassign the first element of list2 without affecting
    list1:

    >>> list2[0] = 'only list 2'
    >>> list1

    [{'foo': 'new value for mutable'}, 'bla']
    >>> list2

    ['only list 2', 'bla', 'another element']

    Now look at fred_list and barney_list below. Since you use +=,
    fred_list and barney_list stay tied together even when you *think* you
    are only assigning a new value to barney_list[0].

    >>> fred_list = [0]
    >>> barney_list = fred_list
    >>> barney_list += [1]
    >>> barney_list

    [0, 1]
    >>> fred_list

    [0, 1]
    >>> barney_list[0] = 'barney'
    >>> barney_list

    ['barney', 1]
    >>> fred_list

    ['barney', 1]
     
    Steve Howell, Nov 23, 2009
    #6
  7. Roy Smith

    Steve Howell Guest

    On Nov 22, 8:38 pm, Roy Smith <> wrote:
    > In article <>, Lie Ryan <>
    > wrote:
    >
    > > The semantic of the in-place operator is something like:
    > > x += y
    > > becomes
    > > x = x.__iadd__(y)

    >
    > > thus
    > > foo.bar += baz
    > > becomes
    > > foo.bar = foo.bar.__iadd__(baz)

    >
    > > So the call sequence is,
    > > foo.__getattr__('bar') ==> x
    > > x.__iadd__(baz) ==> y
    > > foo.__setattr__('bar', y)

    >
    > I don't get where the __setattr__() call comes from in this situation.  I
    > thought the whole idea of __iadd__(self, other) is that it's supposed to
    > mutate self.  So, why is there another assignment happening after the
    > __iadd__() call?


    Non-mutable types can also use += syntax.

    x = MagicalNumber(42)
    x += 5 # x gets reassigned to MagicalNumber(47)

    There is nothing that says that __iadd__ has to return self, but if
    you are designing a mutable type, you will generally do that.

    http://docs.python.org/reference/datamodel.html

    '''
    These methods are called to implement the augmented arithmetic
    assignments (+=, -=, *=, /=, //=, %=, **=, <<=, >>=, &=, ^=, |=).
    These methods should attempt to do the operation in-place (modifying
    self) and return the result (which could be, but does not have to be,
    self). If a specific method is not defined, the augmented assignment
    falls back to the normal methods. For instance, to execute the
    statement x += y, where x is an instance of a class that has an
    __iadd__() method, x.__iadd__(y) is called. If x is an instance of a
    class that does not define a __iadd__() method, x.__add__(y) and
    y.__radd__(x) are considered, as with the evaluation of x + y.
    '''
     
    Steve Howell, Nov 23, 2009
    #7
  8. Roy Smith

    Lie Ryan Guest

    Roy Smith wrote:
    > In article <4b0a01aa$>, Lie Ryan <>
    > wrote:
    >
    >> The semantic of the in-place operator is something like:
    >> x += y
    >> becomes
    >> x = x.__iadd__(y)
    >>
    >> thus
    >> foo.bar += baz
    >> becomes
    >> foo.bar = foo.bar.__iadd__(baz)
    >>
    >> So the call sequence is,
    >> foo.__getattr__('bar') ==> x
    >> x.__iadd__(baz) ==> y
    >> foo.__setattr__('bar', y)

    >
    > I don't get where the __setattr__() call comes from in this situation. I
    > thought the whole idea of __iadd__(self, other) is that it's supposed to
    > mutate self. So, why is there another assignment happening after the
    > __iadd__() call?


    The formal name of the __iop__ is "agumented assignment". The name
    doesn't talk about in-place; but it does talk about assignment. The
    semantic defines that augmented assignment operation is done in-place
    *when the left-hand side object supports it* (i.e. it does not require
    in-place mutation).

    """
    The __iadd__ hook should behave similar to __add__, returning the
    result of the operation (which *could* be `self') which is to be
    assigned to the variable `x'.
    """

    For a more complete description, see:
    http://www.python.org/dev/peps/pep-0203/
     
    Lie Ryan, Nov 23, 2009
    #8
  9. Roy Smith

    Terry Reedy Guest

    Roy Smith wrote:
    > In article <4b0a01aa$>, Lie Ryan <>
    > wrote:
    >
    >> The semantic of the in-place operator is something like:
    >> x += y
    >> becomes
    >> x = x.__iadd__(y)


    Except that the expression x is evaluated just once instead of twice.

    >> thus
    >> foo.bar += baz
    >> becomes
    >> foo.bar = foo.bar.__iadd__(baz)
    >>
    >> So the call sequence is,
    >> foo.__getattr__('bar') ==> x
    >> x.__iadd__(baz) ==> y
    >> foo.__setattr__('bar', y)

    >
    > I don't get where the __setattr__() call comes from in this situation.


    Augmented *ASSIGNMENT* is a type of assignment.

    The dis module can be used to see what CPython does.
    >>> from dis import dis
    >>> def f():

    foo.bar += baz

    >>> dis(f)

    2 0 LOAD_GLOBAL 0 (foo)
    3 DUP_TOP
    4 LOAD_ATTR 1 (bar)
    7 LOAD_GLOBAL 2 (baz)
    10 INPLACE_ADD
    11 ROT_TWO
    12 STORE_ATTR 1 (bar)
    ....
    This amounts to what Roy said, with x and y being temporary entries on
    the stack.

    Terry Jan Reedy
     
    Terry Reedy, Nov 30, 2009
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Paul K

    Trying to understand...

    Paul K, Nov 19, 2003, in forum: ASP .Net
    Replies:
    2
    Views:
    356
    Paul K
    Nov 19, 2003
  2. =?Utf-8?B?QmlsbCBCb3Jn?=

    Trying to understand ticket/cookie expiration

    =?Utf-8?B?QmlsbCBCb3Jn?=, Oct 8, 2004, in forum: ASP .Net
    Replies:
    0
    Views:
    363
    =?Utf-8?B?QmlsbCBCb3Jn?=
    Oct 8, 2004
  3. Replies:
    8
    Views:
    371
    Stephen Sprunk
    Dec 17, 2006
  4. Zouplaz
    Replies:
    3
    Views:
    131
    Brian Candler
    Oct 11, 2008
  5. Frank Millman

    Trying to understand 'import' a bit better

    Frank Millman, Mar 4, 2012, in forum: Python
    Replies:
    0
    Views:
    119
    Frank Millman
    Mar 4, 2012
Loading...

Share This Page