Static variable vs Class variable

[paul.melis]

It still is true.

a += b

rebinds a. Period. Which is the _essential_ thing in my post, because
this rebinding semantics are what confused the OP.

Doesn't this depend on wether "a" supports __iadd__ or not? Section
3.4.7 of the docs say

"""
If a specific method is not defined, the augmented operation falls
back to the normal methods. For instance, to evaluate the expression x
+=y, where x is an instance of a class that has an __iadd__() method,
x.__iadd__(y) is called. If x is an instance of a class that does not
define a __iadd__() method, x.__add__(y) and y.__radd__(x) are
considered, as with the evaluation of x+y.
"""

So if a.__iadd__ exists, a += b is executed as a.__iadd__(b), in which
case there's no reason to rebind a.

However, this confuses the heck out of me:
.... l = []
....

class B(A): pass ....
B.__dict__ {'__module__': '__main__', '__doc__': None}
B.l []
B.l.append('1')
B.l ['1']
B.__dict__ {'__module__': '__main__', '__doc__': None}
B.l.__iadd__('2') ['1', '2']
B.l ['1', '2']
B.__dict__ {'__module__': '__main__', '__doc__': None}
B.l += '3'
B.__dict__

{'__module__': '__main__', '__doc__': None, 'l': ['1', '2', '3']}

Why is B.l set for the += case only? B.l.__iadd__ obviously exists.

Paul

 

[Marc 'BlackJack' Rintsch]

Doesn't this depend on wether "a" supports __iadd__ or not?

No. As shown several times in this thread already.

Section 3.4.7 of the docs say

"""
If a specific method is not defined, the augmented operation falls
back to the normal methods. For instance, to evaluate the expression x
+=y, where x is an instance of a class that has an __iadd__() method,
x.__iadd__(y) is called. If x is an instance of a class that does not
define a __iadd__() method, x.__add__(y) and y.__radd__(x) are
considered, as with the evaluation of x+y.
"""

So if a.__iadd__ exists, a += b is executed as a.__iadd__(b), in which
case there's no reason to rebind a.

`__iadd__` *may* doing the addition in place and return `self` but it is
also allowed to return a different object. So there is always a rebinding.

However, this confuses the heck out of me:
... l = []
...

class B(A): pass ...
B.__dict__ {'__module__': '__main__', '__doc__': None}
B.l []
B.l.append('1')
B.l ['1']
B.__dict__ {'__module__': '__main__', '__doc__': None}
B.l.__iadd__('2')

['1', '2']

Here you see that the method actually returns an object!

B.l ['1', '2']
B.__dict__ {'__module__': '__main__', '__doc__': None}
B.l += '3'
B.__dict__

{'__module__': '__main__', '__doc__': None, 'l': ['1', '2', '3']}

Why is B.l set for the += case only? B.l.__iadd__ obviously exists.

Because there is always a rebinding involved.

Ciao,
Marc 'BlackJack' Rintsch

 

[Duncan Booth]

On Oct 10, 8:23 am, "Diez B. Roggisch" <[email protected]> wrote:

Doesn't this depend on wether "a" supports __iadd__ or not? Section
3.4.7 of the docs say

"""
If a specific method is not defined, the augmented operation falls
back to the normal methods. For instance, to evaluate the expression x
+=y, where x is an instance of a class that has an __iadd__() method,
x.__iadd__(y) is called. If x is an instance of a class that does not
define a __iadd__() method, x.__add__(y) and y.__radd__(x) are
considered, as with the evaluation of x+y.
"""

So if a.__iadd__ exists, a += b is executed as a.__iadd__(b), in which
case there's no reason to rebind a.

You misunderstand the documentation, what you quoted doesn't say that
the assignment is suppressed. If a.__iadd__ exists, a += b is executed
as a = a.__iadd__(b)

The options for a+=b are:

a = a.__iadd__(b)
a = a.__add__(b)
a = b.__radd__(a)

but the assignment always happens, it is only what gets executed for the
right hand side which varies.

 

[paul.melis]

You misunderstand the documentation, what you quoted doesn't say that
the assignment is suppressed. If a.__iadd__ exists, a += b is executed
as a = a.__iadd__(b)

The options for a+=b are:

a = a.__iadd__(b)
a = a.__add__(b)
a = b.__radd__(a)

but the assignment always happens, it is only what gets executed for the
right hand side which varies.

Curious, do you have the relevant section in the docs that describes
this behaviour?

Paul

 

[Duncan Booth]

Curious, do you have the relevant section in the docs that describes
this behaviour?

Yes, but mostly by implication. In section 3.4.7 of the docs, the sentence
before the one you quoted says:

These methods should attempt to do the operation in-place (modifying
self) and return the result (which could be, but does not have to be,
self).

The 'does not have to be self' tells you that the result of __iadd__ is
used, i.e there is still an assignment going on.

Just read all of that paragraph carefully. It says that if there is no
__iadd__ method it considers calling __add__/__radd__. Nowhere does it say
that it handles the result of calling the methods differently.

 

[paul.melis]

Yes, but mostly by implication. In section 3.4.7 of the docs, the sentence
before the one you quoted says:

These methods should attempt to do the operation in-place (modifying
self) and return the result (which could be, but does not have to be,
self).

The 'does not have to be self' tells you that the result of __iadd__ is
used, i.e there is still an assignment going on.

Just read all of that paragraph carefully. It says that if there is no
__iadd__ method it considers calling __add__/__radd__. Nowhere does it say
that it handles the result of calling the methods differently.

Right, the paragraph is actually pretty clear after a second reading.
I find it surprising nonetheless, as it's easy to forget to return a
result when you're implementing a method that does an in-place
operation, like __iadd__:
.... def __init__(self, v):
.... self.v = v
.... def __iadd__(self, other):
.... self.v += other
....True


Paul

 

[Hrvoje Niksic]

Right, the paragraph is actually pretty clear after a second
reading. I find it surprising nonetheless, as it's easy to forget
to return a result when you're implementing a method that does an
in-place operation, like __iadd__:

I've recently been bitten by that, and I don't understand the
reasoning behind __iadd__'s design. I mean, what is the point of an
*in-place* add operation (and others) if it doesn't always work
in-place?

 

[Duncan Booth]

I've recently been bitten by that, and I don't understand the
reasoning behind __iadd__'s design. I mean, what is the point of an
*in-place* add operation (and others) if it doesn't always work
in-place?

A very common use case is using it to increment a number:

x += 1

If += always had to work inplace then this would throw an exception: an
inplace addition would be meaningless for Python numbers.

 

[Hrvoje Niksic]

I've recently been bitten by [rebinding the var to what __iadd__
returns], and I don't understand the reasoning behind __iadd__'s
design. I mean, what is the point of an *in-place* add operation
(and others) if it doesn't always work in-place?

A very common use case is using it to increment a number:

I'm aware of that; but remember that there's still __add__. It would
be sufficient for numbers not to implement __iadd__. And, in fact,
they already don't:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'int' object has no attribute '__iadd__'

The current implementation of += uses __add__ for addition and
__iadd__ for addition that may or may not be in-place. I'd like to
know the rationale for that design.

 

[Marc 'BlackJack' Rintsch]

I've recently been bitten by [rebinding the var to what __iadd__
returns], and I don't understand the reasoning behind __iadd__'s
design. I mean, what is the point of an *in-place* add operation
(and others) if it doesn't always work in-place?

A very common use case is using it to increment a number:

I'm aware of that; but remember that there's still __add__. It would
be sufficient for numbers not to implement __iadd__. And, in fact,
they already don't:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'int' object has no attribute '__iadd__'

The current implementation of += uses __add__ for addition and
__iadd__ for addition that may or may not be in-place. I'd like to
know the rationale for that design.

Simply not to introduce special cases I guess. If you write ``x.a += b``
then `x.a` will be rebound whether an `a.__iadd__()` exists or not.
Otherwise one would get interesting subtle differences with properties for
example. If `x.a` is a property that checks if the value satisfies some
constraints ``x.a += b`` would trigger the set method only if there is no
`__iadd__()` involved if there's no rebinding.

Ciao,
Marc 'BlackJack' Rintsch

 

[Steven D'Aprano]

The current implementation of += uses __add__ for addition and __iadd__
for addition that may or may not be in-place. I'd like to know the
rationale for that design.

Everything you need is in the PEP:

http://www.python.org/dev/peps/pep-0203/

 

[Duncan Booth]

The current implementation of += uses __add__ for addition and
__iadd__ for addition that may or may not be in-place. I'd like to
know the rationale for that design.

Apart from the obvious short answer of being consistent (so you don't
have to guess whether or not a+=b is going to do an assignment), I think
the decision was partly to keep the implementation clean.

Right now an inplace operation follows one of three patterns:

load a value LOAD_FAST or LOAD_GLOBAL
do the inplace operation (INPLACE_ADD etc)
store the result (STORE_FAST or STORE_GLOBAL)

or:

compute an expression
DUP_TOP
LOAD_ATTR
do the inplace operation
ROT_TWO
STORE_ATTR

or:

compute two expressions
DUP_TOPX 2
BINARY_SUBSCR
do the inplace operation
ROT_THREE
STORE_SUBSCR

I'm not sure I've got all the patterns here, but we have at least three
different cases with 0, 1, or 2 objects on the stack and at least 4
different store opcodes.

If the inplace operation wasn't always going to store the result, then
either we would need 4 times as many INPLACE_XXX opcodes, or it would
have to do something messy to pop the right number of items off the
stack and skip the following 1 or 2 instructions.

I see from PEP203 that ROT_FOUR was added as part of the implementation,
so I guess I must have missed at least one other bytecode pattern for
augmented assignment (which turns out to be slice assignment with a
stride).

 

[Duncan Booth]

Simply not to introduce special cases I guess. If you write ``x.a +=
b`` then `x.a` will be rebound whether an `a.__iadd__()` exists or
not. Otherwise one would get interesting subtle differences with
properties for example. If `x.a` is a property that checks if the
value satisfies some constraints ``x.a += b`` would trigger the set
method only if there is no `__iadd__()` involved if there's no
rebinding.

Unfortunately that doesn't work very well. If the validation rejects the
new value the original is still modified:
def setx(self, value):
if len(value)>2:
raise ValueError
self._x = value
def getx(self):
return self._x
x = property(getx, setx)

o = C()
o.x = []
o.x += ['a']
o.x += ['b']
o.x += ['c']

Traceback (most recent call last):
File "<pyshell#27>", line 1, in <module>
o.x += ['c']
File "<pyshell#22>", line 4, in setx
raise ValueError
ValueError

o.x ['a', 'b', 'c']

 

[Paul Melis]

Simply not to introduce special cases I guess. If you write ``x.a +=
b`` then `x.a` will be rebound whether an `a.__iadd__()` exists or
not. Otherwise one would get interesting subtle differences with
properties for example. If `x.a` is a property that checks if the
value satisfies some constraints ``x.a += b`` would trigger the set
method only if there is no `__iadd__()` involved if there's no
rebinding.

Unfortunately that doesn't work very well. If the validation rejects the
new value the original is still modified:

def setx(self, value):
if len(value)>2:
raise ValueError
self._x = value
def getx(self):
return self._x
x = property(getx, setx)

o = C()
o.x = []
o.x += ['a']
o.x += ['b']
o.x += ['c']

Traceback (most recent call last):
File "<pyshell#27>", line 1, in <module>
o.x += ['c']
File "<pyshell#22>", line 4, in setx
raise ValueError
ValueError
['a', 'b', 'c']

Now that's really interesting. I added a print "before" and print
"after" statement just before and after the self._x = value and these
*do not get called* after the exception is raised when the third
element is added.

 

[Marc 'BlackJack' Rintsch]

class C(object):

def setx(self, value):
if len(value)>2:
raise ValueError
self._x = value
def getx(self):
return self._x
x = property(getx, setx)

o = C()
o.x = []
o.x += ['a']
o.x += ['b']
o.x += ['c']

Traceback (most recent call last):
File "<pyshell#27>", line 1, in <module>
o.x += ['c']
File "<pyshell#22>", line 4, in setx
raise ValueError
ValueError

o.x

['a', 'b', 'c']

Now that's really interesting. I added a print "before" and print
"after" statement just before and after the self._x = value and these
*do not get called* after the exception is raised when the third
element is added.

Well, of course not. Did you really expect that!? Why?

Ciao,
Marc 'BlackJack' Rintsch

 

[Paul Melis]

class C(object):
def setx(self, value):
if len(value)>2:
raise ValueError
self._x = value
def getx(self):
return self._x
x = property(getx, setx)
o = C()
o.x = []
o.x += ['a']
o.x += ['b']
o.x += ['c']
Traceback (most recent call last):
File "<pyshell#27>", line 1, in <module>
o.x += ['c']
File "<pyshell#22>", line 4, in setx
raise ValueError
ValueError
o.x
['a', 'b', 'c']

Now that's really interesting. I added a print "before" and print
"after" statement just before and after the self._x = value and these
*do not get called* after the exception is raised when the third
element is added.

Well, of course not. Did you really expect that!? Why?

Because o.x *is* updated, i.e. the net result of self._x = value is
executed.

Paul

 

[Paul Melis]

class C(object):
def setx(self, value):
if len(value)>2:
raise ValueError
self._x = value
def getx(self):
return self._x
x = property(getx, setx)
o = C()
o.x = []
o.x += ['a']
o.x += ['b']
o.x += ['c']
Traceback (most recent call last):
File "<pyshell#27>", line 1, in <module>
o.x += ['c']
File "<pyshell#22>", line 4, in setx
raise ValueError
ValueError
o.x
['a', 'b', 'c']
Now that's really interesting. I added a print "before" and print
"after" statement just before and after the self._x = value and these
*do not get called* after the exception is raised when the third
element is added.

Well, of course not. Did you really expect that!? Why?

Because o.x *is* updated, i.e. the net result of self._x = value is
executed.

Argh, the getter is of course used to append the third element, after
which the rebinding triggers the exception. Got it now...

Paul

 

[Hrvoje Niksic]

Simply not to introduce special cases I guess. If you write ``x.a
+= b`` then `x.a` will be rebound whether an `a.__iadd__()` exists
or not. Otherwise one would get interesting subtle differences with
properties for example. If `x.a` is a property that checks if the
value satisfies some constraints ``x.a += b`` would trigger the set
method only if there is no `__iadd__()` involved if there's no
rebinding.

Note that such constraint is easily invalidated simply with:

foo = x.a
foo += b

 

[Hrvoje Niksic]

Apart from the obvious short answer of being consistent (so you don't
have to guess whether or not a+=b is going to do an assignment), I think
the decision was partly to keep the implementation clean.

The implementation, by necessity, supports a lot of historical cruft,
so keeping it clean was not a priority. (This is not a criticism, I
like how they kept backward compatibility.)

Right now an inplace operation follows one of three patterns:

[...]

Thanks for pointing it out; I didn't realize just how much work went
into the new assignment opcodes. Given the complexity, it's a wonder
they're there at all!

 

[bambam]

Everything you need is in the PEP:

http://www.python.org/dev/peps/pep-0203/

Which illustrates that the proposal, while simplified for implementation,
is not exactly what was desired*

""" is both more readable and less error prone, because it is
instantly obvious to the reader that it is <x> that is being
changed, and not <x> that is being replaced
"""

As we see from this thread, it is not instantly obvious to the reader;
the meaning of "changed, not replaced" is ambiguous.

[david]

* That is, unless ambiguity was the desideratum