Operator commutativity

[Henrik Faber]

Hi there,

when I have a python class X which overloads an operator, I can use that
operator to do any operation for example with an integer

y = X() + 123

however, say I want the "+" operator to be commutative. Then

y = 123 + X()

should have the same result. However, since it does not call __add__ on
an instance of X, but on the int 123, this fails:

TypeError: unsupported operand type(s) for +: 'int' and 'X'

How can I make this commutative?

Best regards,
Henrik

 

[Arnaud Delobelle]

Hi there,

when I have a python class X which overloads an operator, I can use that
operator to do any operation for example with an integer

y = X() + 123

however, say I want the "+" operator to be commutative. Then

y = 123 + X()

should have the same result. However, since it does not call __add__ on
an instance of X, but on the int 123, this fails:

TypeError: unsupported operand type(s) for +: 'int' and 'X'

How can I make this commutative?

Overload X.__radd__() as well

HTH

 

[Paul Rudin]

How can I make this commutative?

Incidentally - this isn't really about commutativity at all - the
question is how can you define both left and right versions of add,
irrespective of whether they yield the same result.

I think __radd__ is what you're after.

 

[Henrik Faber]

Incidentally - this isn't really about commutativity at all - the
question is how can you define both left and right versions of add,
irrespective of whether they yield the same result.

Right. The operator+ in my case just happens to be commutative and I
wanted a language way to express this.

I think __radd__ is what you're after.

It is, thank you very much - I knew there was some way to get this done
nicely. Perfect!

Best regards,
Henrik

 

[Roy Smith]

Right. The operator+ in my case just happens to be commutative and I
wanted a language way to express this.


It is, thank you very much - I knew there was some way to get this done
nicely. Perfect!

__radd__() only solves the problem if the left-hand operand has no
__add__() method itself.

class C1:
def __add__(self, other):
print "C1.__add__()"
def __radd__(self, other):
print "C1.__radd__()"

class C2:
def __add__(self, other):
print "C2.__add__()"
def __radd__(self, other):
print "C2.__radd__()"

c1 = C1()
c2 = C2()

c1 + c2
c2 + c1

$ python radd.py
C1.__add__()
C2.__add__()

 

[Ethan Furman]

__radd__() only solves the problem if the left-hand operand has no
__add__() method itself.

Only true if the left-hand operand is so ill-behaved it doesn't check to
see if it makes sense to add itself to the right-hand operand. If it
doesn't know how, it should `return NotImplemented` -- Python will then
try __radd__ on the left-hand operand.

Also, if the right-hand operand is a subclass of the left-hand operand
then Python will try right-hand_operand.__radd__ first.

Now, if the left-hand operand *does* know how (or thinks it does, which
could be another matter entirely), and the right-hand operand is *not* a
subclass of the left-hand operand, then you are correct -- the
right-hand operand wil not be called.

~Ethan~

 

[Terry Reedy]

Roy Smith wrote:

Only true if the left-hand operand is so ill-behaved it doesn't check to
see if it makes sense to add itself to the right-hand operand. If it
doesn't know how, it should `return NotImplemented` -- Python will then
try __radd__ on the left-hand operand.

Also, if the right-hand operand is a subclass of the left-hand operand
then Python will try right-hand_operand.__radd__ first.

The builtin classes like int are (should be, sans bug) well-behaved.

Now, if the left-hand operand *does* know how (or thinks it does, which
could be another matter entirely), and the right-hand operand is *not* a
subclass of the left-hand operand, then you are correct -- the
right-hand operand wil not be called.

So the potential problems arise with two user classes.

 

[Westley Martínez]

Hi there,

when I have a python class X which overloads an operator, I can use that
operator to do any operation for example with an integer

y = X() + 123

however, say I want the "+" operator to be commutative. Then

y = 123 + X()

should have the same result. However, since it does not call __add__ on
an instance of X, but on the int 123, this fails:

TypeError: unsupported operand type(s) for +: 'int' and 'X'

How can I make this commutative?

Best regards,
Henri

def __radd__(self, other):
return self.__add__(self, other)

 

[Steven D'Aprano]

def __radd__(self, other):
return self.__add__(self, other)

Which, inside a class, can be simplified to:

__radd__ = __add__

 

[Roy Smith]

Which, inside a class, can be simplified to:

__radd__ = __add__

Ooh, I could see that leading to some weird diagnostics. For example:

class Foo:
def __add__(self, other):
raise Exception

__radd__ = __add__

f1 = Foo()
print 1 + f1

produces:

../add.py
Traceback (most recent call last):
File "./add.py", line 11, in <module>
print 1 + f1
File "./add.py", line 5, in __add__
raise Exception
Exception

which leaves the user wondering why __add__() was called when clearly
__radd__() should have been. The way Westley wrote it (modulo fixing
the __add__() call signature) produces:

../add.py
Traceback (most recent call last):
File "./add.py", line 11, in <module>
print 1 + f1
File "./add.py", line 8, in __radd__
return self.__add__(other)
File "./add.py", line 5, in __add__
raise Exception
Exception

which at least is a stack trace that shows that __radd__() was called.

 

[Steven D'Aprano]

Ooh, I could see that leading to some weird diagnostics.

"Weird"?

You've lived a sheltered life if you think a function being known under
two names is weird. Wait until you discover monkey-patching built-ins for
fun and profit!

 

[Westley Martínez]

Ooh, I could see that leading to some weird diagnostics. For example:

class Foo:
def __add__(self, other):
raise Exception

__radd__ = __add__

f1 = Foo()
print 1 + f1

produces:

./add.py
Traceback (most recent call last):
File "./add.py", line 11, in <module>
print 1 + f1
File "./add.py", line 5, in __add__
raise Exception
Exception

which leaves the user wondering why __add__() was called when clearly
__radd__() should have been. The way Westley wrote it (modulo fixing
the __add__() call signature) produces:

./add.py
Traceback (most recent call last):
File "./add.py", line 11, in <module>
print 1 + f1
File "./add.py", line 8, in __radd__
return self.__add__(other)
File "./add.py", line 5, in __add__
raise Exception
Exception

which at least is a stack trace that shows that __radd__() was called.

Calling __radd__ = __add__ simply creates a reference to __add__ named
__radd__, it doesn't create a new method.

 

[Peter Pearson]

Also, if the right-hand operand is a subclass of the left-hand operand
then Python will try right-hand_operand.__radd__ first.

I don't think it works that way for me:

Python 2.6.5 (r265:79063, Apr 16 2010, 13:09:56).... def __add__(self, other):
.... print "C1.__add__()"
........ def __radd__(self, other):
.... print "C2.__radd__()"
....

 

[Ethan Furman]

I don't think it works that way for me:

Python 2.6.5 (r265:79063, Apr 16 2010, 13:09:56)
... def __add__(self, other):
... print "C1.__add__()"
...
... def __radd__(self, other):
... print "C2.__radd__()"
...
C1.__add__()

Oh, it has to be a new-style class. Sorry.

~Ethan~

 

[Steven D'Aprano]

[snip]

Also, if the right-hand operand is a subclass of the left-hand operand
then Python will try right-hand_operand.__radd__ first.

I don't think it works that way for me:

Python 2.6.5 (r265:79063, Apr 16 2010, 13:09:56)

class C1():

... def __add__(self, other):
... print "C1.__add__()"
...

class C2(C1):

... def __radd__(self, other):
... print "C2.__radd__()"
...

C1() + C2()

C1.__add__()

Oh, it has to be a new-style class. Sorry.

Even so, I still don't think it quite works the way you have suggested.



class C1(object):
def __add__(self, other):
return "__add__(%s, %s)" % (self, other)
def __radd__(self, other):
return "__radd__(%s, %s)" % (self, other)

class C2(C1):
pass

'__add__(<__main__.C1 object at 0xb7f79b0c>, <__main__.C2 object at
0xb7f79b6c>)'


I've tried that in both Python 2.6 and 3.2 and get the same result.

However, consider this slight variation:

class D1(object):
def __add__(self, other):
return "__add__(%s, %s)" % (self, other)

class D2(D1):
def __radd__(self, other):
return "__radd__(%s, %s)" % (self, other)

'__radd__(<__main__.D2 object at 0xb7c2c36c>, <__main__.D1 object at
0xb7c2cb0c>)'



After playing around with various combinations of C1, C2, D1 and D2, it
seems to me that the rule is:

If the right-hand argument is a subclass of the left-hand argument, AND also
defines __radd__ directly rather than inheriting it, then its __radd__
method is called before the left-hand argument's __add__ method.


which strikes me as a strangely specific and not very useful rule. I suspect
it might be an accident of implementation rather than a deliberate feature.

 

[Chris Angelico]

If the right-hand argument is a subclass of the left-hand argument, AND also
defines __radd__ directly rather than inheriting it, then its __radd__
method is called before the left-hand argument's __add__ method.

which strikes me as a strangely specific and not very useful rule. I suspect
it might be an accident of implementation rather than a deliberate feature.

It makes sense, but in a weird way. (Maybe I understand it because I'm
half Dutch? heh) It means that a subclass can override addition for
itself - presumably, it'll define __add__ and __radd__ both - and that
the only time you'd get a false positive (where a function thinks it
can handle the addition but actually there's a better one) is when
it's a subclass. So this is probably correct behaviour, but it's a
fairly weird and esoteric rule.

ChrisA

 

[Mark Dickinson]

After playing around with various combinations of C1, C2, D1 and D2, it
seems to me that the rule is:

If the right-hand argument is a subclass of the left-hand argument, AND also
defines __radd__ directly rather than inheriting it, then its __radd__
method is called before the left-hand argument's __add__ method.

which strikes me as a strangely specific and not very useful rule. I suspect
it might be an accident of implementation rather than a deliberate feature.

I'm 99.9% sure it's deliberate rather than an accident of
implementation. See the note in the docs at:

http://docs.python.org/reference/datamodel.html#emulating-numeric-types

Support that you're subclassing int, (class MyInt(int): ...) and you
want to intercept additions of the form 3 + MyInt(4) (perhaps because
you want MyInt to be 'contagious', so that an arithmetic operation
that combines an int and a MyInt returns a MyInt). How would you
achieve this without this rule?

 

[Ethan Furman]

I'm 99.9% sure it's deliberate rather than an accident of
implementation. See the note in the docs at:

http://docs.python.org/reference/datamodel.html#emulating-numeric-types

Support that you're subclassing int, (class MyInt(int): ...) and you
want to intercept additions of the form 3 + MyInt(4) (perhaps because
you want MyInt to be 'contagious', so that an arithmetic operation
that combines an int and a MyInt returns a MyInt). How would you
achieve this without this rule?

I think Steven's objection was in not calling subclass.__radd__ when
__radd__ is inherited rather than directly overridden. Interestingly
enough, the following works as expected (at least, as I expected ...

class C1(object):
def __add__(self, other):
print "C1.__add__(%r, %r)" % (self, other)

class C2(C1):
def __radd__(self, other):
print "C2.__radd__(%r, %r)" % (self, other)

class C3(C2):
pass

C1() + C2() # --> C2.__radd__(<...C2...>, <...C1...)
C1() + C3() # --> C2.__radd__(<...C3...>, <...C1...>)
C2() + C3() # --> C1.__add__(<...C2...>, <...C3...>)


~Ethan~

 

[Ethan Furman]

After playing around with various combinations of C1, C2, D1 and D2, it
seems to me that the rule is:

If the right-hand argument is a subclass of the left-hand argument, AND also
defines __radd__ directly rather than inheriting it, then its __radd__
method is called before the left-hand argument's __add__ method.


which strikes me as a strangely specific and not very useful rule. I suspect
it might be an accident of implementation rather than a deliberate feature.

Update to rule:

If the right-hand argument is a subclass of the left-hand argument AND a
__radd__ is defined anywhere between the left-hand argument's class up
to and including the right-hand argument's class, then __radd__ is
called, otherwise the left-hand arguments __add__ is called.

And it makes perfect sense -- a + b is, after all, an __add__ function;
the only reason to call __radd__ instead is if it has been defined for a
subclass, and the only reason to define it is because it needs something
different from a + b that a doesn't know about.

Probably not a clear explanation -- maybe somebody else can dress it up
a bit.

~Ethan~