Python dot-equals (syntax proposal)

[Patrick Maupin]

The += family of operators really do rebind the symbol, not
modify the object.

   >>> from decimal import Decimal
   >>> d = Decimal(42)
   >>> e = Decimal(18)
   >>> orig = d
   >>> d += e
   >>> d
   Decimal("60")
   >>> e
   Decimal("18")
   >>> orig
   Decimal("42")
   >>> d is orig
   False

If your suggestion that += *modifies* the object, then orig would
now unintuitively contain 60 and "d is orig" would return True.

Well, I wrote "conceptually" (which I believe is true; it's certainly
true for me) and "sometimes actually" (which I know is true):

x = [1,2,3,4,5]
y = x
x += [6]
y

[1, 2, 3, 4, 5, 6]

x = set()
y = x
x |= set([1])
y

set([1])

SO, if you find those results "unintuitive", perhaps you should
upgrade your understanding of python. Personally, I don't find any of
the results I gave, or the results you gave, surprising, so I'm not
saying my "conceptually and sometimes actually modifies the result" is
right for *you* but it works great for me.

This doesn't preclude you from implementing a self-mutating +=
style __add__ method and returning "self", but it's usually a bad
idea unless it's dire for performance (and even then, think it
over a couple times).

Well, you should submit a bug report to fix the operation of lists and
sets for a starter.

But first, you might want to read PEP 203 -- augmented assignments. I
particularly like the section which says:

"The idea behind augmented assignment in Python is that it isn't just
an easier way to write the common practice of storing the result of a
binary operation in its left-hand operand, but also a way for the left-
hand operand in question to know that it should operate `on itself',
rather than creating a modified copy of itself."

There are a lot of sections which have a similar flavor. If (which I
doubt), the "augmented dot" is accepted, it won't necessarily have the
same meaning. x = x.foo could replace x with any other kind of
object, and I view it as a replacement, while I view x += foo as a
modification.

Regards,
Pat

 

[D'Arcy J.M. Cain]

Nope it's not. A full-time operator in python have a reflected version

Hi. Please watch the attributions. I didn't write that. I was the
one who said that '.' was not an operator.

 

[Steven D'Aprano]

Nope it's not. A full-time operator in python have a reflected version
(e.g. __radd__), which dot does not have.

What are the reflected versions of __eq__ and __ne__ (binary == and !=
operators)?

And __neg__, __pos__ and __inv__ (for the unary - + and ~ operators)?

And the three-argument form of __pow__ for power(1, 2, x)?

And Python's object system
makes it that the argument to __getattr__ is always a string even though
there might be a valid variable that corresponds to it:

That is nothing to do with the object system, it is related to the
semantics of Python syntax. a.b doesn't mean "apply the binary dot
operator to arguments a and b". It is syntactic sugar for "look for an
attribute named 'b' on object a". As such, the operands that __getattr__
receives are the object a and the *name* b (implemented as a string).

Also, the implementation of attribute lookup is quite complex, with all
sorts of special cases and optimizations.

a = MyClass()
b = MyClass()
print a . b

I've often wanted to figure out a way to (ab)use python's dot operator
for function composition (i.e. f.g(x) ==> f(g(x)) ). There's no way to
do it, not without being way too hackish.

That's a good example of where the difference does make a difference.

 

[Steven D'Aprano]

The += family of operators really do rebind the symbol, not modify the
object.

They potentially do both, depending on the object, even for built-ins.

[...]

I'm not sure why you took the trouble to import Decimal for this example,
when you could have shown the same thing with built-ins int or float. All
three types are immutable.

A counter example with a mutable type:

a = []
b = a
a += [2]
a [2]
b

[2]

thus demonstrating that __iadd__ modifies in place as well as rebinds for
at least one mutable type.

This doesn't preclude you from implementing a self-mutating += style
__add__ method and returning "self", but it's usually a bad idea

Obviously the Python dev team don't agree with that

Just to prove that += for lists is not an accident:

a = set()
b = a
a |= set([1])
a set([1])
b

set([1])

 

[Chris Rebert]

This doesn't preclude you from implementing a self-mutating += style
__add__ method and returning "self", but it's usually a bad idea

Obviously the Python dev team don't agree with that

Just to prove that += for lists is not an accident:

a = set()
b = a
a |= set([1])
a set([1])
b

set([1])

In both cases, __iOP__ operator methods are being used, not vanilla
__OP__ methods, so neither of your examples are relevant to Mr.
Chase's point.

Cheers,
Chris

 

[Steven D'Aprano]

Obviously the Python dev team don't agree with that

Just to prove that += for lists is not an accident:

[...]
In both cases, __iOP__ operator methods are being used, not vanilla
__OP__ methods, so neither of your examples are relevant to Mr. Chase's
point.

I'm sorry, I read Tim's reference to __add__ (instead of __iadd__) as a
typo. Having __add__ mutate self would be a strange thing to do.

 

[Patrick Maupin]

In both cases, __iOP__ operator methods are being used, not vanilla
__OP__ methods, so neither of your examples are relevant to Mr.
Chase's point.

Well, Tim's main assertion was: "The += family of operators really do
rebind the symbol, not modify the object."

So, using __iadd__ to disprove this blanket assertion is certainly
relevant.

Regards,
Pat

 

[Aahz]

They potentially do both, depending on the object, even for built-ins.

No, they always rebind; sometimes they modify the object and sometimes
they rebind the original target to the same object.

 

[Alf P. Steinbach]

No, they always rebind; sometimes they modify the object

If they always rebind and sometimes modify object then they "potentially do
both", and so the "No" at the start of the sentence contradicts this later part.

and sometimes
they rebind the original target to the same object.

At the Python level that seems to be an undetectable null-operation. Granted one
could see something going on in a machine code or byte code debugger. But making
that distinction (doing nothing versus self-assignment) at the Python level
seems, to me, to be meaningless.


Cheers,

- Alf

 

[Chris Rebert]

If they always rebind and sometimes modify object then they "potentially do
both", and so the "No" at the start of the sentence contradicts this later
part.



At the Python level that seems to be an undetectable null-operation. Granted
one could see something going on in a machine code or byte code debugger.
But making that distinction (doing nothing versus self-assignment) at the
Python level seems, to me, to be meaningless.

There are some circumstances where the subtle distinction matters.
Consider x.y += z where x is an instance of a class that overrides
__setattr__ and __getattribute__ and x.y results in a mutable object.
Not doing the assignment can result in a noticeable difference in
behavior since __setattr__ won't get called. Yes, this is a slightly
obscure case, but it does come up.

Cheers,
Chris

 

[Lie Ryan]

That is nothing to do with the object system, it is related to the
semantics of Python syntax. a.b doesn't mean "apply the binary dot
operator to arguments a and b". It is syntactic sugar for "look for an
attribute named 'b' on object a". As such, the operands that __getattr__
receives are the object a and the *name* b (implemented as a string).

You just described *exactly* the reason why dot is not, and cannot be an
operator.

 

[Lie Ryan]

What are the reflected versions of __eq__ and __ne__ (binary == and !=
operators)?

Python do not have them now, but it does make sense if python have
them[1]. OTOH, given current python's language semantic, __rgetattr__
doesn't make any sense; adding __rgetattr__ would require a quite
fundamental change to the language's semantic, primarily how attribute
resolution works.

[1] though they'd probably be dismissed as bad idea since equality and
inequality are supposed to be a symmetric relation; reflected
(in)equality makes it way too easy to break that premise

And __neg__, __pos__ and __inv__ (for the unary - + and ~ operators)?

And the three-argument form of __pow__ for power(1, 2, x)?

I know you're a famed nitpicker, but don't be silly, reflected operator,
by definition, only makes sense for binary operator.

 

[Terry Reedy]

On 02.05.2010 06:06, * Aahz:

At the Python level that seems to be an undetectable null-operation.

If you try t=(1,2,3); t[1]+=3, if very much matters that a rebind occurs.

Granted one could see something going on in a machine code or byte code
debugger. But making that distinction (doing nothing versus
self-assignment) at the Python level seems, to me, to be meaningless.

Please do not confuse things. Augmented *assignment* must be understood
as assignment. Failure to do so leads (and has lead) newbies into
confusion, and puzzled posts on this list.

Terry Jan Reedy

 

[Steven D'Aprano]

You just described *exactly* the reason why dot is not, and cannot be an
operator.

This would be relevant if I said it was an operator. I did not. I said it
was a de facto operator that behaves similarly enough to operators as to
make it reasonable to talk about "dot operator". I still stand by that.

That doesn't imply that Python's implementation of a.b has to be
identical in every last detail to Python's implementation of a+b, because
it clearly isn't. But there are sufficient similarities to justify saying
that it duck-types as an operator. This isn't meant to be a vigorous
statement of fact in the same manner than "CPython implements lists as
arrays of pointers" is a statement of fact. It's meant to be a hand-wavy
"dot act kinda-sorta like an operator" manner.

I'm sorry if I failed to make that clear enough. I thought that
explicitly stating that it wasn't a real operator would be sufficient.

 

[Steven D'Aprano]

What are the reflected versions of __eq__ and __ne__ (binary == and !=
operators)?

Python do not have them now, but it does make sense if python have
them[1]. OTOH, given current python's language semantic, __rgetattr__
doesn't make any sense; adding __rgetattr__ would require a quite
fundamental change to the language's semantic, primarily how attribute
resolution works.

[1] though they'd probably be dismissed as bad idea since equality and
inequality are supposed to be a symmetric relation; reflected
(in)equality makes it way too easy to break that premise

And __neg__, __pos__ and __inv__ (for the unary - + and ~ operators)?

And the three-argument form of __pow__ for power(1, 2, x)?

I know you're a famed nitpicker, but don't be silly, reflected operator,
by definition, only makes sense for binary operator.

Binary operators aren't the only kind of operator, and you claimed that:

"A full-time operator in python have a reflected version".

But there are full-time operators that don't have reflected versions, so
your claim is just *wrong*. It would still be wrong even if you had
limited yourself to binary operators.

I have agreed with you that there are useful things people might want to
do (e.g. function composition) that you can't do because the "dot
operator" isn't a *real* operator with exactly the same semantics as
"plus operator", "multiply operator" and friends. I think we're in
violent agreement, and merely disagreeing over semantics. There's no need
to continue arguing against a position I haven't actually taken

 

[Steven D'Aprano]

On 02.05.2010 06:06, * Aahz:

At the Python level that seems to be an undetectable null-operation.

If you try t=(1,2,3); t[1]+=3, if very much matters that a rebind
occurs.

Granted one could see something going on in a machine code or byte code
debugger. But making that distinction (doing nothing versus
self-assignment) at the Python level seems, to me, to be meaningless.

Please do not confuse things. Augmented *assignment* must be understood
as assignment. Failure to do so leads (and has lead) newbies into
confusion, and puzzled posts on this list.

I think that if you think *only* about Python's standard namespaces, self-
assignment is more or less a no-op. I can't think of any way that x = x
could do anything other than use CPU cycles, *if* you limit yourself to
the standard global or function local namespaces.

But if you think about custom namespace types, and item assignment (e.g.
the example you gave with a tuple), the situation becomes different.
Here's a nice example, using Python 3.1:
.... x = 1
.... x = x
....
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in B
File "<stdin>", line 4, in __setitem__
TypeError: can't rebind constants

Thus proving that self-assignment is not necessarily a no-op. How did I
make that work? It takes a custom dict and a bit of metaclass magic:


class ConstantNamespace(dict):
def __setitem__(self, key, value):
if key in self:
raise TypeError("can't rebind constants")
super(ConstantNamespace, self).__setitem__(key, value)

class WriteOnceClass(type):
@classmethod
def __prepare__(metacls, name, bases):
return ConstantNamespace()
def __new__(cls, name, bases, classdict):
return type.__new__(cls, name, bases, classdict)

class A(metaclass=WriteOnceClass):
pass

 

[Albert van der Horst]

At least a few times a day I wish python had the following shortcut
syntax:

vbl.=func(args)

this would be equivalent to

vbl = vbl.func(args)

example:

foo = "Hello world"
foo.=split(" ")
print foo
# ['Hello', 'world']

and I guess you could generalize this to

vbl.=[some text]
#
vbl = vbl.[some text]

e.g.

temp.=children[0]
# temp = temp.children[0]

thoughts?

Useless if you use meaningful names for your variables & attributes.

It may happen that one object attribute refer to an object of the same
type, but it is quite rare that both can share the same name anyway.

Possible use cases:

1/
car = Car()
car = car.wheel # ???

2/
wheel = Car() # ???
wheel = wheel.wheel # ???

3/
currentCar = Car()
currentCar = currentCar.nextCar

The syntax you prose will be applicable on very little assignements (use
case 3). I'm not sure it's worth it.

Note how related it is to the requirement to have a _radd_ operator.

It amounts to the argument that
a op= b
requires that a and b have somewhat "similar" "type", or that
the "type" of a doesn't really change as a result from the operation.

This is IMHO an argument against the .= pseudo-operator.

JM

Groetjes Albert

 

[Alf P. Steinbach]

* Terry Reedy:

* Alf P. Steinbach:

* Aahz:

At the Python level that seems to be an undetectable null-operation.

If you try t=(1,2,3); t[1]+=3, if very much matters that a rebind occurs.
Testing:

>>> t = ([], [], [])
>>> t ([], [], [])
>>> t[0] += ["blah"]

Traceback (most recent call last):

>>> t (['blah'], [], [])
>>> _

</test>

Yep, it matters.

Is this change-but-raise-exception a bug?

I seem to have a knack for running into bugs.

Please do not confuse things. Augmented *assignment* must be understood
as assignment. Failure to do so leads (and has lead) newbies into
confusion, and puzzled posts on this list.

OK.

But I think it would be less confusing, less breaking of expectations, if, for
the example above, += reduced to the functionality of extend(), with no x.


Cheers, & thanks,

- Alf

 

[Steven D'Aprano]

* Terry Reedy:

* Alf P. Steinbach:

* Aahz:

and sometimes
they rebind the original target to the same object.

At the Python level that seems to be an undetectable null-operation.

If you try t=(1,2,3); t[1]+=3, if very much matters that a rebind
occurs.
Testing:

t = ([], [], [])
t ([], [], [])
t[0] += ["blah"]

Traceback (most recent call last):

t (['blah'], [], [])
_

</test>

Yep, it matters.

Is this change-but-raise-exception a bug?

I seem to have a knack for running into bugs.

No, I don't believe so -- I believe that it is behaving exactly as
advertised. But it is absolutely a gotcha.

Consider:

.... def __init__(self, value=0):
.... self.value = value
.... def __add__(self, other):
.... self.value = self.value + other
.... return self
.... def __str__(self):
.... return "%s" % self.value
.... __repr__ = __str__
....

x = K(42)
x + 5 47
t = (None, x)
t (None, 47)

t[1] + 3 50
t (None, 50)
t[1] += 1

Traceback (most recent call last):
(None, 51)


Unintuitive, yes. Possibly a bad design, maybe. Surprising, absolutely.
But not a bug, as it's working exactly as promised. += is conceptually
two steps: perform an addition, and perform an assignment afterward. That
addition is sometimes performed in-place, but regardless of whether it is
or not, the assignment is always attempted.

 

[Peter Otten]

t = ([], [], [])
t ([], [], [])
t[0] += ["blah"]

Traceback (most recent call last):

t (['blah'], [], [])
_

</test>

Yep, it matters.

Is this change-but-raise-exception a bug?

No.

a[0] += b

translates to

a.__setitem__(0, a.__getitem__(0).__iadd__(b))

assuming a[0] has an __iadd__() method. It should be obvious that only the
the last operation, the outer a.__setitem__(...), will fail here.
A possible fix might be a changed order of evaluation:

_internal_set = a.__setitem__
_internal_set(0, a.__getitem__(0).__iadd__(b))

I don't know if there are arguments against this other than increased
compiler complexity.

Or one could argue that

a += b

should have been implemented as

a = a + b

or

a = a.__add__(b)

which is currently used as the fallback when there is no __iadd__() method
and which gives a more intuitive behaviour at the cost of a greater
overhead. But it's a little late for that discussion, for that language.

Peter