Scope rule pecularities

[Antoon Pardon]

Well to test things out I wrote the following:


class Int:

def __init__(self, v):
self.value = v

def __add__(self, term):
return Int(self.value + term.value)

def __iadd__(self, term):
self.value += term.value
return self.value

def __str__(self):
return `self.value`

a1 = Int(14)
a2 = Int(15)
b = Int(23)

print a1, a2

def foo():

a1 += b
a2.__iadd__(b)

foo()

print a1, a2


Now the a1 += b line doesn't work, it produces the following error:

UnboundLocalError: local variable 'a1' referenced before assignment.

The a2.__iadd__(b) line however works without trouble.


Now I think I understand what is causing this, but I think this
kind of thing shouldn't happen. If a += b is just syntatic sugar
for a.__iadd__(b) then the first should be acceptable where the
second is acceptable.

 

[Yermat]

[...]
a1 = Int(14)
a2 = Int(15)
b = Int(23)

print a1, a2

def foo():

a1 += b
a2.__iadd__(b)

foo()

print a1, a2


Now the a1 += b line doesn't work, it produces the following error:

UnboundLocalError: local variable 'a1' referenced before assignment.

The a2.__iadd__(b) line however works without trouble.


Now I think I understand what is causing this, but I think this
kind of thing shouldn't happen. If a += b is just syntatic sugar
for a.__iadd__(b) then the first should be acceptable where the
second is acceptable.

Here we are again !
Do you mind to try :

def foo():
global a1, a2
a1 += b
a2.__iadd__(b)

Note also that the second line were not executed so you can't know if it
were working...

 

[Antoon Pardon]

[...]
a1 = Int(14)
a2 = Int(15)
b = Int(23)

print a1, a2

def foo():

a1 += b
a2.__iadd__(b)

foo()

print a1, a2


Now the a1 += b line doesn't work, it produces the following error:

UnboundLocalError: local variable 'a1' referenced before assignment.

The a2.__iadd__(b) line however works without trouble.


Now I think I understand what is causing this, but I think this
kind of thing shouldn't happen. If a += b is just syntatic sugar
for a.__iadd__(b) then the first should be acceptable where the
second is acceptable.

Here we are again !
Do you mind to try :

def foo():
global a1, a2
a1 += b
a2.__iadd__(b)

Note also that the second line were not executed so you can't know if it
were working...

I know it worked because I commented out the first and tried again.

Beside you missed the point. I don't need solutions, I know the
work arounds to make it work. The point is that although a += b
is supposed to be syntactic sugar for a.__iadd__(b), you need
a global (and even that won't help if you have nested functions
and the a is on an intermediate level) to make the first work
but not for the second.

 

[Terry Reedy]

Now I think I understand what is causing this, but I think this
kind of thing shouldn't happen. If a += b is just syntatic sugar
for a.__iadd__(b) then the first should be acceptable where the
second is acceptable.

For immutables, a += b is better thought of as syntactic sugar for a = a+b.

tjr

 

[Guest]

Hello Antoon,

Well to test things out I wrote the following:


class Int:

def __init__(self, v):
self.value = v

def __add__(self, term):
return Int(self.value + term.value)

def __iadd__(self, term):
self.value += term.value
return self.value

This should be "return self".

def __str__(self):
return `self.value`

return "Int %i" % self.value

here will make it obvious.

a1 = Int(14)
a2 = Int(15)
b = Int(23)

print a1, a2

def foo():

a1 += b
a2.__iadd__(b)

foo()

print a1, a2


Now the a1 += b line doesn't work, it produces the following error:

UnboundLocalError: local variable 'a1' referenced before assignment.

The a2.__iadd__(b) line however works without trouble.


Now I think I understand what is causing this, but I think this
kind of thing shouldn't happen. If a += b is just syntatic sugar
for a.__iadd__(b) then the first should be acceptable where the
second is acceptable.

a1 += b

is not a shortcut for

a1.__iadd__ (b)

but for

a1 = a1.__iadd__ (b)

with the advantage of a1 evaluated only once.
So, a1 is assumed to be locally available in foo ().

Greetings,

Holger

 

[Peter Otten]

work arounds to make it work. The point is that although a += b
is supposed to be syntactic sugar for a.__iadd__(b), you need
a global (and even that won't help if you have nested functions
and the a is on an intermediate level) to make the first work
but not for the second.

a += b is not just syntactic sugar for a.__iadd__(b), it also rebinds a:
.... def __iadd__(self, other):
.... return "marker"
....
The equivalent a += b would then roughly be a = a.__iadd__(b).

Peter

 

[Donn Cave]

Beside you missed the point. I don't need solutions, I know the
work arounds to make it work. The point is that although a += b
is supposed to be syntactic sugar for a.__iadd__(b), you need
a global (and even that won't help if you have nested functions
and the a is on an intermediate level) to make the first work
but not for the second.

+= is a bit of a wart. The only value in it is that you can
execute this read-and-modify sequence with a single access in
some cases where accesses matter because of side effects.
(Single access higher up in an expression - e.g., if we have
a[0][0] += 1, you skip an a[0].) If that seems like a silly
reason to take on this abominable kludge, I'm probably not
explaining it well - wasn't my idea.

Donn Cave, (e-mail address removed)

 

[Carl Banks]

Beside you missed the point. I don't need solutions, I know the
work arounds to make it work. The point is that although a += b
is supposed to be syntactic sugar for a.__iadd__(b), you need
a global (and even that won't help if you have nested functions
and the a is on an intermediate level) to make the first work
but not for the second.

+= is a bit of a wart. The only value in it is that you can
execute this read-and-modify sequence with a single access in
some cases where accesses matter because of side effects.
(Single access higher up in an expression - e.g., if we have
a[0][0] += 1, you skip an a[0].) If that seems like a silly
reason to take on this abominable kludge, I'm probably not
explaining it well - wasn't my idea.

There is one other little benefit: it lets you avoid having to type
and maintain two identical expressions.

I kind of felt the same way you did until one day I had to write a
bunch of assignments looking like this:

cell[4*(i+1)+j+1].wallratio[k] = cell[4*(i+1)+j+1].wallratio[k] - 0.1

(It was for a rogue-like game BTW. Yes, if -= didn't exist, of course
I would set w = cell[4*(i+1)+j+1].wallratio and use w[k] = w[k] - 0.1
today. Using -= is still an improvement over that.)

One other thing: for things like Numerical Python, where efficiency
matters, using += is usually faster than +. That is, a[:] = c, a +=
b, is faster than a[:] = c+b.

A lot of nice little reasons to have it so I wouldn't call it a kludge
and certainly not abominable.

 

[Carl Banks]

a1 += b

is not a shortcut for

a1.__iadd__ (b)

but for

a1 = a1.__iadd__ (b)

with the advantage of a1 evaluated only once.

Nitpick:

a1 is only evaluated once in both of them. If it were (for example)
a1[0] instead of a1, then a1 would be evaluated once in the former,
twice in the latter.

 

[Antoon Pardon]

a += b is not just syntactic sugar for a.__iadd__(b), it also rebinds a:

... def __iadd__(self, other):
... return "marker"
...

The equivalent a += b would then roughly be a = a.__iadd__(b).

Well I would say that seems to go against the intention as stated
in the reference manual. += and associated should attempt the
operation in place. If you do an operation in place there is
no need to rebind the variable.

 

[Andrew Bennetts]

Op 2004-05-06, Peter Otten schreef <[email protected]>: [...]

The equivalent a += b would then roughly be a = a.__iadd__(b).

Well I would say that seems to go against the intention as stated
in the reference manual. += and associated should attempt the
operation in place. If you do an operation in place there is
no need to rebind the variable.

You can't do the operation in-place with immutable values such as numbers
and tuples, though. So to make "x += 1" work (where is is some integer),
rebinding is necessary.

-Andrew.

 

[Josiah Carlson]

Well I would say that seems to go against the intention as stated

in the reference manual. += and associated should attempt the
operation in place. If you do an operation in place there is
no need to rebind the variable.

Except when the /referenced object/ is immutable. Name rebinding is
necessary to reference the newly created immutable object that was
created through the +=, -=, /=, %=, *=, &=, |=, or ^= operations.

If you have some magical way of not rebinding the name, I'm sure we'd
all like to hear it.

- Josiah

 

[Andrew Bennetts]

I wrote:
[...]

You can't do the operation in-place with immutable values such as numbers
and tuples, though. So to make "x += 1" work (where is is some integer),

^^
Obviously I meant to say "x" here.

-Andrew.

 

[Antoon Pardon]

Except when the /referenced object/ is immutable. Name rebinding is
necessary to reference the newly created immutable object that was
created through the +=, -=, /=, %=, *=, &=, |=, or ^= operations.

IMO having operators that should work in place for object that
are immutable, sound contradictory. Having the following code:

a = ...
b = ...
c = a
a += b

I now expect a and c still to be the same object.

 

[Donn Cave]

A lot of nice little reasons to have it so I wouldn't call it a kludge
and certainly not abominable.

I guess this boils down to semantics, but that's flawed reasoning.
I can cite lot of nice reasons to have motor vehicles, but I won't
try to tell you they smell sweet, or that the heavy metals and gunk
washed off the road purify the soil, or that your friends who have
been killed or maimed on the road are still alive and well. Whatever
+= buys us, it's a triumph of expedience over elegance.

Donn Cave, (e-mail address removed)

 

[Josiah Carlson]

Except when the /referenced object/ is immutable. Name rebinding is

IMO having operators that should work in place for object that
are immutable, sound contradictory. Having the following code:

a = ...
b = ...
c = a
a += b

I now expect a and c still to be the same object.

If 'a' is mutable, and 'a' has the proper __iadd__ operator, then 'a'
and 'c' will be the same object when you are done. However, integers,
strings, floats,... are immutable, so 'a' and 'c' won't be the same
object in those cases (Python only rebinds objects that are specified,
due to the whole "explicit is better than implicit" Zen).

The real thing to remember is that Python isn't <insert the language you
expect Python to behave like>, and your expectations are not what
actually happens. What you think /should/ happen is not what /does/
happen, and I am quite sure it is not what /Guido thinks should happen/.

Once you wrap your mind around the implications of mutables and
immutables, and what you can/can't do with them, then perhaps the +=
semantics will seem less confusing.

- Josiah

 

[Greg Ewing]

IMO having operators that should work in place for object that
are immutable, sound contradictory. Having the following code:

But restricting += etc. to work only on mutables would
mean you wouldn't be able to do things like

x = 42
x += 1

which is a very handy thing to be able to do. The current
semantics are a pragmatic compromise which seems to work
fairly well in most situations.

I suggested once that the assignment could perhaps be
skipped if the __iadd__ method returned the original
object. That would allow a_tuple += something to work
in the case where the object is in-place modifiable.

 

[Antoon Pardon]

But restricting += etc. to work only on mutables would
mean you wouldn't be able to do things like

x = 42
x += 1

which is a very handy thing to be able to do.

But it creates confusions because the semantics
is not consistent.

IMO lacking an assignment that copies the value
of one object into another was a mistake and
which creats IMO a lot of difficulties.

If I have a function with an object as a paramter.
and this object is mutable. Now within this function
I find a second object with the value I want the
argument to have. Now I can't simply copy the
second object over the argument but I have to
copy attribute by attribute from the second object
into the argument.

 

[Antoon Pardon]

If 'a' is mutable, and 'a' has the proper __iadd__ operator, then 'a'
and 'c' will be the same object when you are done. However, integers,
strings, floats,... are immutable, so 'a' and 'c' won't be the same
object in those cases (Python only rebinds objects that are specified,
due to the whole "explicit is better than implicit" Zen).

Python does more than enought things in an implicit way.

The real thing to remember is that Python isn't <insert the language you
expect Python to behave like>,

I think it is reasonable to expect a language to behave consistently.

and your expectations are not what
actually happens. What you think /should/ happen is not what /does/
happen, and I am quite sure it is not what /Guido thinks should happen/.

So? Is what Guido thinks should happen above criticism. I think
that if what Guido thinks should happen makes the language
behave inconsistenly then

Once you wrap your mind around the implications of mutables and
immutables,

Well the more I wrap my mind around them, the less I like
them.

and what you can/can't do with them, then perhaps the +=
semantics will seem less confusing.

I would like to be able the understand what += does to
an object without the need of knowing it is mutable or
not.

 

[Duncan Booth]

IMO lacking an assignment that copies the value
of one object into another was a mistake and
which creats IMO a lot of difficulties.

If I have a function with an object as a paramter.
and this object is mutable. Now within this function
I find a second object with the value I want the
argument to have. Now I can't simply copy the
second object over the argument but I have to
copy attribute by attribute from the second object
into the argument.

Why wouldn't you just return the second value (or a copy of it) as a result
from the function? That is usually a more flexible choice since it gives
the caller the option of either replacing the original value or using the
modified value somewhere different.

Anyway, if you really need to do this then the mutable object should have
some sort of updateState method which takes the second object as a
parameter. That way the original object can have control over which
attributes get overwritten and which don't.