Class Variable Access and Assignment

[Eric Nieuwland]

It is really simple. When you say b.a then the instance variable 'a'
is looked up first. If it does not exist then a class variable lookup
is done.

This mixing of class and instance variable might be the cause of
confusion...

I think of it as follows:
1 When the class statement ends a class object is created which is
filled by all the statements inside the class statement
This means all variables and functions (methods) are created according
to the description.
NOTE This happens just once.
2 When an instance of the class is created, what effectively happens is
that a shallow copy of the class object is made.
Simple values and object references are copied.

This explains:
- why methods and complex objects (e.g. lists) are shared among
instances of a class and the class itself
- simple values are not shared

--eric

 

[venk]

Again (blink) quoting from the docs "
For targets which are attribute references, the initial value is
retrieved with a getattr() and the result is assigned with a setattr().
Notice that the two methods do not necessarily refer to the same
variable. When getattr() refers to a class variable, setattr() still
writes to an instance variable. For example:

class A:
x = 3 # class variable
a = A()
a.x += 1 # writes a.x as 4 leaving A.x as 3
"

I felt a wee bit clear after going thru the doc... attribute
referencing is not the same as searching the variable in the enclosing
scopes.... But, i still feel the inconsistency...

 

[Antoon Pardon]

hey,
did u read my reply fully? i too feel that this matter of raising
unbound local error in one case and not raising it in the other must be
analysed...

Yes, it seems I didn't respond to your satisfaction, but since you
don't provide details I can't clarify.

quoting from the documentation
"If a name binding operation occurs anywhere within a code block, all
uses of the name within the block are treated as references to the
current block. This can lead to errors when a name is used within a
block before it is bound. This rule is subtle. Python lacks
declarations and allows name binding operations to occur anywhere
within a code block. The local variables of a code block can be
determined by scanning the entire text of the block for name binding
operations."

Well I wonder. Would the following code be considered a name binding
operation:

b.a = 5

 

[Stefan Arentz]

....

Fine, we have the code:

b.a += 2

We found the class variable, because there is no instance variable,
then why is the class variable not incremented by two now?

Because it really is executed as:

b.a = b.a + 2

1. get 't'b.a and store it in a temporary 't' (found the instance)
2. add 2 to 't'
3. store 't' in 'b.a'

The last operation stores it into an instance variable.

So? Python being a dynamic language doesn't prevent the following to fail:

... a += 2
...
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "<stdin>", line 2, in f
UnboundLocalError: local variable 'a' referenced before assignment

See the 'global' keyword.

s.

 

[Stefan Arentz]

...


Because it really is executed as:

b.a = b.a + 2

1. get 't'b.a and store it in a temporary 't' (found the instance)

Oops.

1. get b.a and store it in a temporary 't' (found the class variable 'a')

S.

 

[Magnus Lycka]

No matter wat the OO model is, I don't think the following code
exhibits sane behaviour:

class A:
a = 1

b = A()
b.a += 2
print b.a
print A.a

Which results in

3
1

.... a = [1]
....

>>> b=C()
>>> b.a += [2]
>>> b.a [1, 2]
>>> C.a

[1, 2]

I can understand that Guido was a bit reluctant to introduce
+= etc into Python, and it's important to understand that they
typically behave differently for immutable and mutable objects.

 

[Steven D'Aprano]

I don't suppose you'd care to enlighten us on what you'd regard as the
superior outcome?

class A:
a = []
b = A()
b.append(3)
print b.a
print a.a

Compare and contrast.

I take it then that you believe that ints like 1 should be mutable like
lists? Because that is what the suggested behaviour implies.

Ah, what a grand thing that would be! We could say:

0 += 1
1 += 2
7 -= 1

and then have 0 + 1 == 7. Think of the obfuscated code we could write!

 

[Antoon Pardon]

Again (blink) quoting from the docs "
For targets which are attribute references, the initial value is
retrieved with a getattr() and the result is assigned with a setattr().
Notice that the two methods do not necessarily refer to the same
variable. When getattr() refers to a class variable, setattr() still
writes to an instance variable. For example:

class A:
x = 3 # class variable
a = A()
a.x += 1 # writes a.x as 4 leaving A.x as 3
"

I felt a wee bit clear after going thru the doc... attribute
referencing is not the same as searching the variable in the enclosing
scopes.... But, i still feel the inconsistency...

The documentation is IMO no help in arguing whether the behaviour is
sane or not. Unsane documented behaviour is still unsane.

So yes the documentation explains why such a result can happen,
but that doesn't change the fact that in this case we have
only one a.x on that line and the processing behind the scenes
refers to an x in two different name spaces.

I think that is unsane behaviour and giving an explanation on
why this particular behaviour arises from the current implemenation
doesn't change that.

 

[Magnus Lycka]

There is no instance variable at that point. How can it add 2, to
something that doesn't exist at the moment.

Because 'a += 1' is only a shorthand for 'a = a + 1' if a is an
immutable object? Anyway, the behaviour is well documented.

http://docs.python.org/ref/augassign.html says:

An augmented assignment expression like x += 1 can be rewritten as x = x
+ 1 to achieve a similar, but not exactly equal effect. In the augmented
version, x is only evaluated once. Also, when possible, the actual
operation is performed in-place, meaning that rather than creating a new
object and assigning that to the target, the old object is modified instead.

....

For targets which are attribute references, the initial value is
retrieved with a getattr() and the result is assigned with a setattr().
Notice that the two methods do not necessarily refer to the same
variable. When getattr() refers to a class variable, setattr() still
writes to an instance variable. For example:

class A:
x = 3 # class variable
a = A()
a.x += 1 # writes a.x as 4 leaving A.x as 3

 

[Steven D'Aprano]

Yes it does. If the b.a refers to the instance variable, then an
AttributeError should be raised, because the instance variable doesn't
exist yet, so you can't add two to it.

Then you don't approve of inheritance? That's fine, it is your choice, but
as far as I know, all OO languages include inheritance. I can't imagine
why you would want this to happen:

py> class BetterList(list):
.... def wobble(self):
.... """Wobble a list."""
.... pass
....
py> L = BetterList((1, 2, 3))
py> L.sort()
Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: 'BetterList' object has no attribute 'sort'

(It just goes to show that Explicit is better than Implicit is not
*always* true.)

If the b.a refers to the class variable then two should be added to it.

Neither happens instead we get some hybrid in which an instance varible
is created that gets the value of class variable incrented by two.

Which is precisely the expected behaviour. First you fetch the instance
attribute, which by the rules of inheritance falls back to the value
of the class attribute if it doesn't yet exist (which, in this specific
case, it does not). Then you add two to it, and store the result in the
instance attribute. You can't increment the object 1 because it is
immutable.

There is no instance variable at that point. How can it add 2, to
something that doesn't exist at the moment.

By the standard rules of inheritance.

 

[Antoon Pardon]

...


Because it really is executed as:

b.a = b.a + 2

That is an explanation, not a reason.

1. get 't'b.a and store it in a temporary 't' (found the instance)
2. add 2 to 't'
3. store 't' in 'b.a'

The last operation stores it into an instance variable.

[ I think you mean '(found the class variable)' in line 1 ]

All you are doing here is explain how the current implemantation treats
this. You are not giving arguments for why the result of this
implementation should be considered sane behaviour.

See the 'global' keyword.

You missed the point. If python being a dynamic language would be the
answer to the interaction between instance and class variable, why
then is the interaction between local and global variable different,
why wouldn't the f code be executed as follows

1. get a and store in in a temporary 't' (found the global)
2. add 2 to 't'
3. store 't' in 'a'

The last operation storing it in f's local namespace.

 

[Steven D'Aprano]

No. I don't think a superior outcome is necessary to see that this is
not sane behaviour. I don't care that much on how it gets fixed.

It isn't broken, there is nothing to fix. The code does precisely what the
inheritance model promises to do.

 

[Stefan Arentz]

That is an explanation, not a reason.

I'm just following the

1. get 't'b.a and store it in a temporary 't' (found the instance)
2. add 2 to 't'
3. store 't' in 'b.a'

The last operation stores it into an instance variable.

[ I think you mean '(found the class variable)' in line 1 ]

All you are doing here is explain how the current implemantation treats
this. You are not giving arguments for why the result of this
implementation should be considered sane behaviour.

Ah yes. Well, good luck with that. You seem to have decided that it is not
sane and who am I to argue with that. It depends on your state of mind

The model makes sense in my opinion. If you don't like it then there are
plenty of other languages to choose from that have decided to implement
things differently.

Have fun, take care!

S.

 

[Steven D'Aprano]

I would expect a result consistent with the fact that both times
b.a would refer to the same object.

class RedList(list):
colour = "red"

L = RedList(())

What behaviour would you expect from len(L), given that L doesn't have a
__len__ attribute?

I didn't know I did.

It seems to me that you do.

You didn't appear to be objecting to a line like x = b.a assigning the
value of 1 to x (although perhaps you do). If that was the case, then it
is perfectly reasonable to expect b.a = x + 2 to store 3 into b.a, while
leaving b.__class__.a untouched.

Of course, if you object to inheritance, then you will object to x = b.a
as well.

You are now talking implementation details. I don't care about whatever
explanation you give in terms of implementation details. I don't think
it is sane that in a language multiple occurence of something like b.a
in the same line can refer to different objects

That's an implementation detail only in the sense that "while condition"
is a loop is an implementation detail. It is a *design* detail.

b is a name, and any reference to b (in the same namespace) will refer
to the same object. At least until you rebind it to another object.

But b.a is not a name, it is an attribute lookup, and by Python's rules of
inheritance that lookup will look up attributes in the instance, the
class, and finally any superclasses.

If you persist in thinking of b.a as a name referring to a single object,
of course you will be confused by the behaviour. But that's not what
attribute lookup does.

On the right hand side of an assignment, it will return the first existing
of b.__dict__['a'] or b.__class__.__dict__['a']. On the left hand of an
assignment, it will store into b.__dict__['a'].

I think it even less sane, if the same occurce of b.a refers to two
different objects, like in b.a += 2

Then it seems to me you have some serious design problems. Which would you
prefer to happen?


# Scenario 1
# imaginary pseudo-Python code with no inheritance:
class Paragraph:
ls = '\n' # line separator

para = Paragraph()
para.ls

=> AttributeError - instance has no attribute 'ls'



# Scenario 2
# imaginary pseudo-Python code with special inheritance:
class Paragraph:
ls = '\n' # line separator

linux_para = Paragraph()
windows_para = Paragraph()
windows_para.ls = '\n\r' # magically assigns to the class attribute
linux_para.ls

=> prints '\n\r'


# Scenario 3
# Python code with standard inheritance:
class Paragraph:
ls = '\n' # line separator

linux_para = Paragraph()
windows_para = Paragraph()
windows_para.ls = '\n\r'
linux_para.ls

=> prints '\n'

 

[Steven D'Aprano]

Antoon Pardon enlightened us with:

"b.a" is just a name, not a pointer to a spot in memory. Getting the
value associated with that name is something different from assigning
a new value to that name.

You've got the right concept, but not the right terminology. The "b"
before the dot is a name. The "a" after the dot is a name. But the whole
thing together is not a name: b.a is an attribute reference. You wouldn't
call b["a"] a name, and you shouldn't call b.a a name either.

 

[Steven D'Aprano]

Suppose I have code like this:

for i in xrange(1,11):
b.a = b.a + i

Now the b.a on the right hand side refers to A.a the first time through
the loop but not the next times. I don't think it is sane that which
object is refered to depends on how many times you already went through
the loop.

Well, then you must think this code is *completely* insane too:

py> x = 0
py> for i in range(1, 5):
.... x += i
.... print id(x)
....
140838200
140840184
140843160
140847128



Look at that: the object which is referred to depends on how many times
you've already been through the loop. How nuts is that?

I guess that brings us back to making ints mutable. I can't wait until I
can write 1 - 0 = 99 and still be correct!

 

[Steven D'Aprano]

Fine, we have the code:

b.a += 2

We found the class variable, because there is no instance variable,
then why is the class variable not incremented by two now?

Because b.a += 2 expands to b.a = b.a + 2. Why would you want b.a =
<something> to correspond to b.__class__.a = <something>?

I'm not saying that it couldn't, if that was the model for inheritance you
decided to use. I'm asking why would you want it? What is your usage case
that demonstrates that your preferred inheritance model is useful?

 

[Steven D'Aprano]

This mixing of class and instance variable might be the cause of
confusion...

I think of it as follows:
1 When the class statement ends a class object is created which is
filled by all the statements inside the class statement
This means all variables and functions (methods) are created according
to the description.
NOTE This happens just once.
Yes.

2 When an instance of the class is created, what effectively happens is
that a shallow copy of the class object is made.
Simple values and object references are copied.

No.

py> class Parrot:
.... var = 0
....
py> p = Parrot()
py> Parrot.var is p.var
True
py> Parrot.var = {"Hello world": [0, 1, 2]}
py> Parrot.var is p.var
True

It all boils down to inheritance. When Python does a look up of an
attribute, it looks for an instance attribute first (effectively trying
instance.__dict__['name']). If that fails, it looks up the class second
with instance.__class__.__dict__['name'], and if that fails it goes into a
more complex search path looking up any superclasses (if any).

This explains:
- why methods and complex objects (e.g. lists) are shared among
instances of a class and the class itself
- simple values are not shared

No. it is all about the inheritance, and mutable/immutable objects.

 

[Eric Nieuwland]

2 When an instance of the class is created, what effectively happens
is
that a shallow copy of the class object is made.
Simple values and object references are copied.

No.

py> class Parrot:
... var = 0
...
py> p = Parrot()
py> Parrot.var is p.var
True
py> Parrot.var = {"Hello world": [0, 1, 2]}
py> Parrot.var is p.var
True

It all boils down to inheritance. When Python does a look up of an
attribute, it looks for an instance attribute first (effectively trying
instance.__dict__['name']). If that fails, it looks up the class second
with instance.__class__.__dict__['name'], and if that fails it goes
into a
more complex search path looking up any superclasses (if any).

Note my use of "effectively" and "shallow copy". Your example
demonstrates how Python postpones the shallow copy until you tell the
object to differ from the class/

The examples used only use a class an an instance thereof. They are
valid without inheritance.

No. it is all about the inheritance, and mutable/immutable objects.

NO. You're referring to the implemented mechanism. Other
implementations with the same semantics are possible.

 

[Donn Cave]

... a = [1]
...

b=C()
b.a += [2]
b.a [1, 2]
C.a

[1, 2]

I can understand that Guido was a bit reluctant to introduce
+= etc into Python, and it's important to understand that they
typically behave differently for immutable and mutable objects.

As far as I know, Guido has never added a feature reluctantly.
He can take full responsibility for this misguided wart.

Donn Cave, (e-mail address removed)