Class variable inheritance

[Henry 'Pi' James]

I've just found out that a subclass shares the class variables of its
superclass until it's instantiated for the first time, but not any
more afterwards:

Python 3.1 (r31:73574, Jun 26 2009, 20:21:35) [MSC v.1500 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information..... n = 0
.... def __init__(self):
.... type(self).n += 1((5, 5, 4), (6, 6, 4), (5, 6, 5))

This makes no sense to me at all. Could it possibly be a bug?

 

[Steven D'Aprano]

I've just found out that a subclass shares the class variables

String variables are strings.

Int variables are ints.

Float variables are floats.

List variables are lists.

Class variables are classes.

Classes are first-class objects in Python. Perhaps you mean class
attributes?

of its
superclass until it's instantiated for the first time, but not any more
afterwards: ....
This makes no sense to me at all. Could it possibly be a bug?

You have misinterpreted what you have seen, and if there's a bug, it's in
your code.

When you retrieve an attribute, Python first looks for instance
attributes, then class attributes, then attributes attached to
superclasses.

When you assign to an attribute, Python conceptually uses the exact same
"search path", except that instance assignment always succeeds.

(Well, almost -- but if it fails, you get an exception.)

So in practice:

x = obj.whatever

may return the contents of an instance attribute "whatever", a class
attribute, or an attribute of a superclass. But:

obj.whatever = x

always attempts to store x as an instance attribute, because there's no
way for Python to read your mind and know that you mean a class attribute
unless you say so explicitly. This attempt will either succeed, or it
will raise an exception. Python doesn't try writing further along the
hierarchy of instance/class/superclass(es).

If you wish to write to a class attribute, you have to explicitly say so:

obj.__class__.whatever = x


Your other misunderstanding relates to augmented assignment:

x += 1

does not modify x in place, it is *exactly* equivalent to:

x = x + 1

Given the rules of attribute access, obj.whatever += 1 is exactly
equivalent to:

obj.whatever = obj.whatever + 1

The right hand attribute access finds a class attribute, and the left
hand one sets an instance attribute.

 

[Chris Rebert]

I've just found out that a subclass shares the class variables of its
superclass until it's instantiated for the first time, but not any
more afterwards:

Python 3.1 (r31:73574, Jun 26 2009, 20:21:35) [MSC v.1500 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information....   n = 0
...   def __init__(self):
...     type(self).n += 1...   pass

This makes no sense to me at all. Could it possibly be a bug?

Makes sense to me. To step through what's happening:
(0, 0)

Here, the lookup on B.n fails (that is, B itself has no variable n),
and thus falls back to A.n
Thus, at this point, the expressions `A.n` and `B.n` are equivalent.
(1, 1, 1)

A.__init__() gets called, incrementing A.n; again, B.n falls back to A.n
(2, 2, 2),

(3, 2, 3)

A.__init__() gets called since B did not define one of its own and
this inherited A's.
Therein, type(self) evaluates to B (not A as before).
B.n += 1 is in this case equivalent to:
B.n = B.n +1
Evaluating the right side, B.n falls back A.n once again, and we add 1.
Now the assignment takes place, creating a new variable B.n, *separate* from A.n

From hereon in, lookups of B.n don't fall back to A.n since B now has

its own variable 'n'.
Thus, the values of A.n and B.n differ and the expressions now refer
to 2 distinct variables.

The rest falls out from this and is left as an exercise for the reader.

Cheers,
Chris

 

[HPJ]

Makes sense to me. To step through what's happening:

(0, 0)

Here, the lookup on B.n fails (that is, B itself has no variable n),
and thus falls back to A.n

See, this is what tripped me up, right at the beginning. I thought B
would inherit (as in copy) the variable n from A.

Can you point out to me which part (or parts) of the Language
Reference says this is the way it's supposed to be?

 

[Terry Reedy]

See, this is what tripped me up, right at the beginning. I thought B
would inherit (as in copy) the variable n from A.

Python does not copy objects unless asked to.
Inheritance is a link request, not a copy request.

Can you point out to me which part (or parts) of the Language
Reference says this is the way it's supposed to be?

I could, but I will let you read and find what it says about class
attributes.

tjr

 

[HPJ]

I could, but I will let you read and find what it says about class

attributes.

You think I would have asked specifically about the Language Reference
if I hadn't read it and failed to find what I was looking for?

The closest thing I was able to find was section 3.2. "The standard
type hierarchy", subsection "Custom classes", where it says:

"When the attribute name is not found [in its namespace dictionary],
the attribute search continues in the base classes. This search of the
base classes uses the C3 method resolution order [...]"

That tells me how the lookup works, which I already knew. But it
doesn't tell me what happens when a class is inherited.

Now, one could argue if the attributes were copied, there would be no
need for a lookup in the base classes, so derivatively it says that's
not the case. To this I would say yes, there still would because after
they've been copied through inheritance, attributes can still be
removed via "del". Anyway, this passage, which is the only one I could
find, doesn't provide a direct and conclusive answer to my question.

 

[Steven D'Aprano]

See, this is what tripped me up, right at the beginning. I thought B
would inherit (as in copy) the variable n from A.

Inherit does not mean copy. What makes you think it does? Given the
following:

class A(object):
def foo(self):
return "foo"

class B(A):
pass

would you expect the B class to have a copy of the foo method?

 

[HPJ]

would you expect the B class to have a copy of the foo method?

Sorta. I would expect B to have a copy of the "foo" attribute, which
then refers to the same method as A.foo. So the method itself will be
be copied, but its address stored separately in A.foo and B.foo.

 

[Carl Banks]

Sorta. I would expect B to have a copy of the "foo" attribute, which
then refers to the same method as A.foo. So the method itself will be
be copied, but its address stored separately in A.foo and B.foo.

No, I'm afraid not. Here is what happens. Conceptually, Python
checks for the presence of B.foo, and if it's not there it checks for
foo's presence in the base classes. (In actuality Python premaps
attributes to the approprirate base class, so only two dict lookups
are necessary.)

Python is a very dynamic langauge which allows you to modify class
objects at runtime. So if you inherit from a class, then later modify
that class, what should happen?


class A(object):
foo = 1

class B(A):
pass

A.foo = 2

print B.foo # what should this print, 1 or 2?


You could argue that copying class attributes (so that B.foo would be
1) is a reasonable way to do inheritance, but IMO the referencing
attributes in base classes reflects the underlying concept of
inheritance better.


Carl Banks

 

[HPJ]

Conceptually, Python checks for the presence of B.foo, and if it's

not there it checks for foo's presence in the base classes.

Yes, I have no problem believing you guys that this is what Python
does. Still, my question remains about where in the Language Reference
this is specified. And if the answer is nowhere, than the LR needs to
be amended, for obviously the way inheritance is done is no small
matter and its understanding should not be left to the user's own
intuition.

 

[Mark Hammond]

Yes, I have no problem believing you guys that this is what Python
does. Still, my question remains about where in the Language Reference
this is specified. And if the answer is nowhere, than the LR needs to
be amended, for obviously the way inheritance is done is no small
matter and its understanding should not be left to the user's own
intuition.

http://docs.python.org/reference/datamodel.html#new-style-and-classic-classes
- search for 'method resolution order' for other hits in that document.

HTH,

Mark

 

[HPJ]

http://docs.python.org/reference/datamodel.html#new-style-and-classic...

- search for 'method resolution order' for other hits in that document.

First of all, that's the LR for Python 2, I'm with Python 3. Second of
all, there's one single passage containing the phrase "method
resolution order" in the Python 3 LR, and it's one that I've already
quoted -- and discarded -- previously in this thread.

 

[HPJ]

And by the way, the reason I've come across this problem at all is
because I have something like this:

class A:
class X:
n = 'a'
x = X()

class B(A):
class X(A.X):
n = 'b'
# x = X()

The line commented out was originally not there, but I found out I had
to add it if I wanted B().x.n to be 'b' instead of 'a'.

I might be wrong, but aren't there some (non-obscure) OOP language in
which the equivalent code (without the out-commented line) would have
made B().x an object of type B.X instead of A.X?

 

[Steven D'Aprano]

You think I would have asked specifically about the Language Reference
if I hadn't read it and failed to find what I was looking for?

You must be new to the Internet *wink*

Of course people ask without having made the effort themselves. "Please
sir, will you do my work for me?" is practically the norm on Internet
forums. Only without the please.

The closest thing I was able to find was section 3.2. "The standard type
hierarchy", subsection "Custom classes", where it says:

"When the attribute name is not found [in its namespace dictionary], the
attribute search continues in the base classes. This search of the base
classes uses the C3 method resolution order [...]"

That tells me how the lookup works, which I already knew. But it doesn't
tell me what happens when a class is inherited.

The inheriting class has its base class set, and the MRO (method
resolution order), and that's pretty much it as far as I can tell.

Seems to me that the documentation is a little incomplete in this regard.
It looks to me like one of those "language lawyer" details that needs
fleshing out.

Out of curiosity, are there languages where inheritance means copy?

 

[Carl Banks]

Yes, I have no problem believing you guys that this is what Python
does. Still, my question remains about where in the Language Reference
this is specified. And if the answer is nowhere, than the LR needs to
be amended, for obviously the way inheritance is done is no small
matter and its understanding should not be left to the user's own
intuition.

Well, I don't know if this detail alone is all that grave an omission--
it'll matter to very few users--but it does seem the LRM could use a
subsection on inheritance in general. I'm sure the doc maintainers
would welcome a contribution.

Carl Banks

 

[Carl Banks]

And by the way, the reason I've come across this problem at all is
because I have something like this:

class A:
  class X:
    n = 'a'
  x = X()

class B(A):
  class X(A.X):
    n = 'b'
# x = X()

You've nested classes here, that's a whole new layer of complexity.

I would strongly recommend against nesting classes in Python, unless
the inner class is a smallish class (such as a custom exception type)
that doesn't interact with the outer class. It's just that nested
classes don't ever seem to behave like anyone expects them to. Also
you can't take advantage of Python's lexical scoping with nested
classes (unlike, for instance, Java) so there really isn't much
benefit to nesting them.

The line commented out was originally not there, but I found out I had
to add it if I wanted B().x.n to be 'b' instead of 'a'.

Hm, even if class B did get copies of class A's attributes, B().x.n
would still return 'a'. It seems as if you expect class B to re-
execute A's statements in B's context, or something. That's not how
it works at all.

(Now if you define self.x=X() inside of __init__, that's a different
story.)

I might be wrong, but aren't there some (non-obscure) OOP language in
which the equivalent code (without the out-commented line) would have
made B().x an object of type B.X instead of A.X?

Maybe. For some languages this might be an obvious behavior, but
Python would have different circumstances so it isn't so obvious (or
possible) there.


Carl Banks

 

[Chris Rebert]

On Tue, Sep 8, 2009 at 11:30 PM, Steven

Out of curiosity, are there languages where inheritance means copy?

I think some prototype-based ones handle it that way...

Cheers,
Chris

 

[HPJ]

Maybe.  For some languages this might be an obvious behavior, but

Python would have different circumstances so it isn't so obvious (or
possible) there.

Which means this topic definitely needs to be handled in detail by the
LR, and preferably also in the Tutorial. Otherwise there is no way
anyone coming to Python could know which of the many ways of
inheritance Python follows.

 

[Carl Banks]

Which means this topic definitely needs to be handled in detail by the
LR, and preferably also in the Tutorial.

As I said, I'm sure the doc maintainers would likey welcome your
contributions.

Otherwise there is no way
anyone coming to Python could know which of the many ways of
inheritance Python follows.

"Mo way"? You are definitely overstating things here.


Carl Banks

 

[Lie Ryan]

And by the way, the reason I've come across this problem at all is
because I have something like this:

class A:
class X:
n = 'a'
x = X()

class B(A):
class X(A.X):
n = 'b'
# x = X()

The line commented out was originally not there, but I found out I had
to add it if I wanted B().x.n to be 'b' instead of 'a'.

I might be wrong, but aren't there some (non-obscure) OOP language in
which the equivalent code (without the out-commented line) would have
made B().x an object of type B.X instead of A.X?

Perhaps in a language where classes attribute are declarative. Not in
python where class attributes are executable statements.

Note that when the python interpreter meets this statement:

class B(P):
def foo(self):
print('ab')
X = 'f'

the compiler sees a class statement -> create a new blank class
-> assign P as the new class' parent
-> *execute* the class' body

# new context
the compiler sees a def statement -> *compile* the def's body
-> assign the compiled body to B.foo
the compiler sees X = 'f' statement -> assign 'f' to B.X
# exit the new context

-> assign the new class to B

Your problem is related to how the interpreter *execute* a class
definition rather than the name resolution.