MRO theory

[Clarence]

I'm having problems creating classes because of "can't create a
consistent mro" problems.

I noticed, in a test program, that if the base class list that I
pass to type.__new__ is sorted (using default keys, so presumably
sorting by the id's of the class objects), that the problem goes away.

Now in this test, the id's (memory addresses) of the class objects
are presumably in the same order as the chronological order that
the classes were created in (since the program is simple and doesn't
delete any objects or make any temporaries). This may sound weird,
but I don't actually care what order the base classes appear in as
long as they're all there.

This makes me wonder, since I can't say I understand the theory
(I read the paper referenced in typeobject.c source, my brain
didn't _quite_ melt)
behind mro creation, if it is the case that there cannot be a
consistency problem if each class in the base list was created
after all classes that precede it in the base list.

Can any genius weigh in on this question?

 

[Steve Holden]

I'm having problems creating classes because of "can't create a
consistent mro" problems.

I noticed, in a test program, that if the base class list that I
pass to type.__new__ is sorted (using default keys, so presumably
sorting by the id's of the class objects), that the problem goes away.

Now in this test, the id's (memory addresses) of the class objects
are presumably in the same order as the chronological order that
the classes were created in (since the program is simple and doesn't
delete any objects or make any temporaries). This may sound weird,
but I don't actually care what order the base classes appear in as
long as they're all there.

This makes me wonder, since I can't say I understand the theory
(I read the paper referenced in typeobject.c source, my brain
didn't _quite_ melt)
behind mro creation, if it is the case that there cannot be a
consistency problem if each class in the base list was created
after all classes that precede it in the base list.

Can any genius weigh in on this question?

Without wishing to lay claim to genius, I'd suggest that you don;'t
write programs relying on multiple inheritance until you have a thorough
understanding of its principles.

The inability of the interpreter to create a consistent mro has nothing
to do with the order in which the base classes were created or where
they appear in memory. It's probably because you have introduced a
circularity: you are trying to define something based on A, which is
based on B, which is based on A or something similar.

regards
Steve

 

[Clarence]

Without wishing to lay claim to genius, I'd suggest that you don;'t
write programs relying on multiple inheritance until you have a thorough
understanding of its principles.

The inability of the interpreter to create a consistent mro has nothing
to do with the order in which the base classes were created or where
they appear in memory. It's probably because you have introduced a
circularity: you are trying to define something based on A, which is
based on B, which is based on A or something similar.

No, there is no circularity. The problem is coming about in the JPype
application. It creates a Python class to proxy any Java class or Java
interface that the program uses (directly or indirectly). As far as
the
Java proxies go, when it creates a class it can have at most one
superclass
that is a proxy of a Java class, and arbitrarily many superclasses
which
are proxies of Java interfaces.

The problem basically comes about because interfaces are not classes,
but a class is being created to stand in for each one.

There is real Java code that is causing a class construction failure
because of an inability to construct a consistent mro. I've tried
several
approaches to make it work, and by accident, sorting the list of
interface
proxy classes, using the default comparison, make it work. That made
me
wonder if the theory had something to say. Obviously, if class A is
created
before class B, then B cannot be a base class of A, hence the
chronological
order of class creation _might_ have something to do with creating
acceptable
or unacceptable mro's.

 

[Carl Banks]

No, there is no circularity. The problem is coming about in the JPype
application. It creates a Python class to proxy any Java class or Java
interface that the program uses (directly or indirectly). As far as
the
Java proxies go, when it creates a class it can have at most one
superclass
that is a proxy of a Java class, and arbitrarily many superclasses
which
are proxies of Java interfaces.

The problem basically comes about because interfaces are not classes,
but a class is being created to stand in for each one.

Yes. In Java, because there's no possibility of conflicting methods,
interfaces can be listed in any order. But when proxying Java classes
with Python, the order of interfaces becomes important.

There is real Java code that is causing a class construction failure
because of an inability to construct a consistent mro. I've tried
several
approaches to make it work, and by accident, sorting the list of
interface
proxy classes, using the default comparison, make it work.

It *might* not be an accident.

Because Java interfaces can be listed in any order, it's quite
possible that some interface hierarchies in Java violate Python's MRO
restrictions right out of the box. I would guess that JPype deals
with this situation by applying a consistent ordering to of bases when
it creates a proxy class. And it just might turn out that "order of
creation" is the sort key. Or something else could be the key
(alphabetical by name?). Regardless, sorted order just *might* be the
proper order to list bases to ensure MRO consistency.

Or I could be way off base.

I'd suggest that this is probably a question better answered on the
JPype mailing list, since they would have answers and not just
speculations.

That made
me
wonder if the theory had something to say. Obviously, if class A is
created
before class B, then B cannot be a base class of A, hence the
chronological
order of class creation _might_ have something to do with creating
acceptable
or unacceptable mro's.

I doubt it "happens" to work that way. If sorting the bases works in
general, it's probably deliberate.


Carl Banks

 

[Clarence]

It *might* not be an accident.

Actually, what I meant to write was that my sorting was done by
accident, but the heart of my question is, *was* it an accident? I
don't have enough theory to answer that question.

Because Java interfaces can be listed in any order, it's quite
possible that some interface hierarchies in Java violate Python's MRO
restrictions right out of the box. I would guess that JPype deals
with this situation by applying a consistent ordering to of bases when
it creates a proxy class. And it just might turn out that "order of
creation" is the sort key. Or something else could be the key
(alphabetical by name?). Regardless, sorted order just *might* be the
proper order to list bases to ensure MRO consistency.

I'd suggest that this is probably a question better answered on the
JPype mailing list, since they would have answers and not just
speculations.

It's a theoretical question, and Python implements the theory. And no,
JPype does not deal with it at all; that's what I'm trying to fix.

I know sorting by name does not work because that's one of the things
I tried. You can start out with sorted base class lists, but the
merging
involved in creating the mro of a new class destroys that ordering. I
find it very reasonable, however, to think that the creation order of
the classes (implying constraints on the possible superclass
relationships)
may well be a property that the merge will always leave untouched.

Hence my question.