[newbie] super() and multiple inheritance

[hermy]

Hi,
I'm trying to figure out how to pass constructor arguments to my
superclasses in a multiple inheritance situation.

As I understand it, using super() is the preferred way to call
the next method in method-resolution-order. When I have parameterless
__init__ methods, this works as expected.
However, how do you solve the following simple multiple inheritance
situation in python ?

class A(object):
def __init__(self,x):
super(A,self).__init__(x)
print "A init (x=%s)" % x

class B(object):
def __init__(self,y):
super(B,self).__init__(y)
print "B init (y=%s)" % y

class C(A,B):
def __init__(self,x,y):
super(C,self).__init__(x,y) <-------- how to do this ???
print "C init (x=%s,y=%s)" % (x,y)

What I want is that when I create a class C object
x = C(10,20)
that the x argument of C's __init__ is used to initialize the
A superclass, and the y argument is used to initialize the B
superclass.
In C++, I would do this using initilaization lists, like:
C::C(int x, int y) : A(x), B(y) { ... }

I'm probably overlooking some basic stuff here, but I haven't
been able to figure this out. Googling got me lots of examples,
but all with empty __init__ argument lists (which obviously works,
but is too trivial in practice).

regards,
herman

 

[Steven Bethard]

As I understand it, using super() is the preferred way to call
the next method in method-resolution-order. When I have parameterless
__init__ methods, this works as expected.
However, how do you solve the following simple multiple inheritance
situation in python ?

class A(object):
def __init__(self,x):
super(A,self).__init__(x)
print "A init (x=%s)" % x

class B(object):
def __init__(self,y):
super(B,self).__init__(y)
print "B init (y=%s)" % y

class C(A,B):
def __init__(self,x,y):
super(C,self).__init__(x,y) <-------- how to do this ???
print "C init (x=%s,y=%s)" % (x,y)

Unfortunately, super() doesn't mix too well with hierarchies that change
the number of arguments to a method. One possibility:

class A(object):
def __init__(self, x, **kwargs):
super(A, self).__init__(x=x, **kwargs)
print "A init (x=%s)" % x

class B(object):
def __init__(self, y, **kwargs):
super(B, self).__init__(y=y, **kwargs)
print "B init (y=%s)" % y

class C(A,B):
def __init__(self, x, y):
super(C, self).__init__(x=x,y=y)
print "C init (x=%s,y=%s)" % (x,y)

Then you can get::

py> C(1, 2)
B init (y=2)
A init (x=1)
C init (x=1,y=2)
<__main__.C object at 0x00B9FA70>

But you have to make sure to always pass the **kwargs around.

STeVe

 

[Carl Banks]

Hi,
I'm trying to figure out how to pass constructor arguments to my
superclasses in a multiple inheritance situation.

As I understand it, using super() is the preferred way to call
the next method in method-resolution-order. When I have parameterless
__init__ methods, this works as expected.
However, how do you solve the following simple multiple inheritance
situation in python ?

class A(object):
def __init__(self,x):
super(A,self).__init__(x)
print "A init (x=%s)" % x

class B(object):
def __init__(self,y):
super(B,self).__init__(y)
print "B init (y=%s)" % y

class C(A,B):
def __init__(self,x,y):
super(C,self).__init__(x,y) <-------- how to do this ???
print "C init (x=%s,y=%s)" % (x,y)

What I want is that when I create a class C object
x = C(10,20)
that the x argument of C's __init__ is used to initialize the
A superclass, and the y argument is used to initialize the B
superclass.
In C++, I would do this using initilaization lists, like:
C::C(int x, int y) : A(x), B(y) { ... }

Well, technically, you can't do this in C++ at all.

A little explanation. In multiple inheritance situations, Python has a
significant difference from regular C++ inheritance: in C++, if you
multiply-inherit from two different classes that both inherit from the
same base class, the resulting structure has two copies of the data
associated with the base class. In Python, only there is only one
copy. If you want only one copy of the base class's data in C++, you
must use virtual inheritance.

But here's the thing: in C++, you can't initialize a virtual base class
in the constructor. A virtual base class is always initialized with
the default constructor. The reason for this is obvious: otherwise,
you could end up initializing the virtual base twice with different
arguments.

This also explains why, in Python, super is preferred for multiple
inheritance: it guarantees that each base class's __init__ is called
only once. This comes at the price of less flexibility with the
function arguments, but in Python, at least you can use function
arguments.

So now, let's talk about solutions.

Now that we know why super is preferred, we can make a somewhat
intelligent decision whether to go against the advice. If you know
your inheritance hierarchy is not going to have any two classes
inheriting from the same base class (except for object), then you could
just call each class's __init__ directly, same as you would have done
with old-style classes. There is no danger of initializing any base
class twice and no reason for super to be preferred here.

A.__init__(self,x)
B.__init__(self.y)

But if you can't or don't want to do this, you'll have to make some
concessions with the argument lists. One thing to do would have A and
B both accept x and y, using only the one it needs. A more general
approach might be to use keyword arguments. For example (you can
improve upon this):

class A(object):
def __init__(self,**kwargs):
use(kwargs['x'])

class B(object):
def __init__(self,**kwargs):
use(kwargs['y'])

class C(A,B):
def __init__(self,**kwargs):
super(C,self).__init__(**kwargs)

C(x=1,y=2)

I'm probably overlooking some basic stuff here,

Unfortunately, it doesn't appear that you are. You'll have to choose
between calling base class __init__s old-style, or fiddling with their
argument lists.


Carl Banks

 

[hermy]

Carl Banks schreef:

Hi,
I'm trying to figure out how to pass constructor arguments to my
superclasses in a multiple inheritance situation.

As I understand it, using super() is the preferred way to call
the next method in method-resolution-order. When I have parameterless
__init__ methods, this works as expected.
However, how do you solve the following simple multiple inheritance
situation in python ?

class A(object):
def __init__(self,x):
super(A,self).__init__(x)
print "A init (x=%s)" % x

class B(object):
def __init__(self,y):
super(B,self).__init__(y)
print "B init (y=%s)" % y

class C(A,B):
def __init__(self,x,y):
super(C,self).__init__(x,y) <-------- how to do this ???
print "C init (x=%s,y=%s)" % (x,y)

What I want is that when I create a class C object
x = C(10,20)
that the x argument of C's __init__ is used to initialize the
A superclass, and the y argument is used to initialize the B
superclass.
In C++, I would do this using initilaization lists, like:
C::C(int x, int y) : A(x), B(y) { ... }

Well, technically, you can't do this in C++ at all.

A little explanation. In multiple inheritance situations, Python has a
significant difference from regular C++ inheritance: in C++, if you
multiply-inherit from two different classes that both inherit from the
same base class, the resulting structure has two copies of the data
associated with the base class. In Python, only there is only one
copy. If you want only one copy of the base class's data in C++, you
must use virtual inheritance.

But here's the thing: in C++, you can't initialize a virtual base class
in the constructor. A virtual base class is always initialized with
the default constructor. The reason for this is obvious: otherwise,
you could end up initializing the virtual base twice with different
arguments.

This also explains why, in Python, super is preferred for multiple
inheritance: it guarantees that each base class's __init__ is called
only once. This comes at the price of less flexibility with the
function arguments, but in Python, at least you can use function
arguments.

So now, let's talk about solutions.

Now that we know why super is preferred, we can make a somewhat
intelligent decision whether to go against the advice. If you know
your inheritance hierarchy is not going to have any two classes
inheriting from the same base class (except for object), then you could
just call each class's __init__ directly, same as you would have done
with old-style classes. There is no danger of initializing any base
class twice and no reason for super to be preferred here.

A.__init__(self,x)
B.__init__(self.y)

But if you can't or don't want to do this, you'll have to make some
concessions with the argument lists. One thing to do would have A and
B both accept x and y, using only the one it needs. A more general
approach might be to use keyword arguments. For example (you can
improve upon this):

class A(object):
def __init__(self,**kwargs):
use(kwargs['x'])

class B(object):
def __init__(self,**kwargs):
use(kwargs['y'])

class C(A,B):
def __init__(self,**kwargs):
super(C,self).__init__(**kwargs)

C(x=1,y=2)

I'm probably overlooking some basic stuff here,

Unfortunately, it doesn't appear that you are. You'll have to choose
between calling base class __init__s old-style, or fiddling with their
argument lists.


Carl Banks

Thanx, I think I got it (please correct me if I'm wrong):
o super(C,self) determines the next class in the inheritance hierarchy
according to
method resolution order, and simply calls the specified method on it
(in this case
__init__ with the specified argument list.
o since I cannot now beforehand where my class will appear in the
inheritance
hierarchy when __init__ is called, I need to pass on all the
arguments and let
my method decide which ones to use.

On the other hand, when I use old-style, s.a. B.__init__(args), I
specify statically
in which class the method lookup should occur.

Unfortunately, new and old style don't mix well (as I found out by
experimenting a little),
so for new code I should probably stick to new style, and use super.

Which leads me to my original problem. Your suggestion with **kwargs
works fine,
but only if it's used consistently, and I'll probably need to do some
name-mangling
to encode class names in parameter names in order to avoid name
clashes.

Unfortunately, this solution is (as far as I know) not universally
accepted, so if I want
to multiply inherit from other people's classes (who didn't follow this
solution), it won't
work. Short: I can make it work if I write all the classes myself, I
can't make it work if
I try to re-use other people's code.
Which is a pity, since this means that I can't even use multiple
inheritance for mixin
classes (where I know that no diamond will appear in the hierarchy, and
I have a simple
tree - except for the shared object superclass).

So, for the moment my conclusion is that although Python has some
syntax for
multiple inheritance, it doesn't support it very well, and I should
probably stick to
single inheritance.
Also, if there would be some language support for the argument passing
problem
(especially for __init__ methods), multiple inheritance would work just
fine. Hopefully
some future version of Python will solve this.

regards,
herman

 

[Carl Banks]

Thanx, I think I got it (please correct me if I'm wrong):
o super(C,self) determines the next class in the inheritance hierarchy
according to
method resolution order, and simply calls the specified method on it
(in this case
__init__ with the specified argument list.
o since I cannot now beforehand where my class will appear in the
inheritance
hierarchy when __init__ is called, I need to pass on all the
arguments and let
my method decide which ones to use.

On the other hand, when I use old-style, s.a. B.__init__(args), I
specify statically
in which class the method lookup should occur.

Unfortunately, new and old style don't mix well (as I found out by
experimenting a little),
so for new code I should probably stick to new style, and use super.

Well I wasn't suggesting you mix styles; I was suggesting all old, all
the way here. With due care, the effect of using this style over the
entire hierarchy is to call the __init__ functions possibly out of MRO
sequence. You have to pay some attention to what methods are called in
__init__ anyways, so this isn't exactly a whole new dimension of
carefulness.

Which leads me to my original problem. Your suggestion with **kwargs
works fine,
but only if it's used consistently, and I'll probably need to do some
name-mangling
to encode class names in parameter names in order to avoid name
clashes.

Meh. Do nameclashes happen so often that it's better to implement a
handmade name mangling scheme rather than just change a variable name
when one pops up?

Unfortunately, this solution is (as far as I know) not universally
accepted, so if I want
to multiply inherit from other people's classes (who didn't follow this
solution), it won't
work. Short: I can make it work if I write all the classes myself, I
can't make it work if
I try to re-use other people's code.

I would suggest that, if you're trying to make a mixin of completely
different classes, super is only one of many worries. There's all
sorts of things that could go wrong when you do that in Python. When
you have a class that's not expecting to be in an inheritance
hierarchy, it might not even be calling super. Or if it's a derived
class, it might be using old-style calls to the subclass. Sometimes
classes have incompatible C-footprints and can't be mixed in. Python
objects use a single namespace for their instance variables, so if
these two classes happen to use the same for a variable, you're pretty
much out of luck. If any of the classes use a different metaclass
there could be all sorts of problems.

Bottom line is: if you're going to multiply inherit from two classes,
they have to cooperate in some fashion. You have to be careful, and
sometimes you can't avoid modifying the base classes. (Yes, you can do
that.)

Which is a pity, since this means that I can't even use multiple
inheritance for mixin
classes (where I know that no diamond will appear in the hierarchy, and
I have a simple
tree - except for the shared object superclass).

Um, sure you can. You can't necessarily, but it doesn't mean you can't
never.

So, for the moment my conclusion is that although Python has some
syntax for
multiple inheritance, it doesn't support it very well, and I should
probably stick to
single inheritance.

"well supported" is a relative term. I don't disagree that MI can be
unwieldy in Python, but it seems to me that throwing it out of the
toolshed is a bit extreme here. There's just times when MI is the best
and most elegant solution to a problem, and not using it because you
have to fiddle with __init__ args a little is probably not a good
thing.

Also, if there would be some language support for the argument passing
problem
(especially for __init__ methods), multiple inheritance would work just
fine. Hopefully
some future version of Python will solve this.

Not likely at all. I think the Python philosophy is that MI is a thing
that's occasionally very useful, but not vital. I think the designers
feel that a little unwieldiness in MI is a small price to pay compared
the language changes needed to take away that unwieldiness. For
instance, a foolproof way to avoid name clashes in mixin classes would
require a complete overhaul of namespaces in Python, and that is not
going to happen.

At best, what you might get is some sort of protocol (maybe a
decorator) that standardizes calling the base class methods, coupled
with a very strong suggestion to use it in all future code for all
classes.


Carl Banks

 

[Michele Simionato]

Hermy:

So, for the moment my conclusion is that although Python has some
syntax for multiple inheritance, it doesn't support it very well, and I should
probably stick to single inheritance.

This is not much a problem of Python, the problem is that multiple
inheritance is
intrinsically HARD to support. If you can avoid it, avoid it. This will
probably make
your application more maintanable.

I your case, I would go with the keyword argument suggestion.

Michele Simionato