Builtn super() function. How to use it with multiple inheritance? Andwhy should I use it at all?

L

Lacrima

Hi!

I have two super classes:

class SuperClass1(object):
def __init__(self, word):
print word

class SuperClass2(object):
def __init__(self, word, word2):
print word, word2

Also I have subclass of these classes:

class SubClass(SuperClass1, SuperClass2):
def __init__(self):
pass

I have two questions.
1) Inside __init__ of SubClass how can I firstly call __init__ of
SuperClass1, and then __init__ of SuperClass2, using builtin super()
function.
2) Why should I use super() at all, if it is very easy to call methods
of super class like this:
class SubClass(SuperClass1, SuperClass2):
def __init__(self):
SuperClass1.__init__(self, 'Python')
SuperClass2.__init__(self, 'Hello', 'world')

Thanks in advance.
 
R

Raymond Hettinger

Hi!

I have two super classes:

class SuperClass1(object):
    def __init__(self, word):
        print word

class SuperClass2(object):
    def __init__(self, word, word2):
        print word, word2

Also I have subclass of these classes:

class SubClass(SuperClass1, SuperClass2):
    def __init__(self):
        pass

I have two questions.
1) Inside __init__ of SubClass how can I firstly call __init__ of
SuperClass1, and then __init__ of SuperClass2, using builtin super()
function.

I would write it like this:


class SuperClass1(object):
def __init__(self, **kwds):
word = kwds.pop('word')
print word
super(SuperClass1, self).__init__(**kwds)

class SuperClass2(object):
def __init__(self, **kwds):
word1 = kwds.pop('word1')
word2 = kwds.pop('word2')
print word1, word2
super(SuperClass2, self).__init__(**kwds)

class SubClass(SuperClass1, SuperClass2):
def __init__(self, **kwds):
super(SubClass, self).__init__(**kwds)

SubClass(word='Python', word1='Hello', word2='World')



2) Why should I use super() at all, if it is very easy to call methods
of super class like this:
class SubClass(SuperClass1, SuperClass2):
    def __init__(self):
        SuperClass1.__init__(self, 'Python')
        SuperClass2.__init__(self, 'Hello', 'world')

That works just fine in this case.
The challenge arises in "diamond diagrams"
such as A->B A->C B->D C->D where both B and C
are written independently of D and both need to call
A's __init__ but that method should only be called
once (not once by B and again by C).

In that case, the super() pattern shown above will
let each parent's method be called exactly once
and guarantee that parents are called before grandparents
and guarantee that the left-to-right ordering of multiple
bases is respected.


Raymond
 
L

Lacrima

I would write it like this:

class SuperClass1(object):
    def __init__(self, **kwds):
        word = kwds.pop('word')
        print word
        super(SuperClass1, self).__init__(**kwds)

class SuperClass2(object):
    def __init__(self, **kwds):
        word1 = kwds.pop('word1')
        word2 = kwds.pop('word2')
        print word1, word2
        super(SuperClass2, self).__init__(**kwds)

class SubClass(SuperClass1, SuperClass2):
    def __init__(self, **kwds):
        super(SubClass, self).__init__(**kwds)

SubClass(word='Python', word1='Hello', word2='World')


That works just fine in this case.
The challenge arises in "diamond diagrams"
such as A->B  A->C  B->D  C->D where both B and C
are written independently of D and both need to call
A's __init__ but that method should only be called
once (not once by B and again by C).

In that case, the super() pattern shown above will
let each parent's method be called exactly once
and guarantee that parents are called before grandparents
and guarantee that the left-to-right ordering of multiple
bases is respected.

Raymond

Hi, Raymond!

Thank you for your answer.

Some things are still not clear. Your example works great. But if I
remove "super(SuperClass1, self).__init__(**kwds)" from SuperClass1's
__init__, the example stops working. That is when I instantiate
SubClass only __init__ of SuperClass1 is called and __init__ of
SuperClass2 is omitted, i.e. only 'Python' is printed. Why is it so?

So as I understand every parent should necessarily call super() at the
end of its __init__ method in order for things to work properly.

But what if SuperClass1 is from third party library? Then I can't
modify it to follow this convention, that is when I instantiate my
SubClass only __init__ from SuperClass1 will be called, and __init__
from SuperClass2 will be omitted.
How to deal with that?

My post is quite intricate, but this is because of my English. Sorry.

Looking forward for help. Thank you.
 
G

Gregory Ewing

Lacrima said:
But what if SuperClass1 is from third party library?

If it hasn't been designed for super(), then you
can't use super() with it.

super() only works when *every* class in the
hierarchy has been designed with it in mind.
 
S

Steven D'Aprano

If it hasn't been designed for super(), then you can't use super() with
it.

super() only works when *every* class in the hierarchy has been designed
with it in mind.


That incorrect. You can certainly use super() with classic classes in the
hierarchy, and super() didn't even exist when they were created.

.... def method(self, arg):
.... return repr(arg)
........ def method(self, arg):
.... return "argument was %s" % Parent.method(self, arg)
........ def method(self, arg):
.... return super(New, self).method(arg).upper()
...."ARGUMENT WAS 'SPAM'"


The problem isn't super(), and people who give glib advise "don't use
super()" are just compounding the problem. The problem is with multiple
inheritance where methods have different method signatures. If you don't
change the method signatures, super() will work fine.

Advising people not to use super() might make people feel virtuous, but
it doesn't do anything to help the reader write non-buggy MI hierarchies.
It pushes the effort of dealing with multiple inheritance onto them,
forcing them to re-implement the MRO, probably badly. Can you re-
implement the C3 algorithm? Have you even heard of it? If you answer No
to either of those questions, chances are high that trying to deal with
the MRO manually will lead to worse bugs than using super().

Should you use super()?

1. If you're doing multiple inheritance with metaclasses, you MUST use
super().

2. If you're using single inheritance only, and never modify method
signatures, there is no reason not to use super().

3. If you're using mixins, you should use super().

4. If you never modify method signatures, then you should use super().

5. If you do modify method signatures, you shouldn't do that (except
possibly in constructors, and even there only cautiously). But if you do
it anyway, then you should use super() *except* in the methods where you
modify the signature.

6. If you don't use super(), chances are that your class hierarchy is
still buggy, but instead of failing loudly and noisily with an exception,
it's silently giving the wrong results.

7. If you can avoid multiple inheritance in favour of another technique
(such as composition), you should strongly consider that.
 
R

Raymond Hettinger

Thank you for your answer.

You're welcome.
Some things are still not clear. Your example works great. But if I
remove "super(SuperClass1, self).__init__(**kwds)" from SuperClass1's
__init__, the example stops working. That is when I instantiate
SubClass only __init__ of SuperClass1 is called and __init__ of
SuperClass2 is omitted, i.e. only 'Python' is printed. Why is it so?

So as I understand every parent should necessarily call super() at the
end of its __init__ method in order for things to work properly.

Yes. That's correct. Python's super() was designed to support
cooperative multiple inheritance. The "cooperative" part means that
every class implementing the target method (such as __init__ in your
example) needs to call super() in order to trigger the next method in
the sequence (the method resolution order or MRO).

But what if SuperClass1 is from third party library? . . .
How to deal with that?

Then, the odds are that that class isn't "cooperating". You can
either wrap the third-party library to add a super() call or you can
switch to an alternate design using composition instead of
inheritance.


Raymond


P.S. Outside of the simple case of single inheritance, the one key to
understanding super() is to forget about the concept of parent
classes. Instead, super() is all about the MRO which is computed
dynamically (unknowable at the time a class is written). Every class
in the MRO implementing the target method *must* call super() to give
the next class in the MRO a chance to run.

IOW, using super() means "I'm in the MRO and I got a chance to run;
now the next class in the MRO gets its chance." Since the MRO is
only knowable at runtime, the sole job of super() is to figure out
which is "the next class in the MRO".
 
G

Gregory Ewing

Raymond said:
Every class
in the MRO implementing the target method *must* call super() to give
the next class in the MRO a chance to run.

EXCEPT for the last one, which must NOT call super!

The posted example happens to work because object has
a default __init__ method that does nothing. But this
is not generally true of other methods, which means you
need a "terminating" class at the end of the MRO whose
methods don't call super.
 
E

Ethan Furman

Gregory said:
EXCEPT for the last one, which must NOT call super!

The posted example happens to work because object has
a default __init__ method that does nothing. But this
is not generally true of other methods, which means you
need a "terminating" class at the end of the MRO whose
methods don't call super.

Speaking of new-style classes only, don't they all end in object? And
if the MRO is only known at run-time, how is one to know at code-time
whether your (new-style) class is at the end of the line?

~Ethan~
 
R

Raymond Hettinger

[Ethan Furman]
Speaking of new-style classes only, don't they all end in object?  And
if the MRO is only known at run-time, how is one to know at code-time
whether your (new-style) class is at the end of the line?

That is a bit of a PITA. One way of handling it is to design your
diamond so that only one class inherits from object and that class
doesn't use super(). Or you can wrap the super call in a try/except
AttributeError.

Cooperative multiple inheritance isn't pretty -- which is just another
good reason to use composition rather that inheritance.


Raymond
 
J

Jean-Michel Pichavant

Steven said:
That incorrect. You can certainly use super() with classic classes in the
hierarchy, and super() didn't even exist when they were created.



... def method(self, arg):
... return repr(arg)
...

... def method(self, arg):
... return "argument was %s" % Parent.method(self, arg)
...

... def method(self, arg):
... return super(New, self).method(arg).upper()
...

"ARGUMENT WAS 'SPAM'"


The problem isn't super(), and people who give glib advise "don't use
super()" are just compounding the problem. The problem is with multiple
inheritance where methods have different method signatures. If you don't
change the method signatures, super() will work fine.

Advising people not to use super() might make people feel virtuous, but
it doesn't do anything to help the reader write non-buggy MI hierarchies.
It pushes the effort of dealing with multiple inheritance onto them,
forcing them to re-implement the MRO, probably badly. Can you re-
implement the C3 algorithm? Have you even heard of it? If you answer No
to either of those questions, chances are high that trying to deal with
the MRO manually will lead to worse bugs than using super().

Should you use super()?

1. If you're doing multiple inheritance with metaclasses, you MUST use
super().

2. If you're using single inheritance only, and never modify method
signatures, there is no reason not to use super().

3. If you're using mixins, you should use super().

4. If you never modify method signatures, then you should use super().

5. If you do modify method signatures, you shouldn't do that (except
possibly in constructors, and even there only cautiously). But if you do
it anyway, then you should use super() *except* in the methods where you
modify the signature.

6. If you don't use super(), chances are that your class hierarchy is
still buggy, but instead of failing loudly and noisily with an exception,
it's silently giving the wrong results.

7. If you can avoid multiple inheritance in favour of another technique
(such as composition), you should strongly consider that.
I think you're missing the point that super for most of us is a
dangerous guess game. It makes implicit what would *really* need to be
explicit.

class Base1(object):
def foo(self):
print 'Base1'


class Base2(object):
def foo(self):
print 'Base1'


class Sub(Base1, Base2):
def foo(self):
# which base version to call ???
# choice A
# use super an pray that it will do what I need (better
know exactly how the MRO works)
# also pray that further readers know as much as I do
about super
# choice B
# use explicit calls so I can choose which algorithm I
want to use, calling Base1, Base2 or both of them
# If the choice is too difficult, that means one thing
=> my inheritance design is crap => rewrite it properly.

Super is known for being required for diamond inheritance, and this
reputation is well earned. Outside this scope, super's not required.

JM
 
S

Steven D'Aprano

Steven D'Aprano wrote:
That incorrect. You can certainly use super() with classic classes in
the hierarchy, and super() didn't even exist when they were created.
The problem isn't super(), and people who give glib advise "don't use
super()" are just compounding the problem. The problem is with multiple
inheritance where methods have different method signatures. If you
don't change the method signatures, super() will work fine.
[...]
I think you're missing the point that super for most of us is a
dangerous guess game. It makes implicit what would *really* need to be
explicit.

No, I'm not missing the point. I'm *disagreeing* with the point. I'm
saying that for simple inheritance hierarchies, super() does no harm, and
it is *required* for most complex inheritance hierarchies. Given that,
why not use it?

I suppose that, if you're inheriting from an existing class which doesn't
use super(), there may be issues (although I'm not convinced that those
issues go away if you avoid super!) but for a new class under your
control, there are no negatives to super() that didn't already exist with
the nature of multiple inheritance.

There is, as far as I can tell, ONE use-case where super() must be
avoided, and that is a use-case that is discouraged for other reasons:
where you change the method signature. If you're changing the method
signature, you're already in dubious territory and better know what
you're doing. You're violating the Liskov Substitution Principle, and if
so, you better have a good reason. (I'm a bit skeptical about the LSP,
but many people much smarter than I think it's virtually gospel, so who
knows?) But either way, if you change the method signature, you're going
to have problems somewhere unless you've really careful. The problems
with super() are merely symptoms of that change.

If you don't change the method signature, then I don't believe super()
does any harm, regardless of whether you have single inheritance or
multiple inheritance, whether it is a diamond diagram or not. The worst
that can be said about it is that super()'s signature is hard to
understand.

I would argue that your argument about "explicit" and "implicit" is
incorrect. You shouldn't want to explicitly specify which class you are
inheriting from, at least not under normal circumstances. It is enough to
say that you are inheriting behaviour. If you have a Widget class, and
you find yourself explicitly calling methods by *named* superclasses,
you're probably doing something wrong.

It should always be possible to insert a class higher up the hierarchy,
or rename a grandparent, without effecting your class. (Obviously you
can't expect to rename an immediate parent class.)

super() is just as explicit as len(), or str.upper(). It says,
explicitly, that it will call the method belonging to one or more
superclass of the given class. That's no more implicit than mylist[5] is
implicit merely because you didn't have to walk the list by hand.


class Base1(object):
def foo(self):
print 'Base1'


class Base2(object):
def foo(self):
print 'Base1'


class Sub(Base1, Base2):
def foo(self):
# which base version to call ???
# choice A
# use super an pray that it will do what I need (better
know exactly how the MRO works)

But the point is, super() knows exactly how the MRO works, so you don't
have to.

# also pray that further readers know as much as I do
about super

Now that's just silly. Do you write all your classes for the lowest
common denominator? Do you assume all you users are ignorant? If you
subclass dict, do you feel the need to "pray" that your users won't try
to use mutable objects as keys?

If you avoid super(), do you "pray" that your users won't try to use your
class in a multiple inheritance situation, and have a buggy class? I bet
you don't. If they misuse your class, that's their responsibility.


# choice B
# use explicit calls so I can choose which algorithm I
want to use, calling Base1, Base2 or both of them

That's a fair point.

super() uses the standard MRO algorithm, it isn't a "Do What I Mean" mind-
reading function. If you want an unusual MRO, then you're responsible for
managing it yourself. Go right ahead, this is Python and you have all the
tools to do so. Nothing forces you to inherit behaviour from the
superclasses at all -- super() is designed for overloading (extending)
the behaviour of methods, and so the assumption is that you want to call
the method in each superclass. But if you're overriding the method
instead, or doing something unusual, then you're in charge.


# If the choice is too difficult, that means one thing
=> my inheritance design is crap => rewrite it properly.

Super is known for being required for diamond inheritance, and this
reputation is well earned. Outside this scope, super's not required.

Of course it's not required. Nor does it do any harm. Far from it -- your
single inheritance subclass might be just what another user needs for a
multiple inheritance class, and by not using super(), you ruin it for him.
 
J

Jean-Michel Pichavant

Steven said:
[snip]

super() is just as explicit as len(), or str.upper(). It says,
explicitly, that it will call the method belonging to one or more
superclass of the given class.
Come on Steven, you're better than this :) .
Everybody can accurately guess what len and upper are doing without
looking at the documentation. No one can guess for super without closely
looking at the documention, or even at some good articles on the net
which try to clear things up about super. And note there is no such
article about len or upper.

As someone already said in this list, the main problem with super is
that it tends to refer to the superclass method while in fact it calls
the next MRO method.

"mro" would have been the proper name for "super".

JM
 
S

Steven D'Aprano

Steven said:
[snip]

super() is just as explicit as len(), or str.upper(). It says,
explicitly, that it will call the method belonging to one or more
superclass of the given class.
Come on Steven, you're better than this :) . Everybody can accurately
guess what len and upper are doing without looking at the documentation.
No one can guess for super without closely looking at the documention,
or even at some good articles on the net which try to clear things up
about super. And note there is no such article about len or upper.

super() is complex because the problem it is solving is a hard problem.
That doesn't make it implicit, any more than (say) itertools.groupby() is
implicit just because it's complex, or urllib2.request() is implicit just
because some people don't know much about web protocols and have to read
articles about it to learn.

As someone already said in this list, the main problem with super is
that it tends to refer to the superclass method while in fact it calls
the next MRO method.

Why do you think that is a problem? That's what it is supposed to do,
because that's what is needed to correctly implement multiple inheritance.

"mro" would have been the proper name for "super".

That's your opinion. In any case, whether super() was called super() or
mro() or aardvark() makes no difference to the functionality or whether
it is useful.
 
G

Gregory Ewing

Steven said:
super() is just as explicit as len(), or str.upper(). It says,
explicitly, that it will call the method belonging to one or more
superclass of the given class.

That's not strictly true. It will call a method belonging to some
class in the mro of self, but that class is not necessarily in
the base list of the class mentioned in the super() call. It's
possible for a super() call to go "sideways" in the inheritance
graph.
 
G

Gregory Ewing

Steven said:
On Thu, 29 Jul 2010 19:29:24 +0200, Jean-Michel Pichavant wrote:
>

That's your opinion. In any case, whether super() was called super() or
mro() or aardvark() makes no difference to the functionality or whether
it is useful.

I think the point is that the name is misleading, because it
makes it *sound* like it's going to call a method in a superclass,
when it fact it might not.
 
J

Jean-Michel Pichavant

Steven said:
On Thu, 29 Jul 2010 19:29:24 +0200, Jean-Michel Pichavant wrote:


[snip]
As someone already said in this list, the main problem with super is
that it tends to refer to the superclass method while in fact it calls
the next MRO method.

Why do you think that is a problem? That's what it is supposed to do,
because that's what is needed to correctly implement multiple inheritance.


"mro" would have been the proper name for "super".

That's your opinion. In any case, whether super() was called super() or
mro() or aardvark() makes no difference to the functionality or whether
it is useful.
I have no problem with dogs nor cats, however I have problem with cats
called dogs and dogs called cats as I'm dealing with industrial
programming, not litterature nor poetry.

JM
 
S

Steven D'Aprano

That's not strictly true. It will call a method belonging to some class
in the mro of self, but that class is not necessarily in the base list
of the class mentioned in the super() call.

Yes, that's what I said. super() can visit any superclass of the given
class, not just one of the immediate base class(es). That's why it's
called super() rather than base() or parent(). It would be rather
pointless if super() was limited to just the base classes.

It's possible for a super()
call to go "sideways" in the inheritance graph.

I doubt that very much. A class F can't inherit behaviour from a class E
merely by virtue of them both being subclasses of the same hierarchy. If
it did, that would be... disturbing.

Example:

E inherits from D.
D inherits from C and B.
C and B both inherit from A.
F also inherits from C.

F and E are "sideways" to each other (sibling classes?), but they don't
inherit from each other.

Perhaps you're referring to the angled lines in a diagram such as:

A
/ \
C B
\ /
D
/ \
E F

F and E don't inherit from each other, because they are sidewards to each
other (they are not in each other's MRO). Regardless of the angles we
draw the lines, all of D, C, B, A are above E (and F). Or downwards if
you prefer to reverse the diagram. Yes, a super call might jog left from
C to B, but only when being called from one of the lower classes D-F.
That's still an upwards call relative to the originator, not sidewards.
 
S

Steven D'Aprano

I think the point is that the name is misleading, because it makes it
*sound* like it's going to call a method in a superclass, when it fact
it might not.

I'm not sure I understand your point here. If you call super() from a
method that doesn't exist in any superclass, then you are correct, it
won't call a method in a superclass, and will raise AttributeError.

But in the more sensible case that you only call super() when there is
actually something to inherit, then of course it calls the method in a
superclass. It certainly doesn't call methods from arbitrary unrelated
classes, only those which are in the MRO. That is, superclasses.

If Z is below A in the hierarchy, then we have no difficulty in
identifying Z as a subclass of A, and likewise we should have no problem
with identifying A as *a* (and not "the") superclass of Z, no matter how
distant they are or how tangled the DIG between them.

The exception would be if you could have loops in the class hierarchy, in
which case the concepts of super- and sub-classes breaks down completely.
But even if some other languages allowed that, Python doesn't, so we're
safe.
 
H

Hrvoje Niksic

Gregory Ewing said:
I think the point is that the name is misleading, because it makes it
*sound* like it's going to call a method in a superclass, when it fact
it might not.

That is indeed confusing to some people, especially those who refuse to
to accept the notion that "superclass" means the same as "next in MRO",
maintaining instead that superclass refers to one of the base classes,
their bases, etc. -- IMO a defensible position.

super might have better been called next_in_mro, next_method, or
next_class, except those are harder to type, and way less catchy than
"super". The Dylan and CLOS operator that super is most closely based
on is called (call-)next-method.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,483
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top