using super

[iu2]

Hi

I'm trying to make a method call automatically to its super using this
syntax:

class A:
chained = ['pr']
def pr(self):
print 'Hello from A'

class B(A):
def pr(self):
print 'Hello from B'

chain(B, A)
b = B()
b.pr()

b.pr() will print
Hello from B
Hello from A

I'm doing it using the 'chained' attribute in class A, and with this
function:

def chain(cls, sup):
for m in dir(cls):
if callable(getattr(cls, m)) and m in cls.chained:
cm = getattr(cls, m)
def m2(*p):
cm(*p)
return getattr(sup, m)(*p)
setattr(cls, m, m2)
return cls

which seeks for all 'chained' methods and adjusts them accordingly.
(had there been class decorators the syntax would have been simpler,
something like

class A:
@make_chained
def pr():
print 'Hello from A'

@chained
class B:
def pr():
print 'Hello from B'
)


My problem is this: Currently I pass the base class to 'chain' -
chain(B, A)

I prefer to write
chain(B)

and let 'chain' use the super of B.

So:
def chain(cls):
for m in dir(cls):
if callable(getattr(cls, m)) and m in cls.chained:
print 'chaning', cls, m
cm = getattr(cls, m)
def m2(*p):
cm(*p)
return getattr(super(cls), m)(*p)
setattr(cls, m, m2)

This is probably wrong because I don't give the object instance to
super (I don't have it!) and I also get the error
TypeError: super() argument 1 must be type, not classobj

Can you please help me with this?

Thanks
iu2

 

[Steven D'Aprano]

Hi

I'm trying to make a method call automatically to its super using this
syntax:

[snip code]


I'm not sure if this is your only problem or not, but super() only works
with new-style classes, not with classic classes. You must inherit from
object, or it cannot possibly work.

Change "class A" to "class A(object)".

However, I suspect your approach may be too complicated. Try this:


def chain(meth): # A decorator for calling super.
def f(self, *args, **kwargs):
result = meth(self, *args, **kwargs)
S = super(self.__class__, self)
getattr(S, meth.__name__)(*args, **kwargs)
return result
f.__name__ = "chained_" + meth.__name__
return f



class A(object):
def foo(self, x):
print "I am %s" % self
return x

class B(A):
@chain
def foo(self, x):
print "This is B!!!"
return x + 1


This is B!!!
I am <__main__.B object at 0xb7cf68ac>
6

 

[Scott David Daniels]

...
I'm not sure if this is your only problem or not, but super() only works
with new-style classes, not with classic classes. You must inherit from
object, or it cannot possibly work.

Change "class A" to "class A(object)".

Absolutely correct.

However, the suggested simpler code cannot work on any released Python:

def chain(meth): # A decorator for calling super.
def f(self, *args, **kwargs):
result = meth(self, *args, **kwargs)
S = super(self.__class__, self)

This line is the problem. The class parameter needs to be the class
name (B in this case) in which the chaining method is defined, not that
of the object itself.

getattr(S, meth.__name__)(*args, **kwargs)
return result
f.__name__ = "chained_" + meth.__name__
return f


class A(object):
def foo(self, x):
print "I am %s" % self
return x

class B(A):
@chain
def foo(self, x):
print "This is B!!!"
return x + 1

You'll see the problem once you figure out what goes wrong with:
class C(B):
@chain
def foo(self, x):
print "This is C!!!"
return x + 2

C().foo(5)


As to the original idea, better to give it up.
Typically code for a "chained" method "foo" that
returns a result will want to (in some way) use
the result from that call in forming its result.
Python's super allows you to place that call where
it belongs in your code (perhaps after some logging
is enabled, for example) and not just "at the spot
the guru insists the chaining happens." The recursion-
like call to the super method is placed explicitly
in Python so you can see how it works. If super is
too tough to explain, I expect single inheritance is
all you are using. simply remind people to call
A.foo(self, <args>) within the definition of foo in
B.

--Scott David Daniels
(e-mail address removed)

 

[Gabriel Genellina]

En Mon, 31 Dec 2007 12:08:43 -0200, Steven D'Aprano

def chain(meth): # A decorator for calling super.
def f(self, *args, **kwargs):
result = meth(self, *args, **kwargs)
S = super(self.__class__, self)
getattr(S, meth.__name__)(*args, **kwargs)
return result
f.__name__ = "chained_" + meth.__name__
return f



class A(object):
def foo(self, x):
print "I am %s" % self
return x

class B(A):
@chain
def foo(self, x):
print "This is B!!!"
return x + 1

If you later inherit from B and try to @chain a method, nasty things
happen... The problem is that the two arguments to super (current class,
and actual instance) are *both* required; you can't fake the first using
self.__class__. But you can't refer to the current class inside the
decorator, because it doesn't exist yet. You could use the decorator to
just "mark" the function to be chained, and then -with the help of a
metaclass- do the actual decoration when the class is created.

def chain(meth):
"""Mark a method to be "chained" later"""
meth.chained = True
return meth

def chain_impl(cls, meth):
"""The original decorator by SD'A"""
def f(self, *args, **kwargs):
result = meth(self, *args, **kwargs)
S = super(cls, self)
getattr(S, meth.__name__)(*args, **kwargs)
return result
f.__name__ = "chained_" + meth.__name__
return f

class ChainedType(type):
def __new__(meta, name, bases, dic):
cls = super(ChainedType, meta).__new__(meta, name, bases, dic)
# replace functions marked "to be chained" with its decorated
version
for name, value in dic.iteritems():
if getattr(value, 'chained', False):
setattr(cls, name, chain_impl(cls, value))
return cls

class A(object):
__metaclass__ = ChainedType
def foo(self, x):
print "I am %s" % self
return x

class B(A):
@chain
def foo(self, x):
print "This is B!!!"
return x + 1

class C(B):
@chain
def foo(self, x):
print "This is C!!!"
return x + 2

py> a = A()
py> a.foo(5)
I am <__main__.A object at 0x00A3C690>
5
py> b = B()
py> b.foo(5)
This is B!!!
I am <__main__.B object at 0x00A3CA90>
6
py> c = C()
py> c.foo(5)
This is C!!!
This is B!!!
I am <__main__.C object at 0x00A3C830>
7

The approach above tries to stay close to the chain decorator as
originally posted. There are other ways that you can search for in the
Python Cookbook.

 

[Steven D'Aprano]

Absolutely correct.

However, the suggested simpler code cannot work on any released Python:

This line is the problem. The class parameter needs to be the class
name (B in this case) in which the chaining method is defined, not that
of the object itself.

One minor correction: the class parameter needs to be the class *itself*,
not the class *name* (which would be the string "B").

I don't quite understand your description though. What do you mean "the
chaining method is defined"? chain() is defined outside of a class.


[snip]

You'll see the problem once you figure out what goes wrong with:
class C(B):
@chain
def foo(self, x):
print "This is C!!!"
return x + 2

C().foo(5)

Hmmm... obviously I did insufficient testing. That's certainly a problem.

 

[Scott David Daniels]

One minor correction: the class parameter needs to be the class *itself*,
not the class *name* (which would be the string "B").

Point taken.

I don't quite understand your description though. What do you mean "the
chaining method is defined"? chain() is defined outside of a class.

The class where f (the chaining method) is defined; equivalently, the
class in which the @chain is used.

-Scott

 

[Steven D'Aprano]

Point taken.


The class where f (the chaining method) is defined; equivalently, the
class in which the @chain is used.

So why doesn't self.__class__ work? That's the class in which @chain is
used.

I can clearly see that it doesn't work, I just don't understand why. I'd
be inclined to chalk it up to super() being a mysterious black box that
makes no sense *wink* except that the following decorator also doesn't
work:


def chain(meth): # A decorator for not calling super.
def f(self, *args, **kwargs):
result = meth(self, *args, **kwargs)
S = self.__class__.__base__
getattr(S, meth.__name__)(self, *args, **kwargs)
return result
return f

 

[Scott David Daniels]

So why doesn't self.__class__ work? That's the class in which @chain is
used.

OK, here's a simple 3-class example:

class A(object):
def meth(self): print 'A.meth:', self.__class__, '---'
def pn(self): return '<A>'

class B(A):
def meth(self):
super(B, self).meth()
print 'B.meth:', self.__class__, super(B, self).pn()
def pn(self): return '<B>'

class C(B):
def meth(self):
super(C, self).meth()
print 'C.meth:', self.__class__, super(C, self).pn()
def pn(self): return '<C>'

c = C()
c.meth()
# Figure out why it printed what it did.

# If not clear yet, how about this:
for class_ in C, B:
print class_.__name__, super(class_, c).pn()

# And a bigger example (re-using A) to show why we
class B0(A):
def meth(self):
super(B0, self).meth()
print 'B0.meth:', self.__class__, super(B0, self).pn()
def pn(self): return '<B0>'

class B1(B0):
def meth(self):
super(B1, self).meth()
print 'B1.meth:', self.__class__, super(B1, self).pn()
def pn(self): return '<B1>'

class B2(B0):
def meth(self):
super(B2, self).meth()
print 'B2.meth:', self.__class__, super(B2, self).pn()
def pn(self): return '<B2>'

class C1(B1, B2):
def meth(self):
super(C1, self).meth()
print 'C1.meth:', self.__class__, super(C1, self).pn()
def pn(self): return '<C1>'

class D1(C1):
def meth(self):
super(D1, self).meth()
print 'D1.meth:', self.__class__, super(D1, self).pn()
def pn(self): return '<D1>'

d = D1()
d.meth()
# Figure out why it printed what it did.

for class_ in D1, C1, B1, B2, B0:
print class_.__name__, super(class_, d).pn()
# Now (after much cogitation) might that do it?

# finally, just a fillip, predict this before you run it:
class E(D1, C):
def meth(self):
super(E, self).meth()
print 'E.meth:', self.__class__, super(E, self).pn()
def pn(self): return '<E>'

e = E()
e.meth()
for class_ in E, D1, C1, B1, B2, B0, C, B:
print class_.__name__, super(class_, e).pn()

I can clearly see that it doesn't work, I just don't understand why. I'd
be inclined to chalk it up to super() being a mysterious black box that
makes no sense *wink* ....

super (and mro) work to get to all the superclasses in an order that
produces subtypes before their supertypes. The diamond inheritance
examples "show" why its needed.

-Scott

 

[iu2]

As to the original idea, better to give it up.
Typically code for a "chained" method "foo" that
returns a result will want to (in some way) use
the result from that call in forming its result.
Python's super allows you to place that call where
it belongs in your code (perhaps after some logging
is enabled, for example) and not just "at the spot
the guru insists the chaining happens."  

Indeed I might want to chain methods in all sort of ways:
@chain_call_before - call the parent's method before the derived
method
@chain_call_after - call the parent's method after the derived method
@chain_call_sum - sum the result of the parent's method with the
result of the derived method
@chain_call_list - make a list from the result of the parent's method
and the result of the derived method

An API for easily adding chaining methods is also something to think
about.
This kind of chaining actually exists in Common Lisp (CLOS) and I wish
for something like that in Python
so I gave it a(n unsuccessful) try.

I think that especially Python, which is known (among other things)
for its object oriented properties,
should support this kind of chaining.

simply remind people to call
A.foo(self, <args>) within the definition of foo in
B.

Sorry, I can't agree to this (although there is nothing else I can
do.. . Reminding is not "simply" at all. Why REMIND people do
stuff and not let Python handle it automatically? What's Python (and
all other powerful languages) for if not for this kind of job?
(Although I think this "chaining" is not supperted by none of them
except for Lisp).

Although I actually work in a C++ team, and all the Python I do is for
myself so I don't have to remind anyone, I think this mechansim is
necessary.
What actually made me intereseted in that is a C++ application I write
where I figured that
chaining would help.

If I translate it to Python, it's something like this:
(I'm still not familiar with new style classes so please forgive me..)

class Inetrface_system:
def get_name(self):
return "No_name"

get_name is an interface function which the programmer calls in the
application.
This is a new system:
class System_A(Interface_system):
def get_name(self):
return "System_A"

There are several dereived classes like System_A.
I need to log each call to get_name for any class, but I just can't
rely on remembering
(by progammers) to add this extra themselves for get_name for each
class they derive:

class Inetrface_system:
def get_name(self):
log_to_file('get_name called')
return "No_name"

class System_A(Interface_system):
def get_name(self):
Interface_system.get_name() # can be forgotten
return "System_A"

One solution is:

class Interface_system():
def get_name_api(self):
log_to_file('get name called')
return get_name()
def get_name(self):
return 'No name'

class System_A(Interface_system):
def get_name(self):
return 'System_A'

So programmers need only override get_name. But this makes it
confusing: One has to override
get_name, but must call get_name_api throughout the application. I
don't like it.
So, recalling CLOS I thought of a better way:


class Inetrface_system:
@cained_call_before
def get_name(self):
log_to_file('get_name called')

@chained
class System_A(Interface_system):
def get_name(self):
return 'System_A'

Now the programmer both overrides get_name, and calls get_name
directly in the application. And
no one has to remember to call the parent's get_name method. It's done
automatically.

(A PEP maybe? I sense you don't really like it.. ;-)
What do you think? Good? Too implicit?

iu2

 

[Michele Simionato]

no one has to remember to call the parent's get_name method. It's done
automatically.

(A PEP maybe? I sense you don't really like it.. ;-)
What do you think? Good? Too implicit?

iu2

No PEP, this would never pass. There would be no way
to stop a method from calling its parent: you would
lose control of your classes, so I think this is a
bad idea. Having said so, it is easy to implement
what you want with a metaclass:

def callParent(*methodnames):
class Meta(type):
def __init__(cls, name, bases, dic):
for methodname in methodnames:
if methodname in dic:
def new_meth(self, method=methodname):
parent = getattr(super(cls, self), method,
None)
if parent: parent()
child = dic.get(method)
if child: child(self)
setattr(cls, methodname, new_meth)
return Meta

class B(object):
__metaclass__ = callParent('get_name')
def get_name(self):
print "This is B.get_name"

class C(B):
def get_name(self):
print "This is C.get_name"


C().get_name()


Now every subclass of B defining a get_name method
will automagically call its parent method.

Michele Simionato

 

[iu2]

No PEP, this would never pass. There would be no way
to stop a method from calling its parent: you would
lose control of your classes, so I think this is a
bad idea.

Not all classes, only classes the programmer chooses to have this
behaviour.

Having said so, it is easy to implement
what you want with a metaclass:

def callParent(*methodnames):
     class Meta(type):
         def __init__(cls, name, bases, dic):

...
Thanks

 

[Steven D'Aprano]

Indeed I might want to chain methods in all sort of ways:
@chain_call_before - call the parent's method before the derived method
@chain_call_after - call the parent's method after the derived method
@chain_call_sum - sum the result of the parent's method with the result
of the derived method
@chain_call_list - make a list from the result of the parent's method
and the result of the derived method

Indeed there are all sorts of ways you might want to call the parent
class' method. How about a

@chain_call__multiply_by_seven_and_append_to_list

decorator?


[snip]

Sorry, I can't agree to this (although there is nothing else I can do..
. Reminding is not "simply" at all. Why REMIND people do stuff and not
let Python handle it automatically?

But Python can't handle it automatically, not unless you do

import read_my_mind_and_do_what_I_want

first. (Module expected in Python 9.7, due out in 2058.)

Since you have to call the superclass yourself, there's not that much
difference between reminding people to call the superclass by hand, and
reminding them to call some decorator.

Admittedly, a decorator for the most common cases would be easier to use
than super(), but I don't think it would make enough of a difference that
it is worth changing Python's object model just to make it possible. As
the Zen of Python says:

"Special cases aren't special enough to break the rules."

(or in this case, to change the rules).

And as others have pointed out, if you really want this desperately
enough, you can create a metaclass to handle it.

 

[iu2]

import read_my_mind_and_do_what_I_want

first. (Module expected in Python 9.7, due out in 2058.)

That's because it seems to you like some esoteric feature.
To me it seems natural with OO.

"Special cases aren't special enough to break the rules."

I thought about this mechanism each time I had to call a base class
method.
Lisp has it, out of the box.
May be it's not so special...

Since you have to call the superclass yourself, there's not that much
difference between reminding people to call the superclass by hand, and
reminding them to call some decorator.

Not necessarily. Once a class is declared with chains, its
deriveratives don't really need the decorator (Python will know about
it).
And even with a decorator, what if the class has 5 methods? That's one
decorator against 5 superclass calls.

Admittedly, a decorator for the most common cases would be easier to use
than super(), but I don't think it would make enough of a difference that
it is worth changing Python's object model just to make it possible. As
...
And as others have pointed out, if you really want this desperately
enough, you can create a metaclass to handle it.

That's right.

 

[Scott David Daniels]

That's because it seems to you like some esoteric feature.
To me it seems natural with OO.

And the language should not bend to the sense of "natural" of one with
a minor commitment to building that language. If this really is a
great idea, implement it, put it out, support it and respond to the
rigors of real use. You'll not convince me, but you needn't. Too
many people want to go to a PEP (or even skip that step) once they have
an idea they think might be good.

The lesson I take from the histories of Ada and C++ are that "the
less in the language the better." It is not enough that a language
be chock-a-block with good ideas; each idea in the language must be
compelling enough to counter-balance the mass that adding that idea
adds.

I thought about this mechanism each time I had to call a base class
method. Lisp has it, out of the box. May be it's not so special...

We accept this seems natural to you. You don't seem to understand why
others might not think so. I fear this is the kind of thing that
separates programmers into two classes: the smart ones that can set up
the chains, and the others to whom you say "don't worry your pretty
little head about it; it all happens auto-magically." Python is
remarkably free of such things, and I am loathe to give that up.
The value reduced auto-magic is that the users of the language
can learn smoothly, rather than climbing steep mountains of
understanding. Since you are relatively new to the language (I
presume so, since you left self out of several method definitions),
see how the language works on its terms first, before suggesting
how to make it more like some other language you like.

The explicit super call inside a method, and the disposition of
its results makes experimentation with different ways to use the
result tempting. There is a temptation to play with the arguments
as well, that runs afoul of the vagaries of multiple inheritance,
that is the reason that I said "if they don't want to understand
super, then just explicitly name the superclass."

--Scott David Daniels
(e-mail address removed)

 

[iu2]

We accept this seems natural to you. You don't seem to understand why
others might not think so. I fear this is the kind of thing that
separates programmers into two classes: the smart ones that can set up
the chains, and the others to whom you say "don't worry your pretty
little head about it; it all happens auto-magically."

Well, that was exactly my original post, see above. I didn't know
about new style classes.
People here responded with helpful information. Since I can program
the feature I want, I've got no complaints.
(I realize that popping PEPs pop the nerves, so, forget about my
suggestion...

little head about it; it all happens auto-magically."  Python is
remarkably free of such things, and I am loathe to give that up.

isn't
@deco_somthing
def my_func():
code..

a magic?

isn't

with file('file.txt') as f:
code..

a magic?


And what about an 'yield' in a middle of a loop? Or dynamic types (to
someont coming from C for example)...

I think Python is full of magics (and I like it very much).

Once you get used to a magic, it ceases being a magic. It becomes
automation. It does thing for you (with your control over it) instead
of you doing these things. Well, I don't even need to say this,
because that's what Python is all about right? Otherwise I would use C+
+..

I don't at all think chaining (or whatever this is actually called) is
magic. But I agree that it is not possible to integrate everything
every one wants into the language.

The value reduced auto-magic is that the users of the language
can learn smoothly, rather than climbing steep mountains of
understanding.  Since you are relatively new to the language (I
presume so, since you left self out of several method definitions),
see how the language works on its terms first, before suggesting
how to make it more like some other language you like.

I wrote the examples quickly to show what I mean, did'nt check them,
sorry.
I must say this:
I actually use Python for 5 years, I've written many many small helper
scripts and several not-too-big applications with it and I was very
pleased. I think it's a great language.
I also wrote in tcl (quite a bit), perl, and Lisp (less) but Python is
the one that I enjoy the most. That's why I feel free to say what I
think is missing (and if you don't like it, I'll try not to do it, I
think Python's developers are doing a great work).

I missed new style classes though... Now I know how to use them (not
fully), but I must say it has been difficult. I'll appreciate some
good documentation about them.

Thanks
iu2

 

[Basilisk96]

I missed new style classes though... Now I know how to use them (not
fully), but I must say it has been difficult. I'll appreciate some
good documentation about them.

Thanks
iu2

This is a decent start:
http://www.python.org/doc/newstyle/

Cheers,
-Basilisk96

 

[Basilisk96]

OK, here's a simple 3-class example:

class A(object):
def meth(self): print 'A.meth:', self.__class__, '---'
def pn(self): return '<A>'

class B(A):
def meth(self):
super(B, self).meth()
print 'B.meth:', self.__class__, super(B, self).pn()
def pn(self): return '<B>'

class C(B):
def meth(self):
super(C, self).meth()
print 'C.meth:', self.__class__, super(C, self).pn()
def pn(self): return '<C>'

c = C()
c.meth()
# Figure out why it printed what it did.

# If not clear yet, how about this:
for class_ in C, B:
print class_.__name__, super(class_, c).pn()

# And a bigger example (re-using A) to show why we
class B0(A):
def meth(self):
super(B0, self).meth()
print 'B0.meth:', self.__class__, super(B0, self).pn()
def pn(self): return '<B0>'

class B1(B0):
def meth(self):
super(B1, self).meth()
print 'B1.meth:', self.__class__, super(B1, self).pn()
def pn(self): return '<B1>'

class B2(B0):
def meth(self):
super(B2, self).meth()
print 'B2.meth:', self.__class__, super(B2, self).pn()
def pn(self): return '<B2>'

class C1(B1, B2):
def meth(self):
super(C1, self).meth()
print 'C1.meth:', self.__class__, super(C1, self).pn()
def pn(self): return '<C1>'

class D1(C1):
def meth(self):
super(D1, self).meth()
print 'D1.meth:', self.__class__, super(D1, self).pn()
def pn(self): return '<D1>'

d = D1()
d.meth()
# Figure out why it printed what it did.

for class_ in D1, C1, B1, B2, B0:
print class_.__name__, super(class_, d).pn()
# Now (after much cogitation) might that do it?

# finally, just a fillip, predict this before you run it:
class E(D1, C):
def meth(self):
super(E, self).meth()
print 'E.meth:', self.__class__, super(E, self).pn()
def pn(self): return '<E>'

e = E()
e.meth()
for class_ in E, D1, C1, B1, B2, B0, C, B:
print class_.__name__, super(class_, e).pn()


super (and mro) work to get to all the superclasses in an order that
produces subtypes before their supertypes. The diamond inheritance
examples "show" why its needed.

-Scott

Cool, thanks for posting this example and clearing that up. Several
times in the past I have used super(self.__class__, cls) instead of
super(Klass_obj, cls), without a clue that it would wreck the
subclasses. My beginner's thought at the time was that it would
provide more flexibility.. Good thing I haven't had to subclass
them..yet

Cheers,
-Basilisk96