super, decorators and gettattribute

R

Richard Szopa

Hello all,

I am playing around w/ Python's object system and decorators and I
decided to write (as an exercise) a decorator that (if applied to a
method) would call the superclass' method of the same name before
doing anything (initially I wanted to do something like CLOS
[1] :before and :end methods, but that turned out to be too
difficult).

However, I cannot get it right (specially, get rid of the eval). I
suspect that I may be misunderstanding something happening between
super objects and __getattribute__ methods.

Here's my code:

def endmethod(fun):
"""Decorator to call a superclass' fun first.

If the classes child and parent are defined as below, it should
work like:
I am parent's foo
I am child's foo.
"""
name = fun.__name__
def decorated(self, *args, **kwargs):
try:
super_object = super(self.__class__, self)

# now I want to achieve something equivalent to calling
# parent.foo(*args, **kwargs)
# if I wanted to limit it only to this example

# this doesn't work: in the example, it calls child's foo,
# entering in an eternal loop (instead of calling parent's
# foo, as I would expect).

# super_object.__getattribute__(name)(*args, **kwargs)

# this does work, but I feel it's ugly
eval('super_object.%s(*args, **kwargs)' % name)
except AttributeError:
pass # if parent doesn't implement fun, we don't care
# about it
return fun(self, *args, **kwargs) # hopefully none

decorated.__name__ = name
return decorated


class parent(object):
def foo(self):
print 'I am parent\'s foo'

class child(parent):
@endmethod
def foo(self):
print "I am foo\'s foo."

if __name__=='__main__':
x = child()
x.foo()

Can anybody tell me how to call a superclass method knowing its name?

Thanks in advance,
-- Richard

[1] http://en.wikipedia.org/wiki/Common_Lisp_Object_System
 
R

Richard Szopa

doing anything (initially I wanted to do something like CLOS
[1] :before and :end methods, but that turned out to be too
difficult).
Click to expand...

Erm, I meant :before and :after methods.

-- Richard
 
M

Mike Meyer

Hello all,

I am playing around w/ Python's object system and decorators and I
decided to write (as an exercise) a decorator that (if applied to a
method) would call the superclass' method of the same name before
doing anything (initially I wanted to do something like CLOS
[1] :before and :end methods, but that turned out to be too
difficult).

However, I cannot get it right (specially, get rid of the eval). I
suspect that I may be misunderstanding something happening between
super objects and __getattribute__ methods.

Here's my code:

def endmethod(fun):
"""Decorator to call a superclass' fun first.

If the classes child and parent are defined as below, it should
work like:
I am parent's foo
I am child's foo.
"""
name = fun.__name__
def decorated(self, *args, **kwargs):
try:
super_object = super(self.__class__, self)
Click to expand...

There's an apparently common bug here: you don't want to pass super
self.__class__, but the class that the method is bound to. The two
aren't the same, as an instance of a subclass will have the subclass
as self.__class__, and not the current class. So super will return the
current class or a subclass of it, meaning (since you invoked this
method from self) you'll wind up invoking this method recursively.
All of which means your decorator is probably going to have to take
the class as an argument.
# now I want to achieve something equivalent to calling
# parent.foo(*args, **kwargs)
# if I wanted to limit it only to this example

# this doesn't work: in the example, it calls child's foo,
# entering in an eternal loop (instead of calling parent's
# foo, as I would expect).

# super_object.__getattribute__(name)(*args, **kwargs)

# this does work, but I feel it's ugly
eval('super_object.%s(*args, **kwargs)' % name)
except AttributeError:
pass # if parent doesn't implement fun, we don't care
# about it
return fun(self, *args, **kwargs) # hopefully none

decorated.__name__ = name
return decorated


class parent(object):
def foo(self):
print 'I am parent\'s foo'

class child(parent):
@endmethod
def foo(self):
print "I am foo\'s foo."

if __name__=='__main__':
x = child()
x.foo()

Can anybody tell me how to call a superclass method knowing its name?
Click to expand...

The same way you call any object's methods if you know it's name":

getattr(super_object, name)(*args, **kwargs)

The code seems to work the way you want:
I am parent's foo
I am foo's foo.

<mike
 
R

Richard Szopa

The same way you call any object's methods if you know it's name":

getattr(super_object, name)(*args, **kwargs)
Click to expand...

Thanks a lot for your answer!

However, I am very surprised to learn that

super_object.__getattr__(name)(*args, **kwargs)

getattr(super_object, name)(*args, **kwargs)

are not equivalent. This is quite odd, at least when with len()
and .__len__, str() and .__str__. Do you maybe know what's the
rationale behind not following that convention by getattr?

Best regards,

-- Richard
 
S

Steven D'Aprano

There's an apparently common bug here: you don't want to pass super
self.__class__, but the class that the method is bound to.
Click to expand...

Given an instance method, is it possible to easily determine what class
it is defined in?

I thought the im_class attribute might do it, but it apparently just
points to self.__class__.
.... def foo(self):
.... pass
........ def bar(self):
.... pass
.... <class '__main__.Bar'>
 
M

Marc 'BlackJack' Rintsch

However, I am very surprised to learn that

super_object.__getattr__(name)(*args, **kwargs)

getattr(super_object, name)(*args, **kwargs)

are not equivalent. This is quite odd, at least when with len()
and .__len__, str() and .__str__. Do you maybe know what's the
rationale behind not following that convention by getattr?
Click to expand...

I think you are confusing `__getattr__` and `__getattribute__` here!
`getattr()` maps to `__getattr__()`, it's `__getattribute__` that's
different.

Ciao,
Marc 'BlackJack' Rintsch
 
R

Richard Szopa

I think you are confusing `__getattr__` and `__getattribute__` here!
`getattr()` maps to `__getattr__()`, it's `__getattribute__` that's
different.
Click to expand...

Well, in my code calling super_object.__getattr__(name)(*args,
**kwargs) and getattr(super_object, name)(*args, **kwargs) gives
*different* effects (namely, the latter works, while the former
doesn't). That kinda suggests that they don't map to each other :).
And that makes me feel confused.

Cheers,

-- Richard
 
T

thebjorn

Well, in my code calling super_object.__getattr__(name)(*args,
**kwargs) and getattr(super_object, name)(*args, **kwargs) gives
*different* effects (namely, the latter works, while the former
doesn't). That kinda suggests that they don't map to each other :).
And that makes me feel confused.

Cheers,

-- Richard
Click to expand...

They do, except for when it comes to what super(..) returns. It isn't
really an object in the sense that they're presented in the tutorial,
but rather a sort of proxy to the methods in the ancestor classes of
the concrete object (self), relative to the current method's class. I
can't imagine that sentence would ease any confusion however, suffice
it to say that you have to call getattr(super(..), 'name') instead of
super(..).__getattr__('name') and you have to call super(..).__len__()
instead of len(super(..)) -- I can't imagine that lessens any
confusion either :-/

super(..) is designed to handle situations like this correctly

class Root(object):
n = 1

class Left(Root):
def foo(self):
print 'n =', self.n
print 'super n = ', super(Left, self).n

class Right(Root):
n = 2

class Leaf(Left,Right):
n = 3

x = Leaf()
x.foo()

the correct output is

n = 3
super n = 2


-- bjorn
 
R

Richard Szopa

They do, except for when it comes to what super(..) returns. It isn't
really an object in the sense that they're presented in the tutorial,
but rather a sort of proxy to the methods in the ancestor classes of
the concrete object (self), relative to the current method's class. I
can't imagine that sentence would ease any confusion however, suffice
it to say that you have to call getattr(super(..), 'name') instead of
super(..).__getattr__('name') and you have to call super(..).__len__()
instead of len(super(..)) -- I can't imagine that lessens any
confusion either :-/
Click to expand...

Surprisingly, I think your first sentence *does* make something more
clear. Let me check if I understand it right: when we call a method on
super(Foo, self) it is as if we were calling call-next-method in
Common Lisp or Dylan (i.e. the method of the class on the right of Foo
in self.mro()). This however does not imply for super to have its dict
the same as the class on the right of Foo---it remains the same as
self's dict.

However, there's one piece that doesn't completely fit to the puzzle:
why does getattr work? The help says:

getattr(...)
getattr(object, name[, default]) -> value

Get a named attribute from an object; getattr(x, 'y') is
equivalent to x.y.
When a default argument is given, it is returned when the
attribute doesn't
exist; without it, an exception is raised in that case.

Does it work on the basis that "getattr(x, 'y') is equivalent to x.y"?
What is then a "named attribute for an object" in Python? It seems not
to be equivalent to the value of the item whose name is 'y' in the
object's class __dict__...

Cheers,

-- Richard
 
M

Michele Simionato

However, there's one piece that doesn't completely fit to the puzzle:
why does getattr work? The help says:

getattr(...)
getattr(object, name[, default]) -> value

Get a named attribute from an object; getattr(x, 'y') is
equivalent to x.y.
When a default argument is given, it is returned when the
attribute doesn't
exist; without it, an exception is raised in that case.

Does it work on the basis that "getattr(x, 'y') is equivalent to x.y"?
What is then a "named attribute for an object" in Python? It seems not
to be equivalent to the value of the item whose name is 'y' in the
object's class __dict__...

Cheers,

-- Richard
Click to expand...

I really need to publish this one day or another, since these
questions
about super keeps coming out:

http://www.phyast.pitt.edu/~micheles/python/super.html
 
G

George Sakkis

Given an instance method, is it possible to easily determine what class
it is defined in?

I thought the im_class attribute might do it, but it apparently just
points to self.__class__.


... def foo(self):
... pass
...>>> class Bar(Foo):

... def bar(self):
... pass
...>>> Bar().bar.im_class # expecting Bar

<class '__main__.Bar'>>>> Bar().foo.im_class # hoping for Foo

<class '__main__.Bar'>
Click to expand...

Something like that seems to work for most cases:

from inspect import getmro

def getdef(obj,attr):
try: objattrs = obj.__dict__
except AttributeError:
objattrs = obj.__slots__
if attr in objattrs:
return obj
for cls in getmro(obj.__class__):
if attr in cls.__dict__:
return cls
<class '__main__.Foo'>


It probably misses some edge cases but I can't think of any off the
top of my head.

George
 
T

thebjorn

Surprisingly, I think your first sentence *does* make something more
clear. Let me check if I understand it right: when we call a method on
super(Foo, self) it is as if we were calling call-next-method in
Common Lisp or Dylan
Click to expand...

I don't remember if CLOS was changed to use C3 Linearization also, but
the concept came from Dylan (http://www.webcom.com/haahr/dylan/
linearization-oopsla96.html) and that's what is implemented in Python.

[...]
However, there's one piece that doesn't completely fit to the puzzle:
why does getattr work? The help says:

getattr(...)
getattr(object, name[, default]) -> value

Get a named attribute from an object; getattr(x, 'y') is
equivalent to x.y. When a default argument is given, it
is returned when the attribute doesn't exist; without it,
an exception is raised in that case.

Does it work on the basis that "getattr(x, 'y') is equivalent to x.y"?
What is then a "named attribute for an object" in Python? It seems not
to be equivalent to the value of the item whose name is 'y' in the
object's class __dict__...
Click to expand...

Conceptually, x.y is always "get the y attribute of x" and the same as
getattr(x, 'y'). Depending on the type of x and y, and your
familiarity with Python internals, what actually happens during a
lookup might be surprising. In the vast majority of cases however, x.y
is equivalent to one of

x.__dict__['y']
type(x).__dict__['y']

but if you're a language geek like me, you might be excited that in
some cases it is

type(x).__dict__['y'].__get__(x, type(x))

which says you get the value of x.y by calling y and passing x as an
argument -- if you know CLOS you'll recognize that it's a primitive
multi-method call. (there are some other special cases too, although
not as exciting ;-)

Much more detail can be found in Raymond's paper on descriptors
(http://users.rcn.com/python/download/Descriptor.htm) and Michele's
paper on super (http://www.phyast.pitt.edu/~micheles/python/
super.html).

-- bjorn
 
R

Richard Szopa

I really need to publish this one day or another, since these
questions
about super keeps coming out:

http://www.phyast.pitt.edu/~micheles/python/super.html
Click to expand...

Thanks, Michele! Your essay was enlightening [2]. Specially if you
take in account super's documentation is slightly outdated :).

I also read Raymond Hettinger's article about descriptors (which you
mention, but don't link to!) and decided to use them to reimplement
methods that always call their superclass [1] method of the same name.
Could you tell me what are the pros and cons of the two approaches
(i.e. writing a decorator function and a decorator descriptor class)?
The former gives slightly shorter code, while the second gives access
to the original function and I somehow feel it is more... classy :)
[code follows]

Cheers,

-- Richard

[1] By which I mean the first class in their MRO.
[2] I also found some other papers of yours about the "not for the
faint of heart" corners of Python: MRO, metaclasses, class
memoization, etc. and they were most enjoyable and interesting.

def callingprevious1(fun):
"""Decorator to call a superclass' fun first. Decorator function
approach.
... def foo(self):
... print "I am foo of parent"
... ... @callingprevious1
... def foo(self):
... print "I am foo of child"
... I am foo of parent
I am foo of child I am foo of parent
I am foo of child
"""

name = fun.__name__
def decorated(self, *args, **kwargs):
try:
super_object = super(self.__class__, self)
getattr(super_object, name)(*args, **kwargs)
except AttributeError:
pass # if parent doesn't implement fun, we don't care
# about it
return fun(self, *args, **kwargs) # hopefully None

decorated.__name__ = name
return decorated

class callingprevious(object):
"""
Decorator making the defined method call the method of the first
superclass in mro at the beginning. Descriptor approach.
... def foo(self):
... print "I am foo of parent"
... ... @callingprevious
... def foo(self):
... print "I am foo of child"
... I am foo of parent
I am foo of child I am foo of parent
I am foo of child
"""

def __init__(self, initval):
self.name = initval.__name__
self.__combine_methods(initval)

def __combine_methods(self, val):
self.val = val
def with_parent(obj, *args, **kwargs):
try:
parent_method = getattr(super(type(obj), obj),
self.val.__name__)
except AttributeError:
pass
else:
parent_method(*args, **kwargs)
return self.val(obj, *args, **kwargs)
with_parent.__name__ = self.val.__name__
self.to_return = with_parent

def __get__(self, obj, objtype):
from types import MethodType

# btw, is it anyhow better than just returning the function?
return MethodType(self.to_return, obj, objtype)

def __set__(self, obj, val):
self.__combine_methods(val)
 
R

Richard Szopa

I don't remember if CLOS was changed to use C3 Linearization also, but
the concept came from Dylan (http://www.webcom.com/haahr/dylan/
linearization-oopsla96.html) and that's what is implemented in Python.
Click to expand...

The Common Lisp ANSI standard is from 1994, and the article you cite
is from 1996, which strongly suggests C3 linearization wasn't included
in CLOS. The most important difference between CLOS and C3
linearization AFAIK is that the latter enforces monotonicity, while
the former doesn't.

Of course, it shouldn't be very difficult to implement the C3 behavior
in Common Lisp using the de facto standard MetaObject Protocol.

(Nb. Dylan was such a nice language... It's a pity it is practically
dead right now.)
but if you're a language geek like me, you might be excited that in
some cases it is

type(x).__dict__['y'].__get__(x, type(x))

which says you get the value of x.y by calling y and passing x as an
argument -- if you know CLOS you'll recognize that it's a primitive
multi-method call. (there are some other special cases too, although
not as exciting ;-)
Click to expand...

Yeah, I also feel the excitement, so probably I am a language geek
too ;-). However, this is still quite far away from full fledged
multimethods. (OTOH trying to get something more from these primitive
multimethods by abusing __get__ looks kind of tempting ;-))

Regards,

-- Richard
 
M

Michele Simionato

Could you tell me what are the pros and cons of the two approaches
(i.e. writing a decorator function and a decorator descriptor class)?
Click to expand...

I prefer to use a class for introspection sake, since
there is no way to get information about an inner function
in a closure, whereas your users can introspect classes
pretty well.
super_object = super(self.__class__, self)
Click to expand...

Notice that using super(self.__class__, self) is a common
mistake: the pitfalls with inheritance were discussed
very recently in this same newsgroup, you should be
able to find the post. What you need is

super(class_where_the_method_is_defined, self)

which is in general different from

super(self.__class__, self)

There is no clean way to determine the current class
in Python < 3.0 (Python 3.0 does it automatically);
if you want to see a hackish way involving
bytecode tricks see
http://groups.google.com/group/comp...hl=en&lnk=gst&q=currentClass#62a2da68961caeb6

(and notice that this is really not recommended).

Michele Simionato
 
R

Rhamphoryncus

Well, in my code calling super_object.__getattr__(name)(*args,
**kwargs) and getattr(super_object, name)(*args, **kwargs) gives
*different* effects (namely, the latter works, while the former
doesn't). That kinda suggests that they don't map to each other :).
And that makes me feel confused.
Click to expand...

Don't think of them as mappings. Think of them as a way for a class
to hook into getattr's protocol, conveniently named similar to getattr
(__getattr__ and __getattribute__). getattr() may call several
methods, no methods at all, change the arguments, etc. Although len()
may seem simple, many others are not so simple.
 
R

Richard Szopa

Unfortunately the links [2], [3] and [4] are not given,
Click to expand...

Luckily, there's Google :)

[2] Article about MRO: http://www.python.org/download/releases/2.3/mro/
[3] Descriptor HowTo: http://users.rcn.com/python/download/Descriptor.htm
[4] Articles about metaclasses (second one relevant to super):
* http://www.ibm.com/developerworks/linux/library/l-pymeta.html,
* http://www-128.ibm.com/developerworks/linux/library/l-pymeta2/?S_TACT=105AGX03&S_CMP=ART,
* http://www.ibm.com/developerworks/linux/library/l-pymeta3.html?S_TACT=105AGX03&S_CMP=ART

Of course, it would be very nice if the article was updated to include
the links.

HTH,

-- Richard
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads

confusion with decorators 10
decorators and mangled names for "private" methods 0
Getting lazy with decorators 8
doctests and decorators 5
Signature-preserving decorators 1
Confusion about decorators 6
Decorators, with optional arguments 3
Descriptors and decorators 3

Members online

Total: 42 (members: 2, guests: 40)
Robots: 214

Forum statistics

Threads
473,767
Messages
2,569,570
Members
45,045
Latest member
DRCM

Latest Threads

Top