cooperation of buitlin methods usingtsuper

  • Thread starter Matthias Oberlaender
  • Start date
M

Matthias Oberlaender

I would like to adopt the cooperation paradigm in conjunction with builtin
methods and operators, such as len, iter, +, * etc.

But the direct approach does not work with the current implementation of
super. For example, 'len(super(Y, y)' will always result in 'len() of unsized
object'.

As far as I understand, this is because builtins don't use a dynamic lookup
chain, but go directly to the slots for the builtins. However, super returns
an instance of class 'super'. Since all super objects share this class, its
slots will not be filled as one might hope.

My workaround is this: I create a subclass of 'super' on the fly each time I
call 'mysuper'. Look at the definition below. It seems to work. But maybe I
have overlooked something. Is my understanding correct? Is 'mysuper' a good
solution? Possible improvements? (e.g. caching of subclasses)

Thanks for comments!


import new

class X(object):
def __len__(self): return 2222

class Y(X):
def __len__(self): return 1111

def mysuper(cls, inst):
return new.classobj('mysuper', (super,) + cls.__bases__, {})(cls, inst)

y = Y()
try:
print len(super(Y, y))
except Exception, msg:
print msg

try:
print len(mysuper(Y, y))
except Exception, msg:
print msg

Output:

len() of unsized object
2222

--
____ __ _/_/ .
( / / ( / / / /

=====================================================================
Matthias Oberlaender, DaimlerChrysler AG, Research Center Ulm
RIC/AP (Machine Perception)
Wilhelm-Runge-Str. 11, P.O. Box 2360, 89013 Ulm, Germany
Phone: +49 731 505 2354 Fax: +49 731 505 4105
Email: (e-mail address removed)
=====================================================================
 
M

Michele Simionato

Matthias Oberlaender said:
methods and operators, such as len, iter, +, * etc.
Fine.

But the direct approach does not work with the current implementation of
super. For example, 'len(super(Y, y)' will always result in 'len() of
unsized object'.

Of course, it must give an error! I think you do not understand how
super works. But don't worry, that's quite common ;)
As far as I understand, this is because builtins don't use a dynamic lookup
chain, but go directly to the slots for the builtins. However, super returns
an instance of class 'super'. Since all super objects share this class, its
slots will not be filled as one might hope.
My workaround is this: I create a subclass of 'super' on the fly each time I
call 'mysuper'. Look at the definition below. It seems to work. But maybe
I have overlooked something. Is my understanding correct? Is 'mysuper' a
good solution? Possible improvements? (e.g. caching of subclasses)
Thanks for comments!

import new
class X(object):
def __len__(self): return 2222
class Y(X):
def __len__(self): return 1111
def mysuper(cls, inst):
return new.classobj('mysuper', (super,) + cls.__bases__, {})(cls, inst)
y = Y()
try:
print len(super(Y, y))
except Exception, msg:
print msg
try:
print len(mysuper(Y, y))
except Exception, msg:
print msg

len() of unsized object
2222

I think you should re-read the documentation and
google on the newsgroup for 'super'. The use case
for super is in multiple inheritance, as in this example:

class B(object):
def __len__(self):
print 'called B.__len__'
return 1111

class C(B):
def __len__(self):
print 'called C.__len__'
return super(C,self).__len__()

class D(B):
def __len__(self):
print 'called D.__len__'
return super(D,self).__len__()

class E(C,D):
pass

print len(E())

The output of this is

called C.__len__
called D.__len__
called B.__len__
1111

Do you see why? Feel free to ask if not. I do not understand what behavior
you expect from 'super'. Can you give more details on your specific use case?

P.S. BTW, in Python 2.2+ you can replace ``new.classobj(name,bases,dic)``
with the built-in ``type(name,bases,dic)``.

--
____ __ _/_/ .
( / / ( / / / /


Michele
 
B

Bengt Richter

Uhm... I do realize now that what I wrote sounds quite presumptuous
indeed.
It was not my intention. The "of course" refers to the current
implementation
of ``super`` which does not do what you ask for. To me this was well
known
because of recent threads on the subject by Bjorn Pettersen:

http://groups.google.com/[email protected]#link1

http://groups.google.com/groups?hl=...per+pettersen&meta=group%3Dcomp.lang.python.*

You see that for sure you are not the only one who is confused about
``super`` and there are dark corners about it. I myself do not know
nothing about its
implementation.


I see now what's your point, which is the same of Pettersen: why

1111

works, whereas

Traceback (most recent call last):
File "<pyshell#7>", line 1, in ?
len(super(C,c))
TypeError: len() of unsized object

does not work? As you say, the reason is that


BTW, the same is true for Python2.3b2. I always use the longest form
of ``super``,so this problems does not bother me, nevertheless I
understand
your point.

I think you should submit the issue to python-dev; maybe there are
technical reasons such that it is necessary to treat special methods
differently and this cannot be avoided. In such a case the
documentation should report that
only the long form ``super(C,c).__len__()`` is correct and that users
should
not use ``len(super(C,c))`` (idem for other special methods).

I would also submit a bug report on sourceforge, since at the best
this is
a documentation bug. It does not seem to have been reported before and
has
already bitten at least two persons on c.l.py., therefore it should be
made
known to the developers
ISTM (and I don't know super much) that in the above, if you factor out
x = super(C,c)
and then compare
len(x)
vs
x.__len__()

then the results above suggest that implementation is not
getattr(x,'__len__')()
but effectively more like
getattr(x.__class__,'__len__')(x)

Note:
... def __len__(self): return 1111
... Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: len() of unsized object

Ok, now let's look at c vs x:
1111

vs
Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: type object 'super' has no attribute '__len__'

I guess generally continuing a failed method lookup in x.__class__.__dict__ or equivalent
and trying to find it as an instance attribute might make this super problem
work, but weren't instance methods intentionally bypassed for the new style?

That would mean solving the problem specifically in super somehow, but how? I guess by
fiddling with lookup of methods of/via super instances? I guess it could be done in C.
I'll have to look in typeobject.c more sometime. It looks already tricky ;-)

Regards,
Bengt Richter
 
M

Michele Simionato

ISTM (and I don't know super much) that in the above, if you factor out
x = super(C,c)
and then compare
len(x)
vs
x.__len__()

then the results above suggest that implementation is not
getattr(x,'__len__')()
but effectively more like
getattr(x.__class__,'__len__')(x)

Note:

... def __len__(self): return 1111
...
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: len() of unsized object

Ok, now let's look at c vs x:

Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: type object 'super' has no attribute '__len__'

I guess generally continuing a failed method lookup in x.__class__.__dict__ or equivalent
and trying to find it as an instance attribute might make this super problem
work, but weren't instance methods intentionally bypassed for the new style?

That would mean solving the problem specifically in super somehow, but how? I guess by
fiddling with lookup of methods of/via super instances? I guess it could be done in C.
I'll have to look in typeobject.c more sometime. It looks already tricky ;-)

Regards,
Bengt Richter

To add trickyness to trickyness, I show you a couple of examples where
the same problems appear. These examples (or examples similar to them)
where pointed out to me by Bjorn Pettersen. The real problem seems to be
in the lookup rule for len(x) which is not always equivalent to
x.__len__() when tricks with __getattr__ are performed:

First example: defining __len__ on the object does not work

class E(object):
def __getattr__(self,name):
if name=='__len__': return lambda:0
Traceback ...

Second example: defining __len__ on the class does not work:

class M(type):
def __getattr__(self,name):
if name=='__len__': return lambda self:0

class F: __metaclass__=M
f.__len__() # AttributeError: 'F' object has no attribute '__len__'

As you see, the problem is that len(x) is not calling x.__len__(),
nor x.__class__.__len__(x); I really would like to know how ``len``
(or ``str``, ``repr``, etc.) work: I think this is the core of
the problem, whereas ``super`` is probably a red herring.

Michele
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,483
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top