cooperation of buitlin methods usingtsuper

Discussion in 'Python' started by Matthias Oberlaender, Jul 2, 2003.

  1. I would like to adopt the cooperation paradigm in conjunction with builtin
    methods and operators, such as len, iter, +, * etc.

    But the direct approach does not work with the current implementation of
    super. For example, 'len(super(Y, y)' will always result in 'len() of unsized
    object'.

    As far as I understand, this is because builtins don't use a dynamic lookup
    chain, but go directly to the slots for the builtins. However, super returns
    an instance of class 'super'. Since all super objects share this class, its
    slots will not be filled as one might hope.

    My workaround is this: I create a subclass of 'super' on the fly each time I
    call 'mysuper'. Look at the definition below. It seems to work. But maybe I
    have overlooked something. Is my understanding correct? Is 'mysuper' a good
    solution? Possible improvements? (e.g. caching of subclasses)

    Thanks for comments!


    import new

    class X(object):
    def __len__(self): return 2222

    class Y(X):
    def __len__(self): return 1111

    def mysuper(cls, inst):
    return new.classobj('mysuper', (super,) + cls.__bases__, {})(cls, inst)

    y = Y()
    try:
    print len(super(Y, y))
    except Exception, msg:
    print msg

    try:
    print len(mysuper(Y, y))
    except Exception, msg:
    print msg

    Output:

    len() of unsized object
    2222

    --
    ____ __ _/_/ .
    ( / / ( / / / /

    =====================================================================
    Matthias Oberlaender, DaimlerChrysler AG, Research Center Ulm
    RIC/AP (Machine Perception)
    Wilhelm-Runge-Str. 11, P.O. Box 2360, 89013 Ulm, Germany
    Phone: +49 731 505 2354 Fax: +49 731 505 4105
    Email:
    =====================================================================
    Matthias Oberlaender, Jul 2, 2003
    #1
    1. Advertising

  2. Matthias Oberlaender <> wrote in message news:<bduh36$rgn$-felb.debis.de>...> I would like to adopt the cooperation paradigm in conjunction with builtin
    > methods and operators, such as len, iter, +, * etc.


    Fine.

    > But the direct approach does not work with the current implementation of
    > super. For example, 'len(super(Y, y)' will always result in 'len() of
    > unsized object'.


    Of course, it must give an error! I think you do not understand how
    super works. But don't worry, that's quite common ;)

    > As far as I understand, this is because builtins don't use a dynamic lookup
    > chain, but go directly to the slots for the builtins. However, super returns
    > an instance of class 'super'. Since all super objects share this class, its
    > slots will not be filled as one might hope.


    > My workaround is this: I create a subclass of 'super' on the fly each time I
    > call 'mysuper'. Look at the definition below. It seems to work. But maybe
    > I have overlooked something. Is my understanding correct? Is 'mysuper' a
    > good solution? Possible improvements? (e.g. caching of subclasses)


    > Thanks for comments!



    > import new


    > class X(object):
    > def __len__(self): return 2222


    > class Y(X):
    > def __len__(self): return 1111


    > def mysuper(cls, inst):
    > return new.classobj('mysuper', (super,) + cls.__bases__, {})(cls, inst)


    > y = Y()
    > try:
    > print len(super(Y, y))
    > except Exception, msg:
    > print msg


    > try:
    > print len(mysuper(Y, y))
    > except Exception, msg:
    > print msg


    >Output:


    > len() of unsized object
    > 2222


    I think you should re-read the documentation and
    google on the newsgroup for 'super'. The use case
    for super is in multiple inheritance, as in this example:

    class B(object):
    def __len__(self):
    print 'called B.__len__'
    return 1111

    class C(B):
    def __len__(self):
    print 'called C.__len__'
    return super(C,self).__len__()

    class D(B):
    def __len__(self):
    print 'called D.__len__'
    return super(D,self).__len__()

    class E(C,D):
    pass

    print len(E())

    The output of this is

    called C.__len__
    called D.__len__
    called B.__len__
    1111

    Do you see why? Feel free to ask if not. I do not understand what behavior
    you expect from 'super'. Can you give more details on your specific use case?

    P.S. BTW, in Python 2.2+ you can replace ``new.classobj(name,bases,dic)``
    with the built-in ``type(name,bases,dic)``.

    --
    ____ __ _/_/ .
    ( / / ( / / / /


    Michele
    Michele Simionato, Jul 2, 2003
    #2
    1. Advertising

  3. On 3 Jul 2003 07:52:44 -0700, (Michele Simionato) wrote:

    >> In <> Michele Simionato wrote:
    >> > Matthias Oberlaender <>

    >> wrote in message news:<bduh36$rgn$-felb.debis.de>...> I would like
    >> to adopt the cooperation paradigm in conjunction with builtin
    >> > > methods and operators, such as len, iter, +, * etc.
    >> >
    >> > Fine.
    >> >
    >> > > But the direct approach does not work with the current implementation of
    >> > > super. For example, 'len(super(Y, y)' will always result in 'len() of
    >> > > unsized object'.
    >> >
    >> > Of course, it must give an error! I think you do not understand how
    >> > super works. But don't worry, that's quite common ;)

    >>
    >> Oh, wait a minute. I think this "of course" is a bit presumptuous.

    >
    >Uhm... I do realize now that what I wrote sounds quite presumptuous
    >indeed.
    >It was not my intention. The "of course" refers to the current
    >implementation
    >of ``super`` which does not do what you ask for. To me this was well
    >known
    >because of recent threads on the subject by Bjorn Pettersen:
    >
    >http://groups.google.com/groups?hl=#link1
    >
    >http://groups.google.com/groups?hl=...per+pettersen&meta=group%3Dcomp.lang.python.*
    >
    >You see that for sure you are not the only one who is confused about
    >``super`` and there are dark corners about it. I myself do not know
    >nothing about its
    >implementation.
    >
    >> >
    >> > I think you should re-read the documentation and
    >> > google on the newsgroup for 'super'. The use case
    >> > for super is in multiple inheritance, as in this example:
    >> >
    >> > class B(object):
    >> > def __len__(self):
    >> > print 'called B.__len__' return 1111
    >> >
    >> > class C(B):
    >> > def __len__(self):
    >> > print 'called C.__len__'
    >> > return super(C,self).__len__()

    >>
    >> This is exactly the style of syntax I want to avoid! I'd rather write more
    >> concisely 'len(super(C,self))'.

    >
    >I see now what's your point, which is the same of Pettersen: why
    >
    >>>> class B(object):

    > def __len__(self): return 1111
    >>>> class C(B): pass

    >...
    >>>> c=C()
    >>>> super(C,c).__len__()

    >1111
    >
    >works, whereas
    >
    >>>> len(super(C,c))

    >Traceback (most recent call last):
    > File "<pyshell#7>", line 1, in ?
    > len(super(C,c))
    >TypeError: len() of unsized object
    >
    >does not work? As you say, the reason is that
    >
    >> at least in Python 2.2.1, its is
    >> a matter of fact that builtins/special method names are treated differently
    >> from ordinary class methods in conjunction with super objects. Do you agree?
    >> My implementation of super seems to fix this gap. Perhaps there are some
    >> other people who would appreciate that too. Or are there good
    >> logical/conceptual reasons againts it? This is what I would like to know.

    >
    >BTW, the same is true for Python2.3b2. I always use the longest form
    >of ``super``,so this problems does not bother me, nevertheless I
    >understand
    >your point.
    >
    >I think you should submit the issue to python-dev; maybe there are
    >technical reasons such that it is necessary to treat special methods
    >differently and this cannot be avoided. In such a case the
    >documentation should report that
    >only the long form ``super(C,c).__len__()`` is correct and that users
    >should
    >not use ``len(super(C,c))`` (idem for other special methods).
    >
    >I would also submit a bug report on sourceforge, since at the best
    >this is
    >a documentation bug. It does not seem to have been reported before and
    >has
    >already bitten at least two persons on c.l.py., therefore it should be
    >made
    >known to the developers
    >

    ISTM (and I don't know super much) that in the above, if you factor out
    x = super(C,c)
    and then compare
    len(x)
    vs
    x.__len__()

    then the results above suggest that implementation is not
    getattr(x,'__len__')()
    but effectively more like
    getattr(x.__class__,'__len__')(x)

    Note:

    >>> class B(object):

    ... def __len__(self): return 1111
    ...
    >>> class C(B): pass

    ...
    >>> c=C()
    >>> super(C,c).__len__()

    1111
    >>> len(super(C,c))

    Traceback (most recent call last):
    File "<stdin>", line 1, in ?
    TypeError: len() of unsized object

    Ok, now let's look at c vs x:

    >>> getattr(c,'__len__')()

    1111
    >>> getattr(c.__class__,'__len__')(c)

    1111

    vs

    >>> getattr(x,'__len__')()

    1111
    >>> getattr(x.__class__,'__len__')(x)

    Traceback (most recent call last):
    File "<stdin>", line 1, in ?
    AttributeError: type object 'super' has no attribute '__len__'

    I guess generally continuing a failed method lookup in x.__class__.__dict__ or equivalent
    and trying to find it as an instance attribute might make this super problem
    work, but weren't instance methods intentionally bypassed for the new style?

    That would mean solving the problem specifically in super somehow, but how? I guess by
    fiddling with lookup of methods of/via super instances? I guess it could be done in C.
    I'll have to look in typeobject.c more sometime. It looks already tricky ;-)

    Regards,
    Bengt Richter
    Bengt Richter, Jul 3, 2003
    #3
  4. (Bengt Richter) wrote in message news:<be23nd$aqp$0@216.39.172.122>...
    > ISTM (and I don't know super much) that in the above, if you factor out
    > x = super(C,c)
    > and then compare
    > len(x)
    > vs
    > x.__len__()
    >
    > then the results above suggest that implementation is not
    > getattr(x,'__len__')()
    > but effectively more like
    > getattr(x.__class__,'__len__')(x)
    >
    > Note:
    >
    > >>> class B(object):

    > ... def __len__(self): return 1111
    > ...
    > >>> class C(B): pass

    > ...
    > >>> c=C()
    > >>> super(C,c).__len__()

    > 1111
    > >>> len(super(C,c))

    > Traceback (most recent call last):
    > File "<stdin>", line 1, in ?
    > TypeError: len() of unsized object
    >
    > Ok, now let's look at c vs x:
    >
    > >>> getattr(c,'__len__')()

    > 1111
    > >>> getattr(c.__class__,'__len__')(c)

    > 1111
    >
    > vs
    >
    > >>> getattr(x,'__len__')()

    > 1111
    > >>> getattr(x.__class__,'__len__')(x)

    > Traceback (most recent call last):
    > File "<stdin>", line 1, in ?
    > AttributeError: type object 'super' has no attribute '__len__'
    >
    > I guess generally continuing a failed method lookup in x.__class__.__dict__ or equivalent
    > and trying to find it as an instance attribute might make this super problem
    > work, but weren't instance methods intentionally bypassed for the new style?
    >
    > That would mean solving the problem specifically in super somehow, but how? I guess by
    > fiddling with lookup of methods of/via super instances? I guess it could be done in C.
    > I'll have to look in typeobject.c more sometime. It looks already tricky ;-)
    >
    > Regards,
    > Bengt Richter


    To add trickyness to trickyness, I show you a couple of examples where
    the same problems appear. These examples (or examples similar to them)
    where pointed out to me by Bjorn Pettersen. The real problem seems to be
    in the lookup rule for len(x) which is not always equivalent to
    x.__len__() when tricks with __getattr__ are performed:

    First example: defining __len__ on the object does not work

    class E(object):
    def __getattr__(self,name):
    if name=='__len__': return lambda:0

    >>> e=E()
    >>> e.__len__()

    0
    >>> len(e)

    Traceback ...

    Second example: defining __len__ on the class does not work:

    class M(type):
    def __getattr__(self,name):
    if name=='__len__': return lambda self:0

    class F: __metaclass__=M

    >>> f=F()
    >>> F.__len__(f) # okay

    0
    >>> len(f) # TypeError: len() of unsized object

    f.__len__() # AttributeError: 'F' object has no attribute '__len__'

    As you see, the problem is that len(x) is not calling x.__len__(),
    nor x.__class__.__len__(x); I really would like to know how ``len``
    (or ``str``, ``repr``, etc.) work: I think this is the core of
    the problem, whereas ``super`` is probably a red herring.

    Michele
    Michele Simionato, Jul 4, 2003
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Neo
    Replies:
    1
    Views:
    519
    Scott Allen
    Jan 7, 2005
  2. Giandomenico Sica

    Call for a cooperation

    Giandomenico Sica, Dec 6, 2005, in forum: Python
    Replies:
    2
    Views:
    291
  3. G. Sica

    Call for a cooperation

    G. Sica, Dec 6, 2005, in forum: Python
    Replies:
    0
    Views:
    306
    G. Sica
    Dec 6, 2005
  4. Synk

    free cad cooperation

    Synk, Mar 24, 2007, in forum: C++
    Replies:
    24
    Views:
    923
    lilburne
    Mar 30, 2007
  5. Kenneth McDonald
    Replies:
    5
    Views:
    315
    Kenneth McDonald
    Sep 26, 2008
Loading...

Share This Page