Class attributes, instances and metaclass __getattribute__

Discussion in 'Python' started by Pedro Werneck, Aug 7, 2006.

  1. Hi all


    I noticed something strange here while explaining decorators to someone.
    Not any real use code, but I think it's worth mentioning.

    When I access a class attribute, on a class with a custom metaclass with
    a __getattribute__ method, the method is used when acessing some
    attribute directly with the class object, but not when you do it from
    the instance.

    <code type='prompt'>
    >>> class M(type):

    .... def __getattribute__(cls, attr):
    .... print cls, attr
    .... return type.__getattribute__(cls, attr)
    ....
    >>> class C(object):

    .... __metaclass__ = M
    ....
    >>> C.x = 'foo'
    >>> C.x

    <class '__main__.C'> x
    'foo'
    >>> o = C()
    >>> o.x

    'foo'
    >>>

    </code>


    Someone at freenode #python channel involved with python-dev sprint
    suggested it might be a bug, worth mentioning... to me it seems like a
    decision to avoid some problems with method and descriptors creation,
    since someone using metaclasses and custom __getattribute__ at the same
    time is asking for trouble, but... I googled for it and tried to find
    something on the list but, nothing. From the source it seems like a
    generic wrapper is used. What's the real case here ?


    Regards,

    --
    Pedro Werneck
     
    Pedro Werneck, Aug 7, 2006
    #1
    1. Advertising

  2. Pedro Werneck wrote:
    > When I access a class attribute, on a class with a custom metaclass with
    > a __getattribute__ method, the method is used when acessing some
    > attribute directly with the class object, but not when you do it from
    > the instance.
    >
    > <code type='prompt'>
    > >>> class M(type):

    > ... def __getattribute__(cls, attr):
    > ... print cls, attr
    > ... return type.__getattribute__(cls, attr)
    > ...
    > >>> class C(object):

    > ... __metaclass__ = M
    > ...
    > >>> C.x = 'foo'
    > >>> C.x

    > <class '__main__.C'> x
    > 'foo'
    > >>> o = C()
    > >>> o.x

    > 'foo'
    > >>>

    > </code>
    >
    >
    > Someone at freenode #python channel involved with python-dev sprint
    > suggested it might be a bug, worth mentioning... to me it seems like a
    > decision to avoid some problems with method and descriptors creation,
    > since someone using metaclasses and custom __getattribute__ at the same
    > time is asking for trouble, but... I googled for it and tried to find
    > something on the list but, nothing. From the source it seems like a
    > generic wrapper is used. What's the real case here ?
    >
    >
    > Regards,
    >
    > --
    > Pedro Werneck


    To me, it seems consistent. As said in
    http://www-128.ibm.com/developerworks/linux/library/l-pymeta2/

    """The availability of metaclass attributes is not transitive; in other
    words, the attributes of a metaclass are available to its instances,
    but not to the instances of the instances. Just this is the main
    difference between metaclasses and superclasses."""

    Since this happens for real attributes, it looks natural that the same
    should
    happen for 'virtual' attributes implemented via '__getattr__' or
    '__getattribute__'.

    Michele Simionato
     
    Michele Simionato, Aug 8, 2006
    #2
    1. Advertising

  3. Hi

    On 8 Aug 2006 00:10:39 -0700
    "Michele Simionato" <> wrote:

    > To me, it seems consistent. As said in
    > http://www-128.ibm.com/developerworks/linux/library/l-pymeta2/
    >
    > """The availability of metaclass attributes is not transitive; in
    > other words, the attributes of a metaclass are available to its
    > instances, but not to the instances of the instances. Just this is the
    > main difference between metaclasses and superclasses."""



    Well... I'm not talking about metaclass attributes... that's perfectly
    consistent, agreed.

    I'm saying that when the class implements a custom __getattribute__,
    when you try to access the instance attributes from itself, it uses it.
    But if the class is a metaclass, instances of its instances have acess
    to the attribute anyway, but don't use the custom __getattribute__ you
    implemented.

    Like the example I mentioned on the previous mail, or (I think in this
    case it's more obvious):

    >>> class N(type):

    .... def __getattribute__(cls, attr):
    .... print 'using N.__getattribute for "%s"'%(attr)
    .... return type.__getattribute__(cls, attr)
    ....
    >>> class M(type):

    .... __metaclass__ = N
    ....
    >>> class C(object):

    .... __metaclass__ = M
    ....
    >>> M.x = 'foo'
    >>> M.x

    using N.__getattribute for "x"
    'foo'
    >>> C.x

    'foo'


    So, in both cases I have access to the class attribute; but in one case
    it's using the bound M.__getattribute__ implemented at the base
    metaclass, but not in the other. It was supposed to use it after the
    bound 'C.__getattribute__' raises AttributeError as expected, but it
    doesn't...

    It's inconsistent, but seems a reasonable decision to me since someone
    can easily mess with descriptors doing with this... but, someone else
    involved with Python dev I talked about thinks it may be a bug.

    And, I'm curious anyway... is it possible to customize attribute access
    in this case in any other way ? What really happens here ?

    >From the typeobject.c source code, seems like when the metaclass

    implements __getattribute__, it uses type.__getattribute__ in this case,
    but I'm not sure.


    > Since this happens for real attributes, it looks natural that the same
    > should happen for 'virtual' attributes implemented via '__getattr__'
    > or '__getattribute__'.


    Well... as I think it's clear now, the case you mentioned is not exactly
    what I'm talking about, but it's good you mentioned about 'virtual'
    attributes, because in this case we have another problem, it's
    inconsistent too and there's no reason for it...

    >>> class M(type):

    .... def __getattr__(cls, attr):
    .... if attr == 'x':
    .... return 'foo'
    ....
    >>> class C(object):

    .... __metaclass__ = M
    ....
    >>> C.y = 'bar'
    >>> C.x

    'foo'
    >>> C.y

    'bar'
    >>> o = C()


    >>> o.x

    ....
    AttributeError: 'C' object has no attribute 'x'
    >>> o.y

    'bar'
    >>>


    So... both 'x' and 'y' are class attributes, but 'x' is a virtual
    attribute implemented with M.__getattr__. From the instance I have
    access to 'y' but not to 'x'.



    Regards,

    --
    Pedro Werneck
     
    Pedro Werneck, Aug 8, 2006
    #3
  4. Pedro Werneck wrote:
    > Hi


    [snip]
    > Well... I'm not talking about metaclass attributes... that's perfectly
    > consistent, agreed.
    >
    > I'm saying that when the class implements a custom __getattribute__,
    > when you try to access the instance attributes from itself, it uses it.
    > But if the class is a metaclass, instances of its instances have acess
    > to the attribute anyway, but don't use the custom __getattribute__ you
    > implemented.


    Attribute lookup for instances of a class never calls metaclass'
    __getattribute__() method. This method is called only when you
    access attributes directly on the class.

    [snip]
    > And, I'm curious anyway... is it possible to customize attribute access
    > in this case in any other way ? What really happens here ?


    There are two distinct methods involved in your example; attribute
    lookup for classes is controled by metaclass' __getattribute__()
    method, while instance attribute lookup is controled by class'
    __getattribute__() method.

    They are basicaly the same, but they never use ``type(obj).attr`` to
    access the class' attributes. The code for these methods would look
    something like this in Python:

    class Object(object):
    """
    Emulates object's and type's behaviour in attribute lookup.
    """

    def __getattribute__(self, name):
    cls = type(self)

    # you normally access this as self.__dict__
    try:
    dict_descriptor = cls.__dict__['__dict__']
    except KeyError:
    # uses __slots__ without dict
    mydict = {}
    else:
    mydict = dict_descriptor.__get__(self, cls)

    # Can't use cls.name because we would get descriptors
    # (methods and similar) that are provided by class'
    # metaclass and are not meant to be accessible from
    # instances.
    classdicts = [c.__dict__ for c in cls.__mro__]

    # We have to look in class attributes first, since it can
    # be a data descriptor, in which case we have to ignore
    # the value in the instance's dict.
    for d in classdicts:
    if name in d:
    classattr = d[name]
    break
    else:
    # None of the classes provides this attribute; perform
    # the normal lookup in instance's dict.
    try:
    return mydict[name]
    except KeyError:
    # Finally if everything else failed, look for the
    # __getattr__ hook.
    for d in classdicts:
    if '__getattr__' in d:
    return d['__getattr__'](self, name)
    msg = "%r object has no attribute %r"
    raise AttributeError(msg % (cls.__name__, name))

    # Check if class' attribute is a descriptor.
    if hasattr(classattr, '__get__'):
    # If it is a non-data descriptor, then the value in
    # instance's dict takes precedence
    if not hasattr(classattr, '__set__') and name in mydict:
    return mydict[name]
    return classattr.__get__(self, cls)

    # Finally, look into instance's dict.
    return mydict.get(name, classattr)

    As you can see, it completely avoids calling metaclass'
    __getattribute__()
    method. If it wouldn't do that, then the metaclass' attributes would
    'leak' to instances of its classes. For example, __name__, __mro__
    and mro() are some of the descriptors provided by type to every class,
    but they are not accesible through instances of these classes,
    and shouldn't be, otherwise they could mask some errors in user's code.

    Ziga
     
    Ziga Seilnacht, Aug 8, 2006
    #4
  5. On 8 Aug 2006 07:24:54 -0700
    "Ziga Seilnacht" <> wrote:

    > [snip]
    > > Well... I'm not talking about metaclass attributes... that's
    > > perfectly consistent, agreed.
    > >
    > > I'm saying that when the class implements a custom __getattribute__,
    > > when you try to access the instance attributes from itself, it uses
    > > it. But if the class is a metaclass, instances of its instances have
    > > acess to the attribute anyway, but don't use the custom
    > > __getattribute__ you implemented.

    >
    > Attribute lookup for instances of a class never calls metaclass'
    > __getattribute__() method. This method is called only when you
    > access attributes directly on the class.


    Well... thanks for the answer.

    As I said on the first mail, I noticed this when I was explaining
    descriptors and methods to someone else... I implemented a pure-python
    Method class to show him exactly how it works.

    But, since their __get__ call is available at the class too, my first
    thought was that it was implemented at the metaclass __getattribute__,
    and when an instance tries to get a class attribute it would fail on its
    own __getattribute__, use the bound method at its class and make the
    call.

    After implementing a test metaclass I noticed it doesn't work this way,
    and even if it's a bit inconsistent (especially in the case of 'virtual'
    attributes), it seemed to me the reasonable thing to do, exactly for
    what you mentioned on your code... someone could easily break a lot of
    stuff doing it the wrong way, instead if not using __dict__.

    I mailed the list because someone else thought it might be a bug and I
    was in doubt... now it's clear it was the right thing to do.


    Regards,

    --
    Pedro Werneck
     
    Pedro Werneck, Aug 8, 2006
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. John Wohlbier
    Replies:
    2
    Views:
    399
    Josiah Carlson
    Feb 22, 2004
  2. ironfroggy
    Replies:
    16
    Views:
    447
    Michele Simionato
    Jun 3, 2005
  3. Amirouche B.
    Replies:
    5
    Views:
    250
    Amirouche B.
    Aug 23, 2011
  4. lars van gemerden
    Replies:
    10
    Views:
    668
    lars van gemerden
    Jan 3, 2012
  5. Steven D'Aprano

    Metaclass of a metaclass

    Steven D'Aprano, Jun 5, 2012, in forum: Python
    Replies:
    1
    Views:
    322
Loading...

Share This Page