Slight metaclass confusion

Discussion in 'Python' started by ben@transversal.com, Sep 9, 2003.

  1. Guest

    I am slightly confused about the way metaclasses work, having read
    "Metaclass Programming In Python, Parts 1 and 2"

    I get the fact that the instance of a metaclass is a class, but in
    this case I fail to see why the following does'nt work:

    >>> class Meta(type):

    .... def __str__(cls):
    .... print "I am " + repr(cls)
    ....
    >>>
    >>> Class = Meta("Fish", (), {})
    >>> Class.__str__()

    Traceback (most recent call last):
    File "<stdin>", line 1, in ?
    TypeError: descriptor '__str__' of 'object' object needs an argument

    Whereas this does:

    >>> class Simple(object):

    .... def __str__(self):
    .... return "I am " + repr(self)
    ....
    >>> obj = Simple()
    >>> obj.__str__()

    I am <__main__.Simple object at 0x402f676c>

    The tutorial does mention this (slightly), but doesn't make it clear
    why this is happening. I am probably missing something obvious though!

    Thanks for the help,

    Ben
    ---
     
    , Sep 9, 2003
    #1
    1. Advertising

  2. wrote:

    > I am slightly confused about the way metaclasses work, having read
    > "Metaclass Programming In Python, Parts 1 and 2"
    >
    > I get the fact that the instance of a metaclass is a class, but in
    > this case I fail to see why the following does'nt work:
    >
    >>>> class Meta(type):

    > ... def __str__(cls):
    > ... print "I am " + repr(cls)
    > ...
    >>>>
    >>>> Class = Meta("Fish", (), {})
    >>>> Class.__str__()

    > Traceback (most recent call last):
    > File "<stdin>", line 1, in ?
    > TypeError: descriptor '__str__' of 'object' object needs an argument
    >
    > Whereas this does:
    >
    >>>> class Simple(object):

    > ... def __str__(self):
    > ... return "I am " + repr(self)
    > ...
    >>>> obj = Simple()
    >>>> obj.__str__()

    > I am <__main__.Simple object at 0x402f676c>
    >
    > The tutorial does mention this (slightly), but doesn't make it clear
    > why this is happening. I am probably missing something obvious though!


    The problem is perhaps the terminology (Unfortunatelly,I don't know a better
    one).

    The metaclass initializes a class. Just imagine, that the textual class
    definition comes fresh from the parser, so you have information about
    possible base classes, methods, attributes. These information can be
    changed within the metaclass __new__ method.
    Think of the metaclass as a factory that turns a class definition into a
    class instance.
    Other than that, they don't interact.

    What a metaclass can do, is to change that information, before the class is
    made into a class object.

    Basicly, the metaclass concept is orthogonal to the inheritence concept of
    OOP.

    >
    > Thanks for the help,
    >
    > Ben
    > ---


    Hope that helps

    Stephan
     
    Stephan Diehl, Sep 9, 2003
    #2
    1. Advertising

  3. Ben Young Guest

    On Tue, 9 Sep 2003, Stephan Diehl wrote:

    > wrote:
    >
    > > I am slightly confused about the way metaclasses work, having read
    > > "Metaclass Programming In Python, Parts 1 and 2"
    > >

    [snip]
    >
    > The problem is perhaps the terminology (Unfortunatelly,I don't know a better
    > one).
    >
    > The metaclass initializes a class. Just imagine, that the textual class
    > definition comes fresh from the parser, so you have information about
    > possible base classes, methods, attributes. These information can be
    > changed within the metaclass __new__ method.
    > Think of the metaclass as a factory that turns a class definition into a
    > class instance.
    > Other than that, they don't interact.
    >
    > What a metaclass can do, is to change that information, before the class is
    > made into a class object.
    >
    > Basicly, the metaclass concept is orthogonal to the inheritence concept of
    > OOP.
    >


    So the way it is implemented in python a class is NOT an instance of a
    metaclass the same way that an instance is an instance of a class. This is
    slightly confusing as this is not the way it is described in all the
    places I can find that talk about python metaclasses.

    In this case, what is the point of using a class when a simple function
    can do the job?

    > Hope that helps
    >
    > Stephan
    >
    >


    Thanks for the reply

    Ben
    ---
     
    Ben Young, Sep 9, 2003
    #3
  4. Ben Young wrote:


    > [snip]
    >>
    >> The problem is perhaps the terminology (Unfortunatelly,I don't know a
    >> better one).
    >>
    >> The metaclass initializes a class. Just imagine, that the textual class
    >> definition comes fresh from the parser, so you have information about
    >> possible base classes, methods, attributes. These information can be
    >> changed within the metaclass __new__ method.
    >> Think of the metaclass as a factory that turns a class definition into a
    >> class instance.
    >> Other than that, they don't interact.
    >>
    >> What a metaclass can do, is to change that information, before the class
    >> is made into a class object.
    >>
    >> Basicly, the metaclass concept is orthogonal to the inheritence concept
    >> of OOP.
    >>

    >
    > So the way it is implemented in python a class is NOT an instance of a
    > metaclass the same way that an instance is an instance of a class. This is
    > slightly confusing as this is not the way it is described in all the
    > places I can find that talk about python metaclasses.


    For all I know, a class is the instance of its metaclass, but this doesn't
    mean that an object a of class A can access methods of A' (the metaclass of
    A). (Please note that this is probably totally wrong. Other people might
    shed light on this more theoretical question)

    >
    > In this case, what is the point of using a class when a simple function
    > can do the job?


    Well, actually, there is probably nothing you can do with metaclasses that
    you couldn't do in other ways.

    The point (if you are not a language lawyer) is that the resulting code is
    (at least in some cases) better readable when using metaclasses.

    The Python cookbook (http://aspn.activestate.com/ASPN/Cookbook/Python) has a
    couple of recipies about metaclasses. Please have a look there if you are
    interested in practical things you can do with metaclasses.

    >
    >> Hope that helps
    >>
    >> Stephan
    >>
    >>

    >
    > Thanks for the reply
    >
    > Ben
    > ---
     
    Stephan Diehl, Sep 9, 2003
    #4
  5. Hans Nowak Guest

    wrote:
    > I am slightly confused about the way metaclasses work, having read
    > "Metaclass Programming In Python, Parts 1 and 2"
    >
    > I get the fact that the instance of a metaclass is a class, but in
    > this case I fail to see why the following does'nt work:


    I'm not 100% sure, but:

    >>>>class Meta(type):

    >
    > ... def __str__(cls):
    > ... print "I am " + repr(cls)
    > ...
    >
    >>>>Class = Meta("Fish", (), {})
    >>>>Class.__str__()

    > Traceback (most recent call last):
    > File "<stdin>", line 1, in ?
    > TypeError: descriptor '__str__' of 'object' object needs an argument


    ....here, you're calling __str__ on a *class*...

    >>>>class Simple(object):

    >
    > ... def __str__(self):
    > ... return "I am " + repr(self)
    > ...
    >
    >>>>obj = Simple()
    >>>>obj.__str__()

    >
    > I am <__main__.Simple object at 0x402f676c>


    ....and here, you're calling it on an *instance*. The reason it doesn't work
    probably has nothing to do with your metaclass, as demonstrated by this:

    >>> class Foo(object): pass

    ....
    >>> Foo.__str__()

    Traceback (most recent call last):
    File "<input>", line 1, in ?
    TypeError: descriptor '__str__' of 'object' object needs an argument
    >>> f = Foo()
    >>> f.__str__()

    '<__main__.Foo object at 0x0162C750>'

    I'm not sure how it all works internally... using str(Foo) works, but
    Foo.__str__() doesn't. Notice that Foo.__str__(Foo) does work.

    I don't think there's anything wrong with your metaclass (though see below ;-),
    just don't call __str__() directly on the class.

    On a side note, it's a bad idea to use 'print' in __str__ or __repr__, even for
    informal test code. I tried the code above in PyCrust, and it broke the
    automatic attribute lookup. This is not really important, but it was a
    surprising side effect.

    HTH,

    --
    Hans ()
    http://zephyrfalcon.org/
     
    Hans Nowak, Sep 9, 2003
    #5
  6. David Mertz Guest

    () wrote previously:
    |"Metaclass Programming In Python, Parts 1 and 2"
    |I get the fact that the instance of a metaclass is a class, but in
    |this case I fail to see why the following does'nt work:

    Read part 2 again. It takes a while to sink in.

    Your problem is that you aren't considering the MRO of Class:

    >>> class Meta(type):

    ... def __str__(cls): # Use 'return', not 'print'
    ... return "I am " + repr(cls)
    ... def foo(cls):
    ... return "I am " + repr(cls)
    ...
    >>> Class = Meta("Fish", (), {})
    >>> Class.foo()

    "I am <class '__main__.Fish'>"

    Class is an instance of Meta, just like you expect. Calling the .foo()
    method shows this. But when you call a method, the method name is
    checked in the -method resolution order- (before the metaclass is
    checked):

    >>> Class.mro()

    [<class '__main__.Fish'>, <type 'object'>]

    Fish doesn't define a method .__str__(), so Python looks in Fish's
    parent, object. Well, object -does- define, .__str__(), so your call
    resolves to:

    >>> object.__str__()

    Traceback (most recent call last):
    File "<stdin>", line 1, in ?
    TypeError: descriptor '__str__' of 'object' object needs an argument

    Of course, since object doesn't define a .foo(), resolution proceeds to
    the metaclass. If you want to get Class' metaclass method, you can let
    Python do the magic for you:

    >>> str(Class)

    "I am <class '__main__.Fish'>"

    Or if you want to be really explicit:

    >>> Class.__class__.__str__(Class)

    "I am <class '__main__.Fish'>"

    Yours, David...

    --
    _/_/_/ THIS MESSAGE WAS BROUGHT TO YOU BY: Postmodern Enterprises _/_/_/
    _/_/ ~~~~~~~~~~~~~~~~~~~~[]~~~~~~~~~~~~~~~~~~~~~ _/_/
    _/_/ The opinions expressed here must be those of my employer... _/_/
    _/_/_/_/_/_/_/_/_/_/ Surely you don't think that *I* believe them! _/_/
     
    David Mertz, Sep 9, 2003
    #6
  7. On 9 Sep 2003 03:59:14 -0700, () wrote:

    >I am slightly confused about the way metaclasses work, having read
    >"Metaclass Programming In Python, Parts 1 and 2"
    >
    >I get the fact that the instance of a metaclass is a class, but in
    >this case I fail to see why the following does'nt work:
    >
    >>>> class Meta(type):

    >... def __str__(cls):
    >... print "I am " + repr(cls)
    >...

    UIAM, you subclassed type, but you didn't override __new__, so ...
    >>>>
    >>>> Class = Meta("Fish", (), {})

    became essentially a pass-through call to type, i.e., effectively the same as
    Class = type("Fish",(),{})
    and since you passed it an empty dict to make its Class.__dict__ from,

    >>>> Class.__str__()

    went looking in Class.__dict__ for '__str__' and couldn't find it. So it
    went looking for it according to Class.mro(), which led to finding it in
    the object base class -- i.e., object.__dict__['__str__'] succeeded. But what it
    got was an unbound method, because neither of two things (accessing an ordinary
    method as an attribute of an instance object, or accessing a class method as an attribute
    of either and instance or its class) was happening to bind to the method. So you got a complaint.
    Note that it's complaining about __str__ of object, not __str__ of your class Meta.

    >Traceback (most recent call last):
    > File "<stdin>", line 1, in ?
    >TypeError: descriptor '__str__' of 'object' object needs an argument
    >

    Note that your def __str__(cls) *does* exist in Meta.__dict__, but that wasn't the
    dict used to construct Class.

    >Whereas this does:
    >
    >>>> class Simple(object):

    >... def __str__(self):
    >... return "I am " + repr(self)
    >...

    In this case you are not overriding __new__ either, but the search for __new__ is finding
    object.__new__, and that is making the object instance for you, of subtype Simple.

    >>>> obj = Simple()

    Actually, I think this amounts to Simple.__call__(), which must call cls.__new__() and go
    down the mro chain to find object.__new__ and call it with cls (which would be Simple).

    >>>> obj.__str__()

    Looking for __str__ in this case is looking in type(obj).__dict__ first and finding it right there.
    You are triggering it via attribute access, which dynamically creates the bound method that supplies
    the self arg. Alternatively, type(obj).__str__ should be the unbound method, to which you could pass
    obj, and type(obj).__dict__['__str__'] should be the plain function that becomes a method by the
    dynamic magic. It can be called with obj as arg like the unbound method.

    >I am <__main__.Simple object at 0x402f676c>
    >
    >The tutorial does mention this (slightly), but doesn't make it clear
    >why this is happening. I am probably missing something obvious though!
    >
    >Thanks for the help,
    >

    To get the effect you originally wanted, one way could be:

    >>> class Meta(type):

    ... def __str__(cls): return 'I am '+ repr(cls)
    ... __str__ = classmethod(__str__)
    ... def __new__(cls, name): return type.__new__(cls, name, (),cls.__dict__.copy())
    ...
    >>> Class = Meta('Fish')
    >>> Class

    <class '__main__.Fish'>
    >>> Class.__str__()

    "I am <class '__main__.Fish'>"

    Interestingly,

    >>> str(Class)

    "I am <class '__main__.Meta'>"

    apparently the str builtin doing str(x) goes after type(x).__str__, e.g.,

    >>> Class.__str__

    <bound method Meta.__str__ of <class '__main__.Fish'>>
    >>> Class.__str__()

    "I am <class '__main__.Fish'>"

    vs

    >>> type(Class).__str__

    <bound method type.__str__ of <class '__main__.Meta'>>
    >>> type(Class).__str__()

    "I am <class '__main__.Meta'>"


    The above is my current view of how things work. I think/hope it's pretty close, but I am not
    speaking from a code walk here, so there may be some details awry. Corrections welcome.

    BTW, the __metaclass__ class variable and how that works is something that belongs in
    a discussion of Python metaclass programming, but someone else can fill that in ;-)

    HTH and does not mislead.

    Regards,
    Bengt Richter
     
    Bengt Richter, Sep 9, 2003
    #7
  8. On 9 Sep 2003 18:56:56 GMT, (Bengt Richter) wrote:

    >On 9 Sep 2003 03:59:14 -0700, () wrote:
    >
    >>I am slightly confused about the way metaclasses work, having read
    >>"Metaclass Programming In Python, Parts 1 and 2"
    >>
    >>I get the fact that the instance of a metaclass is a class, but in
    >>this case I fail to see why the following does'nt work:
    >>
    >>>>> class Meta(type):

    >>... def __str__(cls):
    >>... print "I am " + repr(cls)
    >>...

    >UIAM, you subclassed type, but you didn't override __new__, so ...

    ARGH. I was was partly mistaken, obviously.
    >>>>>
    >>>>> Class = Meta("Fish", (), {})

    >became essentially a pass-through call to type, i.e., effectively the same as

    bzzt. wrong ;-/ It would (I think) have called type.__new__(Meta, 'Fish', (), {})

    > Class = type("Fish",(),{})
    >and since you passed it an empty dict to make its Class.__dict__ from,
    >

    The empty dict did become the Class.__dict__ initial value though, I believe.

    >>>>> Class.__str__()

    >went looking in Class.__dict__ for '__str__' and couldn't find it. So it

    still true
    >went looking for it according to Class.mro(), which led to finding it in
    >the object base class -- i.e., object.__dict__['__str__'] succeeded. But what it
    >got was an unbound method, because neither of two things (accessing an ordinary
    >method as an attribute of an instance object, or accessing a class method as an attribute
    >of either and instance or its class) was happening to bind to the method. So you got a complaint.
    >Note that it's complaining about __str__ of object, not __str__ of your class Meta.
    >
    >>Traceback (most recent call last):
    >> File "<stdin>", line 1, in ?
    >>TypeError: descriptor '__str__' of 'object' object needs an argument
    >>

    >Note that your def __str__(cls) *does* exist in Meta.__dict__, but that wasn't the
    >dict used to construct Class.

    True, but maybe misleading.
    >
    >>Whereas this does:
    >>
    >>>>> class Simple(object):

    >>... def __str__(self):
    >>... return "I am " + repr(self)
    >>...

    >In this case you are not overriding __new__ either, but the search for __new__ is finding
    >object.__new__, and that is making the object instance for you, of subtype Simple.
    >
    >>>>> obj = Simple()

    >Actually, I think this amounts to Simple.__call__(), which must call cls.__new__() and go
    >down the mro chain to find object.__new__ and call it with cls (which would be Simple).
    >

    I need to read code for a while, and not write stuff I have to qualify...

    >>>>> obj.__str__()

    >Looking for __str__ in this case is looking in type(obj).__dict__ first and finding it right there.
    >You are triggering it via attribute access, which dynamically creates the bound method that supplies
    >the self arg. Alternatively, type(obj).__str__ should be the unbound method, to which you could pass
    >obj, and type(obj).__dict__['__str__'] should be the plain function that becomes a method by the
    >dynamic magic. It can be called with obj as arg like the unbound method.
    >
    >>I am <__main__.Simple object at 0x402f676c>
    >>
    >>The tutorial does mention this (slightly), but doesn't make it clear
    >>why this is happening. I am probably missing something obvious though!
    >>
    >>Thanks for the help,
    >>

    >To get the effect you originally wanted, one way could be:
    >
    > >>> class Meta(type):

    > ... def __str__(cls): return 'I am '+ repr(cls)
    > ... __str__ = classmethod(__str__)
    > ... def __new__(cls, name): return type.__new__(cls, name, (),cls.__dict__.copy())
    > ...
    > >>> Class = Meta('Fish')
    > >>> Class

    > <class '__main__.Fish'>
    > >>> Class.__str__()

    > "I am <class '__main__.Fish'>"
    >
    >Interestingly,
    >
    > >>> str(Class)

    > "I am <class '__main__.Meta'>"
    >
    >apparently the str builtin doing str(x) goes after type(x).__str__, e.g.,
    >
    > >>> Class.__str__

    > <bound method Meta.__str__ of <class '__main__.Fish'>>
    > >>> Class.__str__()

    > "I am <class '__main__.Fish'>"
    >
    >vs
    >
    > >>> type(Class).__str__

    > <bound method type.__str__ of <class '__main__.Meta'>>
    > >>> type(Class).__str__()

    > "I am <class '__main__.Meta'>"
    >
    >
    >The above is my current view of how things work. I think/hope it's pretty close, but I am not
    >speaking from a code walk here, so there may be some details awry. Corrections welcome.
    >

    I see David Mertz's post now. I learned that going off the end of the mro search for foo
    then goes on to the metaclass, which presumably means that __str__ would have been found
    if object.__str__ hadn't snagged it first in the mro chain.

    >BTW, the __metaclass__ class variable and how that works is something that belongs in
    >a discussion of Python metaclass programming, but someone else can fill that in ;-)
    >
    >HTH and does not mislead.

    Sorry about the pass-through-to-type mislead. That was plain wrong, along with the unstated
    implication that Meta.__str__ was plain unreachable from the resulting Class. At least I qualified
    it with UIAM ;-/

    Whoa, I'm late. gotta run...

    Regards,
    Bengt Richter
     
    Bengt Richter, Sep 9, 2003
    #8
  9. Ben Young Guest

    On Tue, 9 Sep 2003, David Mertz wrote:

    > () wrote previously:
    > |"Metaclass Programming In Python, Parts 1 and 2"
    > |I get the fact that the instance of a metaclass is a class, but in
    > |this case I fail to see why the following does'nt work:
    >
    > Read part 2 again. It takes a while to sink in.
    >
    > Your problem is that you aren't considering the MRO of Class:
    >
    > >>> class Meta(type):

    > ... def __str__(cls): # Use 'return', not 'print'
    > ... return "I am " + repr(cls)
    > ... def foo(cls):
    > ... return "I am " + repr(cls)
    > ...
    > >>> Class = Meta("Fish", (), {})
    > >>> Class.foo()

    > "I am <class '__main__.Fish'>"
    >
    > Class is an instance of Meta, just like you expect. Calling the .foo()
    > method shows this. But when you call a method, the method name is
    > checked in the -method resolution order- (before the metaclass is
    > checked):
    >
    > >>> Class.mro()

    > [<class '__main__.Fish'>, <type 'object'>]
    >
    > Fish doesn't define a method .__str__(), so Python looks in Fish's
    > parent, object. Well, object -does- define, .__str__(), so your call
    > resolves to:
    >
    > >>> object.__str__()

    > Traceback (most recent call last):
    > File "<stdin>", line 1, in ?
    > TypeError: descriptor '__str__' of 'object' object needs an argument
    >
    > Of course, since object doesn't define a .foo(), resolution proceeds to
    > the metaclass. If you want to get Class' metaclass method, you can let
    > Python do the magic for you:
    >
    > >>> str(Class)

    > "I am <class '__main__.Fish'>"
    >
    > Or if you want to be really explicit:
    >
    > >>> Class.__class__.__str__(Class)

    > "I am <class '__main__.Fish'>"
    >
    > Yours, David...


    Right I think i'm getting there now. Thanks to all those who replied!

    I see now that *instances* if classes must always look in their class dist
    because they _don't have an mro_, whereas classes do and this is always
    looked at first, causing the unbound method error.

    Does this mean function calls are done something like this (untested code)

    mro = getattr(obj, "mro", None)
    if mro:
    for i in mro:
    func = getattr(i, "function")
    if func:
    return func(*args, **kw)

    #Fall through
    return getattr(type(obj), "function")(obj, *args, **kw)

    Ben
    ---

    >
    > --
    > _/_/_/ THIS MESSAGE WAS BROUGHT TO YOU BY: Postmodern Enterprises _/_/_/
    > _/_/ ~~~~~~~~~~~~~~~~~~~~[]~~~~~~~~~~~~~~~~~~~~~ _/_/
    > _/_/ The opinions expressed here must be those of my employer... _/_/
    > _/_/_/_/_/_/_/_/_/_/ Surely you don't think that *I* believe them! _/_/
    >
    >
    >
     
    Ben Young, Sep 10, 2003
    #9
  10. David Mertz Guest

    |mro = getattr(obj, "mro", None)
    |if mro:
    | for i in mro:
    | func = getattr(i, "function")
    | if func:
    | return func(*args, **kw)
    |#Fall through
    |return getattr(type(obj), "function")(obj, *args, **kw)

    Pretty much, but it looks like you have a couple errors in your example.
    I think this is better (also undertested):

    def callmeth(obj, methname, *args, **kw):
    meth = obj.__dict__.get(methname)
    if meth is not None: # Might be in object's dict
    return meth(*args, **kw)
    if hasattr(obj, "mro"): # Might be in the ancestry
    for klass in obj.mro():
    meth = getattr(klass, methname)
    if meth is not None:
    return meth(*args, **kw)
    # Or finally, look in class of obj
    return getattr(type(obj), methname)(*args, **kw)

    I think this does roughly the right thing if a non-callable "meth" is
    encountered.

    Yours, David...

    --
    _/_/_/ THIS MESSAGE WAS BROUGHT TO YOU BY: Postmodern Enterprises _/_/_/
    _/_/ ~~~~~~~~~~~~~~~~~~~~[]~~~~~~~~~~~~~~~~~~~~~ _/_/
    _/_/ The opinions expressed here must be those of my employer... _/_/
    _/_/_/_/_/_/_/_/_/_/ Surely you don't think that *I* believe them! _/_/
     
    David Mertz, Sep 10, 2003
    #10
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Debo
    Replies:
    3
    Views:
    621
    Swampmonster
    Dec 12, 2004
  2. Thomas Womack

    slight csv misbehavior under Windows

    Thomas Womack, Aug 18, 2003, in forum: Python
    Replies:
    1
    Views:
    393
  3. ironfroggy
    Replies:
    16
    Views:
    440
    Michele Simionato
    Jun 3, 2005
  4. Vivek

    Metaclass confusion

    Vivek, Jan 25, 2006, in forum: Ruby
    Replies:
    8
    Views:
    105
    Matthew Moss
    Jan 25, 2006
  5. Steven D'Aprano

    Metaclass of a metaclass

    Steven D'Aprano, Jun 5, 2012, in forum: Python
    Replies:
    1
    Views:
    307
Loading...

Share This Page