RE: Using metaclasses to play with decorators.

Discussion in 'Python' started by Robert Brewer, Jun 18, 2004.

  1. j_mckitrick wrote:
    > You guys are WAY over my head, and I'm supposed to be a C.S.
    > junior! :-\
    >
    > Could someone give me an idea of what metaclasses and
    > decorators _are_?


    Sure thing.

    > I think metaclasses are classes with their base methods
    > overridden, correct?
    >
    > And decorators are a form of delegation, right?


    Oh, and you were doing so well until you guessed... ;)

    > If so, where am I wrong? And how do these techniques make
    > for better code/coding?


    Metaclasses are mind-demolition tools:

    http://www.python.org/2.2/descrintro.html#metaclasses

    As a CS student, you're probably familiar with the concept of a
    'factory': a block of code which forms objects of a given class. In
    Python, classes are also objects. A metaclass can be thought of as a
    'class factory': a factory which produces classes (as opposed to
    instances of classes). To say it another way, a class can loosely be
    considered a template for objects; a metaclass is a template for
    classes.

    Metaclasses allow you to customize the creation of classes. A common use
    is when you have a base class, let'a call it A, and you want to provide
    a behavior for each subclass "class B(A)", C, D... For example, you
    might want to 'register' each subclass in a module-level list. You can
    do this without metaclasses:

    class A(object):
    pass

    class B(A):
    custom_data = {}

    class C(A):
    funky_stuff = []

    subclasses_of_A = [B, C]

    ....but that can get tedious, especially if you aren't writing the
    subclasses (perhaps you're writing a framework). Whenever anyone writes
    a subclass (perhaps in a different module), they have to remember to
    append their new subclass to subclasses_of_A.

    Metaclasses solve this by customizing class-creation code. In our
    registration example, we might write:

    subclasses_of_A = []

    class RegisterA(type):
    def __init__(cls, name, bases, dct):
    subclasses_of_A.append(cls)

    class A(object):
    __metaclass__ = RegisterA

    class B(A):
    custom_data = {}

    class C(A):
    funky_stuff = []

    Testing this (we'll save our module as 'metatest.py'), we obtain
    (PythonWin 2.3.2):

    >>> import metatest
    >>> metatest.subclasses_of_A

    [<class 'metatest.A'>, <class 'metatest.B'>, <class 'metatest.C'>]

    When we declare class A, we are creating a new object 'A' which happens
    to be a class. By giving it a __metaclass__ attribute, we can customize
    the creation of class A. RegisterA doesn't override type.__new__, so
    type.__new__ forms the new class as usual. RegisterA does override
    __init__, so RegisterA.__init__ gets called with our new (empty) class A
    as the first argument, at which point we append metatest.A to our list.
    I'll leave you to figure out how to exclude A but keep all the
    subclasses ;) Exercise 2 would be to make this mechanism generic (that
    is, keep subclasses_of_A separate from subclasses_of_Thing).

    ================================

    Decorators are a very recent discussion; you might want to browse recent
    threads on python-dev for a lot more information. You should certainly
    read PEP 318 (http://www.python.org/peps/pep-0318.html). Basically, a
    decorator is a way to transform a function. Again, this can usually be
    done without metaclasses. The most-frequently-discussed examples are
    staticmethods and classmethods:

    class Thing(object):
    def newThing(cls, attr):
    t = cls()
    t.attr = attr
    return t
    newThing = classmethod(newThing)

    Decorators will hopefully provide a clearer syntax for saying the same
    thing (although the exact syntax is still up for debate):

    class Thing(object):
    def newThing(cls, attr) [classmethod]:
    t = cls()
    t.attr = attr
    return t

    This moves the transformation ('classmethod') closer to the function
    definition, rather than placing it at the end of the function. In
    addition, you don't have to write the name 'newThing' three times. Also,
    multiple transforms could be defined at once.

    Does that help?


    Waiting expectantly for a heapin' helpin' of picked nits,

    Robert Brewer
    MIS
    Amor Ministries
    Robert Brewer, Jun 18, 2004
    #1
    1. Advertising

  2. Robert Brewer wrote:
    > j_mckitrick wrote:
    >
    >>You guys are WAY over my head, and I'm supposed to be a C.S.
    >>junior! :-\
    >>
    >>Could someone give me an idea of what metaclasses and
    >>decorators _are_?

    >
    >
    > Sure thing.
    >
    >
    >>I think metaclasses are classes with their base methods
    >>overridden, correct?
    >>
    >>And decorators are a form of delegation, right?

    >
    >
    > Oh, and you were doing so well until you guessed... ;)
    >
    >
    >>If so, where am I wrong? And how do these techniques make
    >>for better code/coding?

    >
    >
    > Metaclasses are mind-demolition tools:
    >
    > http://www.python.org/2.2/descrintro.html#metaclasses
    >
    > As a CS student, you're probably familiar with the concept of a
    > 'factory': a block of code which forms objects of a given class. In
    > Python, classes are also objects. A metaclass can be thought of as a
    > 'class factory': a factory which produces classes (as opposed to
    > instances of classes). To say it another way, a class can loosely be
    > considered a template for objects; a metaclass is a template for
    > classes.
    >
    > Metaclasses allow you to customize the creation of classes. A common use
    > is when you have a base class, let'a call it A, and you want to provide
    > a behavior for each subclass "class B(A)", C, D... For example, you
    > might want to 'register' each subclass in a module-level list. You can
    > do this without metaclasses:
    >
    > class A(object):
    > pass
    >
    > class B(A):
    > custom_data = {}
    >
    > class C(A):
    > funky_stuff = []
    >
    > subclasses_of_A = [B, C]
    >
    > ....but that can get tedious, especially if you aren't writing the
    > subclasses (perhaps you're writing a framework). Whenever anyone writes
    > a subclass (perhaps in a different module), they have to remember to
    > append their new subclass to subclasses_of_A.
    >
    > Metaclasses solve this by customizing class-creation code. In our
    > registration example, we might write:
    >
    > subclasses_of_A = []
    >
    > class RegisterA(type):
    > def __init__(cls, name, bases, dct):
    > subclasses_of_A.append(cls)
    >
    > class A(object):
    > __metaclass__ = RegisterA
    >
    > class B(A):
    > custom_data = {}
    >
    > class C(A):
    > funky_stuff = []
    >
    > Testing this (we'll save our module as 'metatest.py'), we obtain
    > (PythonWin 2.3.2):
    >
    >
    >>>>import metatest
    >>>>metatest.subclasses_of_A

    >
    > [<class 'metatest.A'>, <class 'metatest.B'>, <class 'metatest.C'>]
    >
    > When we declare class A, we are creating a new object 'A' which happens
    > to be a class. By giving it a __metaclass__ attribute, we can customize
    > the creation of class A. RegisterA doesn't override type.__new__, so
    > type.__new__ forms the new class as usual. RegisterA does override
    > __init__, so RegisterA.__init__ gets called with our new (empty) class A
    > as the first argument, at which point we append metatest.A to our list.
    > I'll leave you to figure out how to exclude A but keep all the
    > subclasses ;) Exercise 2 would be to make this mechanism generic (that
    > is, keep subclasses_of_A separate from subclasses_of_Thing).
    >
    > ================================
    >
    > Decorators are a very recent discussion; you might want to browse recent
    > threads on python-dev for a lot more information. You should certainly
    > read PEP 318 (http://www.python.org/peps/pep-0318.html). Basically, a
    > decorator is a way to transform a function. Again, this can usually be
    > done without metaclasses. The most-frequently-discussed examples are
    > staticmethods and classmethods:
    >
    > class Thing(object):
    > def newThing(cls, attr):
    > t = cls()
    > t.attr = attr
    > return t
    > newThing = classmethod(newThing)
    >
    > Decorators will hopefully provide a clearer syntax for saying the same
    > thing (although the exact syntax is still up for debate):
    >
    > class Thing(object):
    > def newThing(cls, attr) [classmethod]:
    > t = cls()
    > t.attr = attr
    > return t
    >
    > This moves the transformation ('classmethod') closer to the function
    > definition, rather than placing it at the end of the function. In
    > addition, you don't have to write the name 'newThing' three times. Also,
    > multiple transforms could be defined at once.
    >
    > Does that help?
    >
    >
    > Waiting expectantly for a heapin' helpin' of picked nits,
    >
    > Robert Brewer
    > MIS
    > Amor Ministries
    >
    >

    I don't wish to pick nits, but to suggest that GVD defined metaclasses
    as classes whose instances are classes. This seems straighforward.

    I have yet to wrap my mind around decorators.

    Colin W
    Colin J. Williams, Jun 20, 2004
    #2
    1. Advertising

  3. On Sun, 20 Jun 2004 14:48:23 -0400, "Colin J. Williams"
    <> wrote:

    >I have yet to wrap my mind around decorators.


    Decorators are very simple. They are just a way to provide different
    forms of methods, without introducing a new keyword, or some other
    more awkward syntax.

    Say you wanted to define a method that didn't have 'self' as its first
    argument. You could add a new keyword to the language:

    noself methodA(x,y):
    return x + y

    Or you could add a "decorator" to the existing syntax:

    def methodA(x,y) [noself]:
    return x + y

    Change 'noself' to 'staticmethod' and you have one of the current
    proposals in PEP318.

    Don't get distracted by 'staticmethod' and other mumbo-jumbo
    terminology, and you should have no problem with decorators.

    -- Dave
    David MacQuigg, Jun 23, 2004
    #3
  4. Robert Brewer

    David Fraser Guest

    David MacQuigg wrote:
    > On Sun, 20 Jun 2004 14:48:23 -0400, "Colin J. Williams"
    > <> wrote:
    >
    >
    >>I have yet to wrap my mind around decorators.

    >
    >
    > Decorators are very simple. They are just a way to provide different
    > forms of methods, without introducing a new keyword, or some other
    > more awkward syntax.
    >
    > Say you wanted to define a method that didn't have 'self' as its first
    > argument. You could add a new keyword to the language:
    >
    > noself methodA(x,y):
    > return x + y
    >
    > Or you could add a "decorator" to the existing syntax:
    >
    > def methodA(x,y) [noself]:
    > return x + y
    >
    > Change 'noself' to 'staticmethod' and you have one of the current
    > proposals in PEP318.
    >
    > Don't get distracted by 'staticmethod' and other mumbo-jumbo
    > terminology, and you should have no problem with decorators.


    Aren't there other people around who would find
    staticmethod methodA(x,y):
    return x+y
    a clearer and more helpful syntax than
    def methodA(x,y) [staticmethod]:
    return x + y

    I've always wanted a
    method A(x,y):
    self.var += x*y
    which automatically declared self as an initial parameter...

    David
    David Fraser, Jun 23, 2004
    #4
  5. David MacQuigg wrote:
    > On Sun, 20 Jun 2004 14:48:23 -0400, "Colin J. Williams"
    > <> wrote:
    >
    >
    >>I have yet to wrap my mind around decorators.

    >
    >
    > Decorators are very simple. They are just a way to provide different
    > forms of methods, without introducing a new keyword, or some other
    > more awkward syntax.
    >
    > Say you wanted to define a method that didn't have 'self' as its first
    > argument. You could add a new keyword to the language:
    >
    > noself methodA(x,y):
    > return x + y
    >
    > Or you could add a "decorator" to the existing syntax:
    >
    > def methodA(x,y) [noself]:
    > return x + y
    >
    > Change 'noself' to 'staticmethod' and you have one of the current
    > proposals in PEP318.
    >
    > Don't get distracted by 'staticmethod' and other mumbo-jumbo
    > terminology, and you should have no problem with decorators.


    OK, I'll ignore 'staticmethod', but could you tell me how

    def methodA(x, y) [noself]:
    return x + y

    differs in substance from

    def methodA(self, y):
    return self + y

    or
    def methodA(x, y):
    return x + y

    What has been gained by the added syntactic clutter?

    Colin W.
    >
    > -- Dave
    >
    Colin J. Williams, Jun 23, 2004
    #5
  6. Robert Brewer

    Duncan Booth Guest

    "Colin J. Williams" <> wrote in
    news:uRdCc.27945$:

    > OK, I'll ignore 'staticmethod', but could you tell me how
    >
    > def methodA(x, y) [noself]:
    > return x + y
    >
    > differs in substance from
    >
    > def methodA(self, y):
    > return self + y
    >
    > or
    > def methodA(x, y):
    > return x + y
    >
    > What has been gained by the added syntactic clutter?


    Does this help?

    class X:
    def methodA(x, y) [ staticmethod ]:
    return x+y

    def methodB(self,y):
    return self+y

    def methodC(x,y):
    return x+y

    anX = X()

    anX.methodA(2, 3) # returns 5
    anX.methodB(2, 3) # Exception: too many arguments
    anX.methodC(2, 3) # Exception: too many arguments

    X.methodA(2, 3) # returns 5
    X.methodB(2, 3) # Exception: First argument must be an instance of X
    X.methodC(2, 3) # Exception: First argument must be an instance of X
    Duncan Booth, Jun 23, 2004
    #6
  7. On Wed, 23 Jun 2004 07:30:34 -0400, "Colin J. Williams"
    <> wrote:

    >
    >
    >David MacQuigg wrote:
    >> On Sun, 20 Jun 2004 14:48:23 -0400, "Colin J. Williams"
    >> <> wrote:
    >>
    >>
    >>>I have yet to wrap my mind around decorators.

    >>
    >>
    >> Decorators are very simple. They are just a way to provide different
    >> forms of methods, without introducing a new keyword, or some other
    >> more awkward syntax.
    >>
    >> Say you wanted to define a method that didn't have 'self' as its first
    >> argument. You could add a new keyword to the language:
    >>
    >> noself methodA(x,y):
    >> return x + y
    >>
    >> Or you could add a "decorator" to the existing syntax:
    >>
    >> def methodA(x,y) [noself]:
    >> return x + y
    >>
    >> Change 'noself' to 'staticmethod' and you have one of the current
    >> proposals in PEP318.
    >>
    >> Don't get distracted by 'staticmethod' and other mumbo-jumbo
    >> terminology, and you should have no problem with decorators.

    >
    >OK, I'll ignore 'staticmethod', but could you tell me how
    >
    > def methodA(x, y) [noself]:
    > return x + y
    >
    >differs in substance from
    >
    > def methodA(self, y):
    > return self + y
    >
    >or
    > def methodA(x, y):
    > return x + y
    >
    >What has been gained by the added syntactic clutter?


    The example above was to explain decorators, not justify them. If
    they were to be used *only* to clean up the current syntax for
    staticmethod, then I would say a better alternative would be to get
    rid of the need for a separate staticmethod form entirely. ( See the
    thread "Unification of Methods and Functions" for a thorough
    discussion of this topic.)

    Are decorators on functions necessary? I can't think of a simpler or
    more consistent way to handle all the variations proposed in PEP 318.

    Assuming that we *will* have a decorator syntax with many options, I
    think that making [staticmethod] one of those options is appropriate.
    I would still prefer a word more meaningful to new users, however. The
    rational I have heard for "staticmethod" is so contorted, it is not
    worth repeating.

    -- Dave
    David MacQuigg, Jun 23, 2004
    #7
  8. Duncan Booth wrote:
    > "Colin J. Williams" <> wrote in
    > news:uRdCc.27945$:
    >
    >
    >>OK, I'll ignore 'staticmethod', but could you tell me how
    >>
    >> def methodA(x, y) [noself]:
    >> return x + y
    >>
    >>differs in substance from
    >>
    >> def methodA(self, y):
    >> return self + y
    >>
    >>or

    class Z:
    def __init__(x, a):
    x.a= a
    >> def methodA(x, y):
    >> return x + y
    >>
    >>What has been gained by the added syntactic clutter?

    >
    >
    > Does this help?

    Yes, this clarifies the static method idea.

    Thanks,

    Colin W.
    >
    > class X:
    > def methodA(x, y) [ staticmethod ]:
    > return x+y
    >
    > def methodB(self,y):
    > return self+y
    >
    > def methodC(x,y):
    > return x+y
    >
    > anX = X()
    >
    > anX.methodA(2, 3) # returns 5
    > anX.methodB(2, 3) # Exception: too many arguments
    > anX.methodC(2, 3) # Exception: too many arguments
    >
    > X.methodA(2, 3) # returns 5
    > X.methodB(2, 3) # Exception: First argument must be an instance of X
    > X.methodC(2, 3) # Exception: First argument must be an instance of X
    Colin J. Williams, Jun 24, 2004
    #8
  9. David MacQuigg wrote:

    > On Wed, 23 Jun 2004 07:30:34 -0400, "Colin J. Williams"
    > <> wrote:
    >
    >
    >>
    >>David MacQuigg wrote:
    >>
    >>>On Sun, 20 Jun 2004 14:48:23 -0400, "Colin J. Williams"
    >>><> wrote:
    >>>
    >>>
    >>>
    >>>>I have yet to wrap my mind around decorators.
    >>>
    >>>
    >>>Decorators are very simple. They are just a way to provide different
    >>>forms of methods, without introducing a new keyword, or some other
    >>>more awkward syntax.
    >>>
    >>>Say you wanted to define a method that didn't have 'self' as its first
    >>>argument. You could add a new keyword to the language:
    >>>
    >>>noself methodA(x,y):
    >>> return x + y
    >>>
    >>>Or you could add a "decorator" to the existing syntax:
    >>>
    >>>def methodA(x,y) [noself]:
    >>> return x + y
    >>>
    >>>Change 'noself' to 'staticmethod' and you have one of the current
    >>>proposals in PEP318.
    >>>
    >>>Don't get distracted by 'staticmethod' and other mumbo-jumbo
    >>>terminology, and you should have no problem with decorators.

    >>
    >>OK, I'll ignore 'staticmethod', but could you tell me how
    >>
    >> def methodA(x, y) [noself]:
    >> return x + y
    >>
    >>differs in substance from
    >>
    >> def methodA(self, y):
    >> return self + y
    >>
    >>or
    >> def methodA(x, y):
    >> return x + y
    >>
    >>What has been gained by the added syntactic clutter?

    >
    >
    > The example above was to explain decorators, not justify them. If
    > they were to be used *only* to clean up the current syntax for
    > staticmethod, then I would say a better alternative would be to get
    > rid of the need for a separate staticmethod form entirely. ( See the
    > thread "Unification of Methods and Functions" for a thorough
    > discussion of this topic.)
    >
    > Are decorators on functions necessary? I can't think of a simpler or
    > more consistent way to handle all the variations proposed in PEP 318.
    >
    > Assuming that we *will* have a decorator syntax with many options, I
    > think that making [staticmethod] one of those options is appropriate.
    > I would still prefer a word more meaningful to new users, however. The
    > rational I have heard for "staticmethod" is so contorted, it is not
    > worth repeating.
    >
    > -- Dave
    >

    PEP 318 seems to focus on the "how" to implement decorators, rather than
    the "why". Is there some accessible explanation of the purpose of them?

    Colin W.
    Colin J. Williams, Jun 24, 2004
    #9
  10. On Thu, 24 Jun 2004 08:57:49 -0400, "Colin J. Williams"
    <> wrote:

    >David MacQuigg wrote:


    >> The example above was to explain decorators, not justify them. If
    >> they were to be used *only* to clean up the current syntax for
    >> staticmethod, then I would say a better alternative would be to get
    >> rid of the need for a separate staticmethod form entirely. ( See the
    >> thread "Unification of Methods and Functions" for a thorough
    >> discussion of this topic.)
    >>
    >> Are decorators on functions necessary? I can't think of a simpler or
    >> more consistent way to handle all the variations proposed in PEP 318.
    >>
    >> Assuming that we *will* have a decorator syntax with many options, I
    >> think that making [staticmethod] one of those options is appropriate.
    >> I would still prefer a word more meaningful to new users, however. The
    >> rational I have heard for "staticmethod" is so contorted, it is not
    >> worth repeating.
    >>
    >> -- Dave
    >>

    >PEP 318 seems to focus on the "how" to implement decorators, rather than
    >the "why". Is there some accessible explanation of the purpose of them?


    The first few sections of the PEP are your best summary ( Abstract,
    Motivation, Background, Design Goals ). Other than that, I would
    search the discussion of this PEP in the python-dev mailing list.
    There are some specific links in the Background section of the PEP.

    -- Dave
    David MacQuigg, Jun 26, 2004
    #10
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Michael Sparks

    Using metaclasses to play with decorators.

    Michael Sparks, Jun 15, 2004, in forum: Python
    Replies:
    4
    Views:
    329
    Michele Simionato
    Jun 18, 2004
  2. Michael Sparks
    Replies:
    6
    Views:
    348
    David MacQuigg
    Jun 18, 2004
  3. Paul Morrow
    Replies:
    2
    Views:
    242
    Larry Bates
    Aug 24, 2004
  4. Anthony Baxter
    Replies:
    3
    Views:
    265
    Jess Austin
    Aug 26, 2004
  5. Paul Morrow
    Replies:
    83
    Views:
    1,212
    Hallvard B Furuseth
    Sep 6, 2004
Loading...

Share This Page