Re: Using metaclasses to play with decorators.

Discussion in 'Python' started by Michael Sparks, Jun 15, 2004.

  1. On 15 Jun 2004, Michele Simionato wrote:

    > The problem of metaclasses-based solutions is that they are hacks.


    Agreed.

    > We don't want everybody to implement his/her version of decorators
    > (yes me too I have implemented mine).


    I was only doing so to toy with them to see what it would offer since it
    was clear from the talk that there was interest in what the community at
    large thought, so I thought I'd have a go and find out what I thought.

    By doing so I came to the viewpoint of why have __doc__ ? Why not have
    access to whatever the first object is - as far as this is practical? If
    it's a dict, so be it, if it's a string, so be it, but provide a uniform
    way of accessing. (eg func.__meta__ perhaps)

    I also came to the opinion that having access to that object is orthoganal
    though complementary to decorators. Without metaclasses I simply would not
    have played around with the ideas, and that's the only area I feel
    comfortable with metaclasses at present - playing with ideas.

    However, your point:

    > Decorators MUST be in the core language. Waiting will only favor the
    > abuse of metaclasses.


    Is something I hadn't considered, and perhaps alot of the people prepared
    to wait beyond 2.4 for a "better" syntax hadn't considered because they
    hadn't realised something like it could be done using metaclasses.

    Indeed, if I hadn't seen Theo de Ridder's excellent talk I certainly
    wouldn't've considered the idea. I'm not sure I would want to use python
    that why myself, but it was impressive.

    Based on playing with metaclasses for different aspects, I might've voted
    for Guido's preferred syntax instead if that possible aspect had been
    raised rather than voting to wait.

    > ...
    > *without* the ability to specify multiple decorators (i.e. let compose
    > the multiple generators explicitely by hand, with a functional.compose
    > function or something like that).


    What if the decorators are things like author, grammar rules, type hints
    (maybe) for external language bindings etc?


    > But I prefer to have a syntax I don't like 100% then waiting for another
    > year or so. Why don't put a given syntax in 2.4a, see how it goes, and
    > change it in 2.4b if it is the case?


    That was part of the rationale behind the "final" decorated_class_four
    metaclass - it allows any syntax you like, you just have to be prepared to
    parse whatever structure you put in the doc string.

    If you 'replaced' the doc string with a (say) doc object (or meta object),
    then some of the usecases for decorators disappear, leaving just the cases
    regarding transformation of the function type. That means whatever syntax
    you do choose for transformational decorators _might_ be less of an issue.
    (I'm not a huge fan of changing syntaxes though, which might well be why I
    like the idea of a doc/meta object)

    I've no idea how complex such a change would be however, and I suspect at
    this point I'm retreading ground others have gone over already.


    Michael.
    Michael Sparks, Jun 15, 2004
    #1
    1. Advertising

  2. Michael Sparks <> wrote in message news:<>...
    > By doing so I came to the viewpoint of why have __doc__ ? Why not have
    > access to whatever the first object is - as far as this is practical? If
    > it's a dict, so be it, if it's a string, so be it, but provide a uniform
    > way of accessing. (eg func.__meta__ perhaps)


    Uhm, that's interesting bu I had a different idea. I think I will post
    about that later.

    > I also came to the opinion that having access to that object is orthoganal
    > though complementary to decorators. Without metaclasses I simply would not
    > have played around with the ideas, and that's the only area I feel
    > comfortable with metaclasses at present - playing with ideas.


    IMO, you really need metaclasses only if you are building a framework such
    as Zope or Twisted, or you are a core Python developer. Mere mortals can have
    only few reasonable usages for metaclasses: to play with them (which is good as
    far as you don't put your experiment in production code), to use them for
    debugging, and to use them as a quick hack to fix code written by others
    or such that you want to change the source code as little as possible.

    > However, your point:
    >
    > > Decorators MUST be in the core language. Waiting will only favor the
    > > abuse of metaclasses.

    >
    > Is something I hadn't considered, and perhaps alot of the people prepared
    > to wait beyond 2.4 for a "better" syntax hadn't considered because they
    > hadn't realised something like it could be done using metaclasses.
    >
    > Indeed, if I hadn't seen Theo de Ridder's excellent talk I certainly
    > wouldn't've considered the idea. I'm not sure I would want to use python
    > that why myself, but it was impressive.


    Even more impressive is the black magic trick to implement lazy evaluation
    and the ternary operator ... or implementing a prototype based object
    system in few lines ... metaclasses ARE fearful.

    > Based on playing with metaclasses for different aspects, I might've voted
    > for Guido's preferred syntax instead if that possible aspect had been
    > raised rather than voting to wait.


    I don't think lot of people will play with metaclasses, and I hope people
    using them will be wise enough to avoid abusing them. So the risk is
    (maybe) small. The reason why I want decorators now is that I have
    started using Zope recently and Zope is in *desperate* need for
    decorators. There will certainly be a delay before decorators will
    enter in Zope after they entered in Python; if they enter in Python
    after 2.4 they will enter in Zope when I will be near retirement ... :-(

    > > ...
    > > *without* the ability to specify multiple decorators (i.e. let compose
    > > the multiple generators explicitely by hand, with a functional.compose
    > > function or something like that).

    >
    > What if the decorators are things like author, grammar rules, type hints
    > (maybe) for external language bindings etc?


    Two options: or you collect the additional attributes in a single decorator

    def txt2html(self, txt) is writtenby(
    author = "M.S",
    email = "M.S@myaddress,com",
    version = "1.0"):
    ...

    or you compose the decorators explicitely

    def txt2html(self, txt) is with_attributes(author, email, version):
    ...

    where with_attributes is a function composing the decorators author, email,
    version, that you have to write by hand. So you are explicit, you
    avoid confusion about the order of the multiple decorators, and you save square
    brackets :)

    Michele Simionato
    Michele Simionato, Jun 16, 2004
    #2
    1. Advertising

  3. On 15 Jun 2004 21:46:22 -0700, (Michele
    Simionato) wrote:

    >IMO, you really need metaclasses only if you are building a framework such
    >as Zope or Twisted, or you are a core Python developer. Mere mortals can have
    >only few reasonable usages for metaclasses: to play with them (which is good as
    >far as you don't put your experiment in production code), to use them for
    >debugging, and to use them as a quick hack to fix code written by others
    >or such that you want to change the source code as little as possible.


    I decided to include a brief discussion of metaclasses in my chapter
    introducing Python OOP for engineering students, but if there is a
    simpler way to do what I need, I could eliminate this section and not
    be tempting students to use (and possibly abuse) metaclasses. (See
    example on p.13 in PythonOOP.doc at
    http://ece.arizona.edu/~edatools/Python )

    I haven't given this much thought, but it occurred to me that making
    the __new__ function work in an ordinary class ( not just a metaclass
    ) would avoid the need for metaclasses in simple cases like my
    example. If the __new__ function is present in a class, then run it
    when any new class is constructed with the current class as an
    ancestor. In my example, all I am doing is adding a class variable
    '_count' to each new class.

    By providing this simple functionality *without* metaclasses, we can
    keep a clear line between what mere mortals need to learn, and what
    the Python developers need.

    -- Dave

    ************************************************************* *
    * David MacQuigg, PhD * email: dmq at gain.com * *
    * IC Design Engineer * phone: USA 520-721-4583 * * *
    * Analog Design Methodologies * * *
    * * 9320 East Mikelyn Lane * * *
    * VRS Consulting, P.C. * Tucson, Arizona 85710 *
    ************************************************************* *
    David MacQuigg, Jun 16, 2004
    #3
  4. Michael Sparks

    Carl Banks Guest

    Michael Sparks wrote:
    > By doing so I came to the viewpoint of why have __doc__ ? Why not have
    > access to whatever the first object is - as far as this is practical? If
    > it's a dict, so be it, if it's a string, so be it, but provide a uniform
    > way of accessing. (eg func.__meta__ perhaps)



    That would be a bit problematic. Here's the main problem:

    def afunc():
    do_something()
    ...

    Here, do_something() is an expression. Is it the first object or not?
    Well, it can't be. Python won't call do_something() until afunc is
    called, but to use it as a decorator, you need to know it's value when
    the function is defined.

    Changing Python so that it evaluates do_something() at runtime is
    totally out of the question. It would cause nearly infinite
    confusion.


    Therefore, do_something() cannot be the first object you speak of.
    Only a Python object with built-in syntax (string, dict, list, tuple,
    integer, etc.), is even feasable as a first-object decorator.

    But this use is still a bit confusing. Constant objects, like strings
    and integers, are not suitable for use as decorators. Collection
    objects, like dicts, can have side effects when evaluated. Are these
    side effects to happen when you call the function, or just when you
    define it?

    In other words, if you want to use dict as "first object decorators",
    you have to live with one of two undesirable possibilities: that side
    effects intended for the decorator get evaluated every function call,
    or that the "first object" has special semantics. (A regular
    docstring, OTOH, has no side effects, and a function using one doesn't
    have any special semantics, so there is no issue with them.)


    --
    CARL BANKS http://www.aerojockey.com/software
    "If you believe in yourself, drink your school, stay on drugs, and
    don't do milk, you can get work."
    -- Parody of Mr. T from a Robert Smigel Cartoon
    Carl Banks, Jun 16, 2004
    #4
  5. David MacQuigg <> wrote in message news:<>...
    > I decided to include a brief discussion of metaclasses in my chapter
    > introducing Python OOP for engineering students, but if there is a
    > simpler way to do what I need, I could eliminate this section and not
    > be tempting students to use (and possibly abuse) metaclasses. (See
    > example on p.13 in PythonOOP.doc at
    > http://ece.arizona.edu/~edatools/Python )
    >
    > I haven't given this much thought, but it occurred to me that making
    > the __new__ function work in an ordinary class ( not just a metaclass
    > ) would avoid the need for metaclasses in simple cases like my
    > example. If the __new__ function is present in a class, then run it
    > when any new class is constructed with the current class as an
    > ancestor. In my example, all I am doing is adding a class variable
    > '_count' to each new class.
    >
    > By providing this simple functionality *without* metaclasses, we can
    > keep a clear line between what mere mortals need to learn, and what
    > the Python developers need.
    >
    > -- Dave


    You seem to think that __new__ is something related to metaclasses only
    (at least I got this impression). It is not. __new__ is the generic
    instance allocator; can be used inside classes to create regular instances
    and inside metaclasses to create classes (and inside meta-metaclasses to
    create metaclasses, etc.). In your example I would just use __init__
    in the metaclass, not __new__: something like

    def __init__(cls, name, bases, dic):
    cls._count = 0
    ...

    But you ask if it makes sense to use a metaclass for this task or not.

    Not really, you could use a function acting as a factory of classes,
    or a classmethod doing the same job. Or even a function setting the
    counter to zero after the class creation in all the classes of your
    hierarchy (using Animal.__subclasses__()). You would lose a bit of elegance
    but you would get the same functionality. Plus, you would avoid cluttering
    too much the minds of your students. Plus, you would avoid embarassing
    questions from the smartest between them (such why I get a meta-type conflict
    if I do this and that?)

    There is no point in teaching everything, let them a bit of fun if they
    want to :)


    Michele Simionato
    Michele Simionato, Jun 17, 2004
    #5
  6. Michael Sparks

    Jeff Epler Guest

    On Wed, Jun 16, 2004 at 04:18:53AM -0700, David MacQuigg wrote:
    > I haven't given this much thought, but it occurred to me that making
    > the __new__ function work in an ordinary class ( not just a metaclass
    > ) would avoid the need for metaclasses in simple cases like my
    > example. [...]


    __new__ does work in an "ordinary class"! It just does something
    different than what you want. Here's a small "rational number" class
    where Rational(x, y) returns an int if x/y is an integer, or a Rational
    instance otherwise.

    class Rational(object):
    def __init__(self, num, den=1):
    self.num = num
    self.den = den

    def __new__(self, num, den=1):
    if num % den == 0:
    return int(num)
    else:
    return object.__new__(Rational)

    def __str__(self): return "%d/%d" % (self.num, self.den)

    # Example
    r, s = Rational(6, 3), Rational(6, 4)
    print r, type(r) # It's an int
    print s, type(s) # It's a Rational


    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.2.1 (GNU/Linux)

    iD8DBQFA0Zb+Jd01MZaTXX0RAlXUAKCsWYcJZp+/pxDZVqvWYJIYV2vqWwCdGtaE
    vcdTmUObo5t7oZGfnOBX+Dc=
    =qCSW
    -----END PGP SIGNATURE-----
    Jeff Epler, Jun 17, 2004
    #6
  7. On Thu, 17 Jun 2004 08:05:03 -0500, Jeff Epler <>
    wrote:

    >On Wed, Jun 16, 2004 at 04:18:53AM -0700, David MacQuigg wrote:
    >> I haven't given this much thought, but it occurred to me that making
    >> the __new__ function work in an ordinary class ( not just a metaclass
    >> ) would avoid the need for metaclasses in simple cases like my
    >> example. [...]

    >
    >__new__ does work in an "ordinary class"! It just does something
    >different than what you want. Here's a small "rational number" class
    >where Rational(x, y) returns an int if x/y is an integer, or a Rational
    >instance otherwise.


    I just read Alex Martelli's 4 pages on metaclasses in "Python in a
    Nutshell", and I'm finally starting to understand this topic. Thanks
    to you and Michele Simionato for catching my misunderstanding on the
    use of __new__.

    I've also corrected the one-page explanation of metaclasses in my OOP
    chapter. I had decided earlier to take this out of the chapter, but
    now I think I will leave it in. Metaclasses have an advantage over
    factory functions that makes learning at least the basics worth while.
    That is my current understanding, anyway. I'm still learning.

    -- Dave

    >class Rational(object):
    > def __init__(self, num, den=1):
    > self.num = num
    > self.den = den
    >
    > def __new__(self, num, den=1):
    > if num % den == 0:
    > return int(num)
    > else:
    > return object.__new__(Rational)
    >
    > def __str__(self): return "%d/%d" % (self.num, self.den)
    >
    ># Example
    >r, s = Rational(6, 3), Rational(6, 4)
    >print r, type(r) # It's an int
    >print s, type(s) # It's a Rational
    David MacQuigg, Jun 18, 2004
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Michael Sparks

    Using metaclasses to play with decorators.

    Michael Sparks, Jun 15, 2004, in forum: Python
    Replies:
    4
    Views:
    328
    Michele Simionato
    Jun 18, 2004
  2. Robert Brewer
    Replies:
    9
    Views:
    302
    David MacQuigg
    Jun 26, 2004
  3. Paul Morrow
    Replies:
    2
    Views:
    240
    Larry Bates
    Aug 24, 2004
  4. Anthony Baxter
    Replies:
    3
    Views:
    264
    Jess Austin
    Aug 26, 2004
  5. Paul Morrow
    Replies:
    83
    Views:
    1,208
    Hallvard B Furuseth
    Sep 6, 2004
Loading...

Share This Page