Recursion error in metaclass

Discussion in 'Python' started by Steven D'Aprano, Jun 11, 2011.

  1. I have a metaclass in Python 3.1:

    class MC1(type):
    @staticmethod
    def get_mro(bases):
    print('get_mro called')
    return type('K', bases, {}).__mro__[1:]
    def __new__(cls, name, bases, dict):
    mro = None
    docstring = dict.get('__doc__')
    if docstring == 'ham':
    mro = cls.get_mro(bases)
    dict['__doc__'] = "spam spam spam"
    # Create the class we want, and return it.
    K = super().__new__(cls, name, bases, dict)
    if mro:
    assert K.__mro__ == (K,) + mro
    return K


    It seems to work fine:

    >>> class A(metaclass=MC1):

    .... pass
    ....
    >>> class B(A):

    .... 'ham'
    ....
    get_mro called
    >>> assert B.__doc__ == 'spam spam spam'
    >>>



    But if I move the call to get_mro outside of the if block, it works fine
    at first, and then blows up with RecursionError:


    class MC2(type):
    @staticmethod
    def get_mro(bases):
    print('get_mro called')
    return type('K', bases, {}).__mro__[1:]
    def __new__(cls, name, bases, dict):
    mro = None
    docstring = dict.get('__doc__')
    mro = cls.get_mro(bases)
    if docstring == 'ham':
    dict['__doc__'] = "spam spam spam"
    # Create the class we want, and return it.
    K = super().__new__(cls, name, bases, dict)
    if mro:
    assert K.__mro__ == (K,) + mro
    return K


    >>> class C(metaclass=MC2):

    .... pass
    ....
    get_mro called
    >>>
    >>> sys.setrecursionlimit(15)
    >>> class D(C):

    .... 'ham'
    ....
    get_mro called
    get_mro called
    get_mro called
    Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    File "<stdin>", line 9, in __new__
    File "<stdin>", line 5, in get_mro
    File "<stdin>", line 9, in __new__
    File "<stdin>", line 5, in get_mro
    File "<stdin>", line 9, in __new__
    File "<stdin>", line 4, in get_mro
    RuntimeError: maximum recursion depth exceeded while calling a Python
    object



    I am utterly perplexed. What's going on here?




    --
    Steven
    Steven D'Aprano, Jun 11, 2011
    #1
    1. Advertising

  2. Steven D'Aprano

    Terry Reedy Guest

    On 6/10/2011 11:34 PM, Steven D'Aprano wrote:
    > I have a metaclass in Python 3.1:
    >
    > class MC1(type):
    > @staticmethod
    > def get_mro(bases):
    > print('get_mro called')
    > return type('K', bases, {}).__mro__[1:]


    The call to type figures out the proper metaclass from bases and
    forwards the call to that (or to its __new__ method). See
    Objects/typeobject.c in the source, or read the docs on metaclasses
    carefully. If the proper metaclass is MC1, ...


    > def __new__(cls, name, bases, dict):
    > mro = None
    > docstring = dict.get('__doc__')
    > if docstring == 'ham':
    > mro = cls.get_mro(bases)


    and you unconditionally call get_mro again, to call this again...

    > dict['__doc__'] = "spam spam spam"
    > # Create the class we want, and return it.
    > K = super().__new__(cls, name, bases, dict)
    > if mro:
    > assert K.__mro__ == (K,) + mro
    > return K


    you are in an endless loop.

    Since uou do not pass dict to get_mro. it passes {} to type and MC1 and
    the test for docstring fails and the loop is broken and the empty class
    is discarded after getting its mro.

    --
    Terry Jan Reedy
    Terry Reedy, Jun 11, 2011
    #2
    1. Advertising

  3. On Sat, 11 Jun 2011 01:33:25 -0400, Terry Reedy wrote:

    > On 6/10/2011 11:34 PM, Steven D'Aprano wrote:
    >> I have a metaclass in Python 3.1:
    >>
    >> class MC1(type):
    >> @staticmethod
    >> def get_mro(bases):
    >> print('get_mro called')
    >> return type('K', bases, {}).__mro__[1:]

    >
    > The call to type figures out the proper metaclass from bases and
    > forwards the call to that (or to its __new__ method).

    [...]
    > Since uou do not pass dict to get_mro. it passes {} to type and MC1 and
    > the test for docstring fails and the loop is broken and the empty class
    > is discarded after getting its mro.


    Thanks for the explanation. You confused me for a while talking about
    MC1, because that's the metaclass that *doesn't* raise an exception, but
    I think I see the issue now.




    --
    Steven
    Steven D'Aprano, Jun 11, 2011
    #3
  4. Steven D'Aprano

    Terry Reedy Guest

    On 6/11/2011 7:38 AM, Steven D'Aprano wrote:
    > On Sat, 11 Jun 2011 01:33:25 -0400, Terry Reedy wrote:
    >
    >> On 6/10/2011 11:34 PM, Steven D'Aprano wrote:
    >>> I have a metaclass in Python 3.1:
    >>>
    >>> class MC1(type):
    >>> @staticmethod
    >>> def get_mro(bases):
    >>> print('get_mro called')
    >>> return type('K', bases, {}).__mro__[1:]

    >>
    >> The call to type figures out the proper metaclass from bases and
    >> forwards the call to that (or to its __new__ method).

    > [...]
    >> Since uou do not pass dict to get_mro. it passes {} to type and MC1 and
    >> the test for docstring fails and the loop is broken and the empty class
    >> is discarded after getting its mro.

    >
    > Thanks for the explanation. You confused me for a while talking about
    > MC1, because that's the metaclass that *doesn't* raise an exception, but


    Sorry, you probably changed that to MC2 for the second example and I did
    not notice. The point is that when either version calls get_mro and
    type, types calls back to the same metaclass, so that unguarding the
    call to get_mro results in looping.

    > I think I see the issue now.


    What may not be obvious from the docs is that the metaclass calculation
    described in the doc section on class statements is carried out within
    type.__new__ (or after a possible patch, called from within that), so
    that type calls are really "a dynamic form of the class statement" even
    when another another metaclass is specified or implied. "Return a new
    type object." includes instances of type subclasses. I am not sure what
    happens with metaclasses that are not type subclasses. There is at least
    one bug report about the metaclass calculation, which is why I happen to
    have read the typeobject.__new__ code. But I have not read the
    build-class code and all the details of class creation. So I may have
    some of the details above wrong.

    --
    Terry Jan Reedy
    Terry Reedy, Jun 11, 2011
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Jp Calderone
    Replies:
    2
    Views:
    310
    Alex Martelli
    Nov 10, 2003
  2. metaclass error

    , Mar 17, 2005, in forum: Python
    Replies:
    3
    Views:
    411
    Michele Simionato
    Mar 18, 2005
  3. ironfroggy
    Replies:
    16
    Views:
    428
    Michele Simionato
    Jun 3, 2005
  4. Replies:
    8
    Views:
    721
    John Reye
    Apr 26, 2012
  5. Steven D'Aprano

    Metaclass of a metaclass

    Steven D'Aprano, Jun 5, 2012, in forum: Python
    Replies:
    1
    Views:
    298
Loading...

Share This Page