Recursion error in metaclass

[Steven D'Aprano]

I have a metaclass in Python 3.1:

class MC1(type):
@staticmethod
def get_mro(bases):
print('get_mro called')
return type('K', bases, {}).__mro__[1:]
def __new__(cls, name, bases, dict):
mro = None
docstring = dict.get('__doc__')
if docstring == 'ham':
mro = cls.get_mro(bases)
dict['__doc__'] = "spam spam spam"
# Create the class we want, and return it.
K = super().__new__(cls, name, bases, dict)
if mro:
assert K.__mro__ == (K,) + mro
return K


It seems to work fine:
.... pass
........ 'ham'
....
get_mro called

But if I move the call to get_mro outside of the if block, it works fine
at first, and then blows up with RecursionError:


class MC2(type):
@staticmethod
def get_mro(bases):
print('get_mro called')
return type('K', bases, {}).__mro__[1:]
def __new__(cls, name, bases, dict):
mro = None
docstring = dict.get('__doc__')
mro = cls.get_mro(bases)
if docstring == 'ham':
dict['__doc__'] = "spam spam spam"
# Create the class we want, and return it.
K = super().__new__(cls, name, bases, dict)
if mro:
assert K.__mro__ == (K,) + mro
return K

.... pass
....
get_mro called.... 'ham'
....
get_mro called
get_mro called
get_mro called
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 9, in __new__
File "<stdin>", line 5, in get_mro
File "<stdin>", line 9, in __new__
File "<stdin>", line 5, in get_mro
File "<stdin>", line 9, in __new__
File "<stdin>", line 4, in get_mro
RuntimeError: maximum recursion depth exceeded while calling a Python
object



I am utterly perplexed. What's going on here?

 

[Terry Reedy]

I have a metaclass in Python 3.1:

class MC1(type):
@staticmethod
def get_mro(bases):
print('get_mro called')
return type('K', bases, {}).__mro__[1:]

The call to type figures out the proper metaclass from bases and
forwards the call to that (or to its __new__ method). See
Objects/typeobject.c in the source, or read the docs on metaclasses
carefully. If the proper metaclass is MC1, ...

def __new__(cls, name, bases, dict):
mro = None
docstring = dict.get('__doc__')
if docstring == 'ham':
mro = cls.get_mro(bases)

and you unconditionally call get_mro again, to call this again...

dict['__doc__'] = "spam spam spam"
# Create the class we want, and return it.
K = super().__new__(cls, name, bases, dict)
if mro:
assert K.__mro__ == (K,) + mro
return K

you are in an endless loop.

Since uou do not pass dict to get_mro. it passes {} to type and MC1 and
the test for docstring fails and the loop is broken and the empty class
is discarded after getting its mro.

 

[Steven D'Aprano]

I have a metaclass in Python 3.1:

class MC1(type):
@staticmethod
def get_mro(bases):
print('get_mro called')
return type('K', bases, {}).__mro__[1:]

The call to type figures out the proper metaclass from bases and
forwards the call to that (or to its __new__ method). [...]
Since uou do not pass dict to get_mro. it passes {} to type and MC1 and
the test for docstring fails and the loop is broken and the empty class
is discarded after getting its mro.

Thanks for the explanation. You confused me for a while talking about
MC1, because that's the metaclass that *doesn't* raise an exception, but
I think I see the issue now.

 

[Terry Reedy]

I have a metaclass in Python 3.1:

class MC1(type):
@staticmethod
def get_mro(bases):
print('get_mro called')
return type('K', bases, {}).__mro__[1:]

The call to type figures out the proper metaclass from bases and
forwards the call to that (or to its __new__ method). [...]
Since uou do not pass dict to get_mro. it passes {} to type and MC1 and
the test for docstring fails and the loop is broken and the empty class
is discarded after getting its mro.

Thanks for the explanation. You confused me for a while talking about
MC1, because that's the metaclass that *doesn't* raise an exception, but

Sorry, you probably changed that to MC2 for the second example and I did
not notice. The point is that when either version calls get_mro and
type, types calls back to the same metaclass, so that unguarding the
call to get_mro results in looping.

I think I see the issue now.

What may not be obvious from the docs is that the metaclass calculation
described in the doc section on class statements is carried out within
type.__new__ (or after a possible patch, called from within that), so
that type calls are really "a dynamic form of the class statement" even
when another another metaclass is specified or implied. "Return a new
type object." includes instances of type subclasses. I am not sure what
happens with metaclasses that are not type subclasses. There is at least
one bug report about the metaclass calculation, which is why I happen to
have read the typeobject.__new__ code. But I have not read the
build-class code and all the details of class creation. So I may have
some of the details above wrong.