name lookup failure using metaclasses with unittests

Discussion in 'Python' started by Ulrich Eckhardt, Apr 10, 2013.

  1. Hi!

    I'm having problems using a metaclass to generate test functions. This
    works when I try to run all tests from the module or test case, but it
    fails when I'm trying to specify a single test function. My environment
    is Python 2.7.3 on MS Windows 7 at the moment. It should be upgraded to
    at least 2.7.4 or better to 3, but see the notes on Python 3 below.

    # my_module.py
    import unittest
    class X(unittest.TestCase):
    def __metaclass__(name, bases, dict):
    # attach function
    def test(self):
    pass
    dict['test_1'] = test
    dict['test_2'] = test
    # create class
    return type(name, bases, dict)

    The error when I'm trying to run "python -m unittest my_module.X.test_1"
    is: "Value error: no such test method in <class 'my_module.X'>: test".
    The astonishing part is that it claims that "test" is not found while I
    asked it to run "test_1". The name it complains about is the name of the
    function inside the metaclass function. In all other cases, like e.g.
    giving "-v" it reports the correct function name. My question here is
    whether I'm doing something wrong or whether I discovered a bug.


    Now, concerning Python 3, it fails to detect any test case at all! My
    guess is that the unittest library was changed to use metaclasses itself
    in order to detect classes derived from unittest.TestCase. Therefore,
    overriding the metaclass breaks test case discovery. My question in that
    context is how do I extend metaclasses instead of overriding it? In
    other words, what is the equivalent to super() for class creation?

    Thank you for your help!

    Uli
    Ulrich Eckhardt, Apr 10, 2013
    #1
    1. Advertising

  2. Ulrich Eckhardt

    Peter Otten Guest

    Ulrich Eckhardt wrote:

    > Hi!
    >
    > I'm having problems using a metaclass to generate test functions. This
    > works when I try to run all tests from the module or test case, but it
    > fails when I'm trying to specify a single test function. My environment
    > is Python 2.7.3 on MS Windows 7 at the moment. It should be upgraded to
    > at least 2.7.4 or better to 3, but see the notes on Python 3 below.
    >
    > # my_module.py
    > import unittest
    > class X(unittest.TestCase):
    > def __metaclass__(name, bases, dict):
    > # attach function
    > def test(self):
    > pass
    > dict['test_1'] = test
    > dict['test_2'] = test
    > # create class
    > return type(name, bases, dict)
    >
    > The error when I'm trying to run "python -m unittest my_module.X.test_1"
    > is: "Value error: no such test method in <class 'my_module.X'>: test".
    > The astonishing part is that it claims that "test" is not found while I
    > asked it to run "test_1". The name it complains about is the name of the
    > function inside the metaclass function. In all other cases, like e.g.
    > giving "-v" it reports the correct function name. My question here is
    > whether I'm doing something wrong or whether I discovered a bug.


    Here's a simpler demo of the problem:

    $ cat tmp.py
    import unittest

    class X(unittest.TestCase):
    def test_1(self): pass
    test_1.__name__ = "test_2"

    $ python -m unittest -v tmp
    test_1 (tmp.X) ... ok

    ----------------------------------------------------------------------
    Ran 1 test in 0.001s

    OK
    $ python -m unittest -v tmp.X.test_1
    Traceback (most recent call last):
    File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
    File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
    File "/usr/lib/python2.7/unittest/__main__.py", line 12, in <module>
    main(module=None)
    File "/usr/lib/python2.7/unittest/main.py", line 94, in __init__
    self.parseArgs(argv)
    File "/usr/lib/python2.7/unittest/main.py", line 149, in parseArgs
    self.createTests()
    File "/usr/lib/python2.7/unittest/main.py", line 158, in createTests
    self.module)
    File "/usr/lib/python2.7/unittest/loader.py", line 128, in
    loadTestsFromNames
    suites = [self.loadTestsFromName(name, module) for name in names]
    File "/usr/lib/python2.7/unittest/loader.py", line 109, in
    loadTestsFromName
    return self.suiteClass([parent(obj.__name__)])
    File "/usr/lib/python2.7/unittest/case.py", line 191, in __init__
    (self.__class__, methodName))
    ValueError: no such test method in <class 'tmp.X'>: test_2

    It looks like this particular invocation relies on class attribute and
    function __name__ being identical.

    Please file a bug report.

    > Now, concerning Python 3, it fails to detect any test case at all! My
    > guess is that the unittest library was changed to use metaclasses itself
    > in order to detect classes derived from unittest.TestCase. Therefore,
    > overriding the metaclass breaks test case discovery. My question in that
    > context is how do I extend metaclasses instead of overriding it? In
    > other words, what is the equivalent to super() for class creation?


    Python 3 does not recognize the __metaclass__ attribute as the metaclass.
    You need to provide it like so:

    def __metaclass__(name, bases, dict):
    ...

    class X(unittest.TestCase, metaclass=__metaclass__):
    pass
    Peter Otten, Apr 10, 2013
    #2
    1. Advertising

  3. Am 10.04.2013 11:52, schrieb Peter Otten:
    > Ulrich Eckhardt wrote:

    [...]
    > It looks like this particular invocation relies on class attribute and
    > function __name__ being identical.
    >
    > Please file a bug report.


    Thanks for confirming this and reducing the test case even more.


    >> Now, concerning Python 3, it fails to detect any test case at all! My
    >> guess is that the unittest library was changed to use metaclasses itself
    >> in order to detect classes derived from unittest.TestCase. Therefore,
    >> overriding the metaclass breaks test case discovery. My question in that
    >> context is how do I extend metaclasses instead of overriding it? In
    >> other words, what is the equivalent to super() for class creation?

    >
    > Python 3 does not recognize the __metaclass__ attribute as the metaclass.
    > You need to provide it like so:
    >
    > def __metaclass__(name, bases, dict):
    > ...
    >
    > class X(unittest.TestCase, metaclass=__metaclass__):
    > pass


    :|

    Doing some research[0, 1] on metaclasses in 2 and 3, I have a few more
    questions...

    The first thing I was wondering was why Python doesn't complain about a
    class property that is marked as special (leading and trailing double
    underscores) but that it knows nothing about. Worse, Python 3 should be
    aware of its legacy and recognize the Python 2 metaclass syntax, even if
    only to reject it loudly. I'm pretty sure there is a reason for that,

    The second question that came up was if there is a way to keep a
    metaclass defined inside the class or if the only way is to provide it
    externally. The reason I like this in-class definition is that for my
    case of autogenerated test functions, everything is in one place which
    used to be in a loop that modified the class after its creation. Maybe
    I'm just too brainwashed by static languages though.

    To get the Python2 feeling back, I have a hack in mind that involves
    creating a baseclass which in turn provides a metaclass that invokes a
    specific function to post-initialize the class, similar to the way
    Python 2 does it automatically, but I'm wondering if there isn't
    anything better. Also PEP 3115 "Metaclasses in Python 3000"[2] seems to
    consider postprocessing of a class definition as better handled by a
    class decorator, which is something I haven't looked at yet.

    Greetings from Hamburg!

    Uli


    [0] http://mikewatkins.ca/2008/11/29/python-2-and-3-metaclasses/
    [1] http://www.artima.com/weblogs/viewpost.jsp?thread=236234
    [2] http://www.python.org/dev/peps/pep-3115/
    Ulrich Eckhardt, Apr 11, 2013
    #3
  4. Am 10.04.2013 11:52, schrieb Peter Otten:
    > It looks like this particular invocation relies on class attribute and
    > function __name__ being identical.
    >
    > Please file a bug report.


    http://bugs.python.org/issue17696

    Uli
    Ulrich Eckhardt, Apr 11, 2013
    #4
  5. On Thu, 11 Apr 2013 08:43:58 +0200, Ulrich Eckhardt wrote:


    > The first thing I was wondering was why Python doesn't complain about a
    > class property that is marked as special (leading and trailing double
    > underscores) but that it knows nothing about.


    Because that breaks backward compatibility.

    You write a class in Python 2.6 or 2.7, and make it backward compatible
    with 2.5:

    class MyIterator(object):
    def __next__(self):
    ...
    next = __next__


    Now you import it into Python 2.5, which has no idea about __next__ so it
    blows up. (For no good reason, since __next__ is harmless in 2.5.)


    > Worse, Python 3 should be
    > aware of its legacy and recognize the Python 2 metaclass syntax, even if
    > only to reject it loudly. I'm pretty sure there is a reason for that,


    That will break any future re-use of __metaclass__. It will also make
    version agnostic code much harder:


    class Meta(type):
    ...

    if sys.version >= '3':
    kwargs = {'metaclass': Meta}
    else:
    kwargs = {}

    class MyClass(object, **kwargs):
    __metaclass__ = Meta



    > The second question that came up was if there is a way to keep a
    > metaclass defined inside the class or if the only way is to provide it
    > externally. The reason I like this in-class definition is that for my
    > case of autogenerated test functions, everything is in one place which
    > used to be in a loop that modified the class after its creation. Maybe
    > I'm just too brainwashed by static languages though.


    Not in general, since the metaclass has to exist independently of the
    class.

    The class is an instance of your metaclass. That means that the metaclass
    must exist first, so it can be instantiated when you define the class.

    However, there is a loophole: Python's metaclass machinery is actually
    more general than just class-of-classes. The metaclass doesn't have to be
    a class, it can be any callable with the same function signature as the
    three-argument version of type. So despite what I said above, you *can*
    embed the metaclass in the class, if the metaclass is a function created
    with lambda (or equivalent):


    # Python 2 version
    class MyClass(object):
    __metaclass__ = (lambda name, bases, dict:
    sys.stdout.write("Spam!\n")
    or type(name, bases, dict)
    )


    # Python 3 version
    class MyClass(object, metaclass=lambda name, bases, dict:
    sys.stdout.write("Spam!\n") and type(name, bases, dict)
    ):
    pass


    But really, except as a trick, why would you do that?


    > To get the Python2 feeling back, I have a hack in mind that involves
    > creating a baseclass which in turn provides a metaclass that invokes a
    > specific function to post-initialize the class, similar to the way
    > Python 2 does it automatically, but I'm wondering if there isn't
    > anything better.


    Seems awfully convoluted and complicated. Python 3 metaclasses work
    exactly the same as Python 2 metaclasses, except the syntax for declaring
    them is slightly different. So if you had this:

    class Meta(type):
    ...

    class MyClass:
    __metaclass__ = Meta
    ...


    just change it to this, and it should work exactly the same:


    class Meta(type):
    ...

    class MyClass(metaclass=Meta):
    ...



    > Also PEP 3115 "Metaclasses in Python 3000"[2] seems to
    > consider postprocessing of a class definition as better handled by a
    > class decorator, which is something I haven't looked at yet.


    Generally, class decorators are less brain-melting than metaclasses.




    --
    Steven
    Steven D'Aprano, Apr 11, 2013
    #5
  6. On 11 April 2013 07:43, Ulrich Eckhardt <> wrote:

    > The second question that came up was if there is a way to keep a metaclass
    > defined inside the class or if the only way is to provide it externally.


    Yes, using metaclasses! I wouldn't recommend it though. Here's a
    proof of concept:

    class MyType(type):
    def __new__(meta, name, bases, attrs):
    try:
    metaclass = attrs.pop('__metaclass__')
    except KeyError:
    return type.__new__(meta, name, bases, attrs)
    else:
    return metaclass(name, bases, attrs)

    class MyObject(metaclass=MyType):
    pass


    >>> class Test(MyObject):

    .... def __metaclass__(name, bases, attrs):
    .... print("Test metaclass")
    .... return MyType(name, bases, attrs)
    ....
    Test metaclass

    --
    Arnaud
    Arnaud Delobelle, Apr 11, 2013
    #6
  7. Am 11.04.2013 10:19, schrieb Steven D'Aprano:
    > if sys.version >= '3':


    Use sys.version_info >= (3,), otherwise your code breaks when upgrading
    to Python 10 and greater. ;^)


    >> The second question that came up was if there is a way to keep a
    >> metaclass defined inside the class or if the only way is to provide it
    >> externally. [...]

    >
    > Not in general, since the metaclass has to exist independently of the
    > class.


    Thanks for your explanations, they are appreciated.


    > The class is an instance of your metaclass. That means that the
    > metaclass must exist first, so it can be instantiated when you
    > define the class.


    I don't like the approach to define the code to post-process a class
    before defining the class. It's a bit like top-posting, it messes up the
    reading order. Since I really intend to post-process the class, it seems
    that metaclasses are simply not the right tool.

    At the moment, this leaves me with two options:

    1. post-process the class

    class X:
    pass
    # attach constants to clas X
    for i in (1, 2, 3):
    setattr(X, 'f{}' % i, i)

    2. generate code inline

    class Y: pass
    # generate constants in local (class-)namespace
    for i in (1, 2, 3):
    locals()['f{}' % i] = i

    In both cases, variables (loop variable 'i') are leaked into the
    surrounding namespace, which is kind-of ugly. The second approach also
    seems a bit hackish and I can't use the class-in-definition there, which
    is limiting when you want to attach e.g. constants of type X to X.


    >> Also PEP 3115 "Metaclasses in Python 3000"[2] seems to
    >> consider postprocessing of a class definition as better handled by a
    >> class decorator, which is something I haven't looked at yet.

    >
    > Generally, class decorators are less brain-melting than metaclasses.


    Alas, they also need to be defined before the class, messing with the
    mentioned order of declaration. They can be used to call a class
    function though which then does the necessary postprocessing...

    3. post-process the class triggered with decorator

    def postprocess_class(cls):
    """invoke postprocess() on the decorated object"""
    cls.postprocess()
    del cls.postprocess
    return cls

    @postprocess_class
    class Z:
    @classfunction
    def postprocess(cls):
    # attach constants to class
    for i in (1, 2, 3):
    setattr(cls, 'f{}' % i, i)


    I guess I'll stay with variant 1 for now, since it requires the least
    amount of code and the least amount of questions from other developers here.

    Thanks everybody!

    Uli
    Ulrich Eckhardt, Apr 12, 2013
    #7
  8. On 4/12/2013 3:17 AM, Ulrich Eckhardt wrote:
    > Am 11.04.2013 10:19, schrieb Steven D'Aprano:
    >> if sys.version >= '3':

    >
    > Use sys.version_info >= (3,), otherwise your code breaks when upgrading
    > to Python 10 and greater. ;^)
    >
    >
    >>> The second question that came up was if there is a way to keep a
    >>> metaclass defined inside the class or if the only way is to provide it
    >>> externally. [...]

    >>
    >> Not in general, since the metaclass has to exist independently of the
    >> class.

    >
    > Thanks for your explanations, they are appreciated.
    >
    >
    > > The class is an instance of your metaclass. That means that the
    > > metaclass must exist first, so it can be instantiated when you
    > > define the class.

    >
    > I don't like the approach to define the code to post-process a class
    > before defining the class. It's a bit like top-posting, it messes up the
    > reading order. Since I really intend to post-process the class, it seems
    > that metaclasses are simply not the right tool.


    Using a post-processing object as a metaclass or decorator necessarily
    requires predefinition. Such objects are usually used more than once.

    For one-off postprocessing, I probably would not bother.

    > At the moment, this leaves me with two options:
    >
    > 1. post-process the class
    >
    > class X:
    > pass
    > # attach constants to clas X
    > for i in (1, 2, 3):
    > setattr(X, 'f{}' % i, i)
    >
    > 2. generate code inline
    >
    > class Y: pass
    > # generate constants in local (class-)namespace
    > for i in (1, 2, 3):
    > locals()['f{}' % i] = i


    Mutating class locals() currently works in CPython, but is explicitly
    not guaranteed to work by the language definition.

    > In both cases, variables (loop variable 'i') are leaked into the
    > surrounding namespace, which is kind-of ugly. The second approach also
    > seems a bit hackish and I can't use the class-in-definition there, which
    > is limiting when you want to attach e.g. constants of type X to X.
    >
    >
    >>> Also PEP 3115 "Metaclasses in Python 3000"[2] seems to
    >>> consider postprocessing of a class definition as better handled by a
    >>> class decorator, which is something I haven't looked at yet.

    >>
    >> Generally, class decorators are less brain-melting than metaclasses.

    >
    > Alas, they also need to be defined before the class, messing with the
    > mentioned order of declaration. They can be used to call a class
    > function though which then does the necessary postprocessing...
    >
    > 3. post-process the class triggered with decorator
    >
    > def postprocess_class(cls):
    > """invoke postprocess() on the decorated object"""
    > cls.postprocess()
    > del cls.postprocess
    > return cls
    >
    > @postprocess_class
    > class Z:
    > @classfunction
    > def postprocess(cls):
    > # attach constants to class
    > for i in (1, 2, 3):
    > setattr(cls, 'f{}' % i, i)
    >
    >
    > I guess I'll stay with variant 1 for now, since it requires the least
    > amount of code and the least amount of questions from other developers
    > here.
    Terry Jan Reedy, Apr 12, 2013
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Daniel

    Python and database unittests

    Daniel, Aug 26, 2008, in forum: Python
    Replies:
    9
    Views:
    765
    M.-A. Lemburg
    Aug 27, 2008
  2. Peter Larsen [CPH]

    Global.asax.cs and unittests

    Peter Larsen [CPH], Sep 5, 2008, in forum: ASP .Net
    Replies:
    3
    Views:
    2,042
    Steven Cheng [MSFT]
    Sep 10, 2008
  3. Cameron Simpson
    Replies:
    3
    Views:
    272
    Terry Reedy
    Apr 23, 2010
  4. Ulrich Eckhardt

    unittests with different parameters

    Ulrich Eckhardt, Nov 22, 2010, in forum: Python
    Replies:
    9
    Views:
    285
    Ulrich Eckhardt
    Nov 24, 2010
  5. Puneet Pattar
    Replies:
    1
    Views:
    130
    Brian Candler
    Dec 14, 2009
Loading...

Share This Page