pyunit: remove a test case on the fly

[chris]

We have a number of TestCase classes that have multiple test methods.
We are interested in removing any of the individual test methods on the
fly (dynamically, at runtime, whatever).

We currently have an "isSupported" method in the TestCase classes that
return a bool by which the greater test harness decides whether to run
the enclosed test methods or not. We would like to have
per-test-method granularity, however, for essentially skipping that
particular test method; basically another isSupported call within the
individual methods. We took a quick look at such options as:
generating the entire test suite, then iterating through and renaming
SomeTestClass.testSomeTest to SomeTestClass.SKIPsomeTest, and seeing if
PyUnit would skip it (since the method name no longer starts with
"test"). But this seemed pretty unwieldy. We have also considered
breaking up the test class to a number of smaller test classes and
calling isSupported with that finer-granularity set of test classes.
That is probably the easiest way, but we do still want to consider
alternatives.

Here's to hoping that other folks out there have had experience with
removing test methods on the fly, and have some wisdom to share.

Cheers

 

[Konstantin Veretennicov]

We have a number of TestCase classes that have multiple test methods.
We are interested in removing any of the individual test methods on the
fly (dynamically, at runtime, whatever).

Here's a simple approach imitating NUnit's CategoryAttribute. I don't
know whether it'll work for you, it depends on what exactly
isSupported() does.

- kv


import unittest

def category(*test_categories):
tc = frozenset(test_categories)
def f(g):
if tc & frozenset(active_categories):
return g
# else return None,
# effectively removing test from TestCase
return f

active_categories = ['mac', 'nt']

class T(unittest.TestCase):

@category('mac')
def test_a(self):
print 'mac only'

@category('posix')
def test_b(self):
print 'posix only'

@category('posix', 'nt')
def test_c(self):
print 'posix or nt'

def test_d(self):
print 'platform-independent'


if __name__ == '__main__':
unittest.main()

 

[Kent Johnson]

We have a number of TestCase classes that have multiple test methods.
We are interested in removing any of the individual test methods on the
fly (dynamically, at runtime, whatever).

We currently have an "isSupported" method in the TestCase classes that
return a bool by which the greater test harness decides whether to run
the enclosed test methods or not. We would like to have
per-test-method granularity, however, for essentially skipping that
particular test method; basically another isSupported call within the
individual methods. We took a quick look at such options as:
generating the entire test suite, then iterating through and renaming
SomeTestClass.testSomeTest to SomeTestClass.SKIPsomeTest, and seeing if
PyUnit would skip it (since the method name no longer starts with
"test"). But this seemed pretty unwieldy. We have also considered
breaking up the test class to a number of smaller test classes and
calling isSupported with that finer-granularity set of test classes.
That is probably the easiest way, but we do still want to consider
alternatives.

ISTM you could write a custom TestLoader that filters the test names, then pass that loader to unittest.main(). For example, assuming a classmethod or staticmethod on the test case that filters test case names, something like this (not tested):

def MyTest(TestCase):
@staticmethod
def isSupportedTest(testName):
return True

def testSomething...

class FilteringTestLoader(TestLoader):
def getTestCaseNames(self, testCaseClass):
names = TestLoader.getTestCaseNames(self, testCaseClass)
names = filter(testCaseClass.isSupportedTest, names)
return names

unittest.main(testLoader=FilteringTestLoader())

You could do something similar with a FilteringTestSuite if that fits your harness better.

Kent