Design advice for unit test asserters

[Gary]

I'm working on a project where I'm doing various file manipulations.
I'm not familiar with the idioms are common or good for this situation,
so I'd appreciate advice and suggestions.

Question 1: A number of checks apply to a set of files. So I'm
wondering whether to have something like:

def test_SomeTest(...):
...
self.AssertAllFilesExist(fileList)

or

def test_SomeTest(...):
...
[ self.AssertFileExists(f) for f in fileList ]

In a statically typed, tiny method design approach, the former would be
preferred. But in Python the latter seems easy enough and more
natural. And I'm wondering if I'm missing some other Python idiom that
would be even better.

Question 2: A more complicated example: My object defines its own
conditions that can be queried, e.g.

def test_SomeTest(...):
myObject.DoSomething()
self._assert(myObject.isFileTypeQ(someFile))

but the parameter someFile is often an expression that should be
encapsulated into the assertion. This gives me two possibilities: I
can define a stand-alone query that takes myObject as a parameter:

def AssertIsTypeQ(theObjectBeingTested, rawFileInfo):
someComputedFile = DoStuff(rawFileInfo)
return theObjectBeingTested.isFileTypeQ(someComputedFile)

and then call it from within various tests as:

self._assert(AssertIsFileTypeQ(myObject, someFile)

or I can define an intermediate unit test case:

class myTester(unittest.TestCase):
def AssertIsFileTypeQ(self, rawFileInfo):
someComputedFile = DoStuff(rawFileInfo)
self.theObject.isFileTypeQ(someComputedFile)

and then define my test cases as

class test_MyObjectX(myTester):

def test_Sometest():
self.theObject = ClassBeingTested()
self.theObject.DoSomething()
self.AssertIsFileTypeQ(someFile)

With these simple cases, this seems like six of one and a half dozen of
the other. But the problem grows, in that there are several of these
assertions, and they often need to be applied to lists of things. At
one point I was getting really complicated by abstracting out both the
looping and the computed file names, passing in a functor containing
the simple test and the name (for the message). I gave up on this and
now I'm leaning towards the first, but again, I wonder if I'm missing a
better or more natural way to solve this problem.

Thanks,

Gary

 

[Kent Johnson]

I'm working on a project where I'm doing various file manipulations.
I'm not familiar with the idioms are common or good for this situation,
so I'd appreciate advice and suggestions.

Question 1: A number of checks apply to a set of files. So I'm
wondering whether to have something like:

def test_SomeTest(...):
...
self.AssertAllFilesExist(fileList)

or

def test_SomeTest(...):
...
[ self.AssertFileExists(f) for f in fileList ]

Personally I avoid using list comprehensions with side-effects; if I don't actually want the list I
write it out:
for f in fileList:
self.AssertFileExists(f)

Question 2: A more complicated example: My object defines its own
conditions that can be queried, e.g.

def test_SomeTest(...):
myObject.DoSomething()
self._assert(myObject.isFileTypeQ(someFile))

but the parameter someFile is often an expression that should be
encapsulated into the assertion. This gives me two possibilities: I
can define a stand-alone query that takes myObject as a parameter:

def AssertIsTypeQ(theObjectBeingTested, rawFileInfo):
someComputedFile = DoStuff(rawFileInfo)
return theObjectBeingTested.isFileTypeQ(someComputedFile)

I would write this method to actually do the assertion:
def AssertIsTypeQ(theObjectBeingTested, rawFileInfo):
someComputedFile = DoStuff(rawFileInfo)
self.assert_(theObjectBeingTested.isFileTypeQ(someComputedFile))

and then call it from within various tests as:
self.AssertIsFileTypeQ(myObject, someFile)

I think of it as extending the standard assertions with domain-specific ones.

I often write methods called checkSomething() that do a bit of work and check the result. So I might
actually call the above method something like checkDoStuffIsFileTypeQ().

So, the convention I use is
- write assertSomething() primitives that just check a condition
- write checkSomething() methods that do some work and check the result
- build the actual tests using the above

Kent

 

[Edvard Majakari]

[ py.test ad follows ]

def test_SomeTest(...):
...
self.AssertAllFilesExist(fileList)

or

def test_SomeTest(...):
...
[ self.AssertFileExists(f) for f in fileList ]

I prefer the latter, because then you'd get error for the missing file -
otherwise you'd just know a file didn't exist (but not which, unless you
explicitly coded that in, no?)

with py.test you could just write:


def file_exists(fname):
assert os.path.isfile(fname)

def test_file_exists():

for f in fileList:
yield file_exists, f

that way you'd generate a separate test for each file in fileList, and easily
see which file would fail the test.

no comment for the rest of your posts though - I'm still learning how to
create good unit tests myself (and I also see you're using standard unit test
modules - I'm already seduced by the no-API py.test library...)

--
# Edvard Majakari Software Engineer
# PGP PUBLIC KEY available Soli Deo Gloria!

$_ = '456476617264204d616a616b6172692c20612043687269737469616e20'; print
join('',map{chr hex}(split/(\w{2})/)),uc substr(crypt(60281449,'es'),2,4),"\n";

 

[Gary]

First, thanks to both Kent and Edvard for useful comments. I certainly
need to consider whether it make sense to switch to py.test at this
time; its simplicity is attractive.

In response to Edvards question:

self.AssertAllFilesExist(fileList) ....

def test_SomeTest(...):
...
[ self.AssertFileExists(f) for f in fileList ]

I prefer the latter, because then you'd get error for the missing file -
otherwise you'd just know a file didn't exist (but not which, unless you
explicitly coded that in, no?)

Yes, my intent is that AssertAllFilesExist(fileList) would be something
like the following, more or less:

def AssertAllFilesExist(fileList):
for f in fileList:
self.assert_(os.path.isfile(f) and not os.path.islink(f),
"File %s is missing" % f)

It's the smarts in these that caused me to instantly see applicability
of the Template Pattern - which became quite a distraction to me,
because after doing it, I think it was obviously unnecessary overkill.

Gary

 

[Edvard Majakari]

Hi again,

First, thanks to both Kent and Edvard for useful comments.
I certainly need to consider whether it make sense to switch to py.test at
this time; its simplicity is attractive.

For what it's worth, I've also coded a simple py.test skeleton generator,
which does have some funky features like creating working tests using existing
doctest strings in the module. For simple cases, tests work out of the box,
but mostly I use it to save typing. Of course, you'd be better off using TDD,
but the program can help you there too; it is able to create a module skeleton
out of py.test suite, though it is not very mature yet. You can grab it from
http://majakari.net/dl/pytestgen/ if you wish to try it.

--
# Edvard Majakari Software Engineer
# PGP PUBLIC KEY available Soli Deo Gloria!
One day, when he was naughty, Mr Bunnsy looked over the hedge into Farmer
Fred's field and it was full of fresh green lettuces. Mr Bunnsy, however, was
not full of lettuces. This did not seem fair. --Mr Bunnsy has an adventure