SetUp functions for multiple test cases

[Georg Schmid]

I've just started working with unittests and already hit a snag. I
couldn't find out how to implement a setup function, that is executed
only _once_ before all of the tests. Specifically, I need this for
testing my database interface, and naturally I don't want to create a
new database in-memory and fill it with example data for every single
test case.
Obviously I could simply execute that setup function separately from
the unittesting procedure, but then there'd be no point in
implementing all my unittests in a clean manner in the first place,
apart from further problems arising due to how my IDE (Eric4) picks
test cases from project files.
Perhaps I missed something about the concept of TestSuites. Thanks in
advance!

 

[Diez B. Roggisch]

I've just started working with unittests and already hit a snag. I
couldn't find out how to implement a setup function, that is executed
only _once_ before all of the tests. Specifically, I need this for
testing my database interface, and naturally I don't want to create a
new database in-memory and fill it with example data for every single
test case.
Obviously I could simply execute that setup function separately from
the unittesting procedure, but then there'd be no point in
implementing all my unittests in a clean manner in the first place,
apart from further problems arising due to how my IDE (Eric4) picks
test cases from project files.
Perhaps I missed something about the concept of TestSuites. Thanks in
advance!

Why don't you simply store the state of the DB being set-up somewhere
(globally for example), and only execute the db-setup-code (inside setUp of
course) depending on the value of that variable?

Diez

 

[Roy Smith]

I've just started working with unittests and already hit a snag. I
couldn't find out how to implement a setup function, that is executed
only _once_ before all of the tests. Specifically, I need this for
testing my database interface, and naturally I don't want to create a
new database in-memory and fill it with example data for every single
test case.

Short answer -- there's no way to do it in the unittest framework.

The True Unit Test Zealots will argue that all tests should be 100%
independent of each other, which means there should be NO common state
between test cases. For that matter, they will also argue that unit tests
should not interface with external resources like databases. And they're
right.

On the other hand, the Real World Test Pragmatists will argue that this is
just not practical in all cases. Real programs have classes which interact
with the outside world, and they need to get tested. You could stub out
the external resource, but that's a lot of work, and may introduce as many
problems as it solves. Sometimes, a big part of what you're testing is
your understanding of the external world (i.e. "does this really work like
it's documented?").

Plus, some operations are just too expensive to do for every test case. I
don't know how long it takes to build your in-memory database. If it takes
one second, it probably makes sense to do it for every test case (unless
you've got thousands of test cases). If it takes 10 minutes, then it makes
sense to do it once and deal with the fact that you're violating True Unit
Test Dogma.

Anyway, do what I do. I run the tests with a:

if __name__ == "__main__":
blah blah

block at the bottom of the test file. Just do your "do once" setup in that
code block and store the result in a global.

You might have your setUp() method re-assign the global to an instance
variable and then your test cases can access it via self.whatever. The
reason for that is if at some point in the future you change your mind and
decide to re-build the database in setUp() for each test, you just have to
change setUp(), not touch the individual test cases.

 

[Georg Schmid]

Short answer -- there's no way to do it in the unittest framework.

The True Unit Test Zealots will argue that all tests should be 100%
independent of each other, which means there should be NO common state
between test cases.  For that matter, they will also argue that unit tests
should not interface with external resources like databases.  And they're
right.

On the other hand, the Real World Test Pragmatists will argue that this is
just not practical in all cases.  Real programs have classes which interact
with the outside world, and they need to get tested.  You could stub out
the external resource, but that's a lot of work, and may introduce as many
problems as it solves.  Sometimes, a big part of what you're testing is
your understanding of the external world (i.e. "does this really work like
it's documented?").

Plus, some operations are just too expensive to do for every test case.  I
don't know how long it takes to build your in-memory database.  If it takes
one second, it probably makes sense to do it for every test case (unless
you've got thousands of test cases).  If it takes 10 minutes, then it makes
sense to do it once and deal with the fact that you're violating True Unit
Test Dogma.

Anyway, do what I do.  I run the tests with a:

if __name__ == "__main__":
   blah blah

block at the bottom of the test file.  Just do your "do once" setup in that
code block and store the result in a global.

You might have your setUp() method re-assign the global to an instance
variable and then your test cases can access it via self.whatever.  The
reason for that is if at some point in the future you change your mind and
decide to re-build the database in setUp() for each test, you just have to
change setUp(), not touch the individual test cases.

Thanks, exactly what I needed to know.

 

[brooklineTom]

Short answer -- there's no way to do it in the unittest framework.

The True Unit Test Zealots will argue that all tests should be 100%
independent of each other, which means there should be NO common state
between test cases. For that matter, they will also argue that unit tests
should not interface with external resources like databases. And they're
right.

On the other hand, the Real World Test Pragmatists will argue that this is
just not practical in all cases. Real programs have classes which interact
with the outside world, and they need to get tested. You could stub out
the external resource, but that's a lot of work, and may introduce as many
problems as it solves. Sometimes, a big part of what you're testing is
your understanding of the external world (i.e. "does this really work like
it's documented?").

Plus, some operations are just too expensive to do for every test case. I
don't know how long it takes to build your in-memory database. If it takes
one second, it probably makes sense to do it for every test case (unless
you've got thousands of test cases). If it takes 10 minutes, then it makes
sense to do it once and deal with the fact that you're violating True Unit
Test Dogma.

Anyway, do what I do. I run the tests with a:

if __name__ == "__main__":
blah blah

block at the bottom of the test file. Just do your "do once" setup in that
code block and store the result in a global.

You might have your setUp() method re-assign the global to an instance
variable and then your test cases can access it via self.whatever. The
reason for that is if at some point in the future you change your mind and
decide to re-build the database in setUp() for each test, you just have to
change setUp(), not touch the individual test cases.

I believe that the answer sought by the OP is the TestResource class
from the original Smalltalk unit test. Sadly, the various language-
specific clones -- including the Python unittest package -- followed
the example of the JUnit hack, which elided this (for a variety of
reasons, some good, some bad). There are two major obstacles in moving
SUnit from Smalltalk to Python: (1) SUnit takes full advantage of
class objects which still have only vestigial implementations in
Python, and (2) SUnit takes full advantage of Smalltalk's restartable
exception semantics.

I've implemented an alternative unit test package, but I requires the
rest of my framework (which addresses the above two restrictions) and
is therefore not yet ready for prime time.

The basic theme of TestResource is simple enough, once the rest of the
unit test framework is properly factored:
1. Tests always run within an instance of TestSuite.
2. The "run" method of test suite (with no arguments) collects and
creates whatever resources are needed by the tests within it.
3. The run method creates an instance of TestResult to collect the
results.
4. Each TestCase instance collects whatever (existing) resources it
needs.
5. When the "run" method (in step 2 above) finishes, all resources are
finalized.

Instances of TestResource have a "setUp" and "tearDown" method, just
like TestCase. These are called once, at steps 2 and 5 above.

I built my test frame from the original SUnit from KentBeck et al and
used an excellent reference Joseph Pelrine (http://www.metaprog.com/
ESUG/TestResources.pdf) to guide my work.