avoiding long startup time in unit testing

[Michele Simionato]

There are situations in which you have to setup a faily sophisticated
environment before running your tests. This may result in a long
startup
time. In my case (using PloneTestCase) the time taken to import all
the
needed libraries, create a plone site and set up the environment, is
so
long that practically I cannot use my good old Pentium II for running
the test cases. Notice that the problem is not in the time spent on
the
tests which is really marginal (I typically run only one or two tests,
I am not talking about running the full test suite), it is all startup
time. This is terrible for me since I usually run the tests often
(say every minute) and if I have a minute of startup time at each run
my productivity is cut in half. It is essentially the same feeling
that
using a compiled language :-(

So, you could tell me "Why don't you use your newer, much faster
laptop
and you are done with it?". The answer is that I have a special
affection
for my old laptop I did a lot of Python work with it and it is
still
fast enough for most applications.

The problem is with the testing framework. If I am just fixing a
misprint in my test case I don't want all the plone site to be
recreated from zero, I want just to re-run the fixed test. For
instance, I would like to highlight the given test with Emacs and
run it, without recreating the site each time.

I did some experiment hacking unittest and I can get something
interesting,
such as a command-line interface where you initialize the stuff only
once,
you reload the module you want to test, and you run the tests for that
module.

However, this is a hack, and I don't know how much robust it is
(probably not
very). I wonder and I ask here on the mailing list if there is already
some
testing framework that does what I am talking about.
I am interested in something general, not necessarely related to
Zope/Plone.
Thanks,

Michele Simionato

 

[John Roth]

There are situations in which you have to setup a faily sophisticated
environment before running your tests. This may result in a long
startup
time. In my case (using PloneTestCase) the time taken to import all
the
needed libraries, create a plone site and set up the environment, is
so
long that practically I cannot use my good old Pentium II for running
the test cases. Notice that the problem is not in the time spent on
the
tests which is really marginal (I typically run only one or two tests,
I am not talking about running the full test suite), it is all startup
time. This is terrible for me since I usually run the tests often
(say every minute) and if I have a minute of startup time at each run
my productivity is cut in half. It is essentially the same feeling
that
using a compiled language :-(

So, you could tell me "Why don't you use your newer, much faster
laptop
and you are done with it?". The answer is that I have a special
affection
for my old laptop I did a lot of Python work with it and it is
still
fast enough for most applications.

The problem is with the testing framework. If I am just fixing a
misprint in my test case I don't want all the plone site to be
recreated from zero, I want just to re-run the fixed test. For
instance, I would like to highlight the given test with Emacs and
run it, without recreating the site each time.

I did some experiment hacking unittest and I can get something
interesting,
such as a command-line interface where you initialize the stuff only
once,
you reload the module you want to test, and you run the tests for that
module.

However, this is a hack, and I don't know how much robust it is
(probably not
very). I wonder and I ask here on the mailing list if there is already
some
testing framework that does what I am talking about.
I am interested in something general, not necessarely related to
Zope/Plone.

Well, that's good because I can't talk about Zope/Plone. The best
answer I have is the standard answer from XP:

Use mock objects and simulators to make your unit tests run
fast, and then run them again (possibly overnight) with the full
production environment to insure that your mocks and simulators
didn't cover up anything.

The other thing is that it isn't that unusual to hack unittest (jUnit,
etc) to initialize a hard-to-initialize resource at the beginning of
a test run and reuse the initialized resource for all the tests. It's
not ideal because it can create hard to find inter-test dependencies,
but sometimes it's necessary to get the most tests running in the
shortest possible time.

John Roth