How to have unittest tests to be executed in the order they appear?

[Giampaolo Rodola']

Hi there.
Is there a way to force unittest to run test methods in the order they
appear?
I'll try to explain.
My test suite appears as something like this:

import unittest
from test.test_support import TestSkipped, run_unittest

class TestCase(unittest.TestCase):

def test_z(self):
...

def test_b(self):
...

def test_d(self):
...

if __name__ == '__main__':
run_unittest(TestCase)


I'd like the tests to be executed in the order they are defined
(test_z, test_b and finally test_d).
Is there a way to do that?

Thanks in advance.


--- Giampaolo
http://code.google.com/p/pyftpdlib/

 

[D'Arcy J.M. Cain]

Hi there.
Is there a way to force unittest to run test methods in the order they
appear?

Since they run in alphabetical order why not just rename them to the
order you want them to run? Something like this:

def test_010_z(self):
def test_020_b(self):
def test_030_d(self):

Note the numbering scheme that allows tests to be added between the
existing ones if necessary.

However, do consider writing your tests so that order is irrelevant.
If a test depends on a previous test running then it isn't a proprt
self-contained test. For one thing, you can't run a single test that
way if you wanted to.

 

[tom arnall]

No, and this is a good thing.

Your test cases should *not* depend on any state from other test
cases; they should function equally well when executed in any
arbitrary sequence. Dependencies between separate test cases (e.g.
"they only work correctly when run in a specific sequence") means
you're not isolating them properly.

Use the TestCase.setUp and TestCase.tearDown methods to handle any
fixtures needed by test cases in each class of test cases. That way,
the fixtures will be set up and torn down between every test case.
Find out about test fixtures in the documentation for unittest
<URLhttp://www.python.org/doc/lib/module-unittest>.

--
\ "All my life I've had one dream: to achieve my many goals." -- |
`\ Homer, _The Simpsons_ |
_o__) |
Ben Finney

a better approach maybe is just to write your own test harness. it's trivial
to write a minimal system, which is then a solid basis for the enhancements
which are best for you.

tom arnall
arcata

 

[Matthew Woodcraft]

No, and this is a good thing.

Your test cases should *not* depend on any state from other test
cases; they should function equally well when executed in any
arbitrary sequence. Dependencies between separate test cases (e.g.
"they only work correctly when run in a specific sequence") means
you're not isolating them properly.

So a mode to randomise the test sequence would be nice to have.

Unittest's behaviour (using alphabetical order) doesn't really help to
detect undesired dependencies (which might be bugs in the test suite or
bugs in the underlying code).

But running tests in the order they appear is often helpful: you can
put the tests for basic stuff before the tests for advanced stuff, and
then if you suddenly get seventeen failing tests, you know that the
first failure is the best bet to investigate first.

-M-

 

[Matthew Woodcraft]

Surely, since "suddenly" implies you changed one small area of the
code, that area of the code is the best place to look for what caused
the failure.

Imagine that "suddenly" immediately follows "I upgraded to etch".

-M-

 

[Roy Smith]

Surely, since "suddenly" implies you changed one small area of the
code, that area of the code is the best place to look for what caused
the failure.

Sometimes it's the environment that's changed. Yes, I know, a good unit
test doesn't depend on the environment, but in real life, that's sometimes
difficult to achieve.

 

[Nick Stinemates]

However, do consider writing your tests so that order is irrelevant.
If a test depends on a previous test running then it isn't a proprt
self-contained test. For one thing, you can't run a single test that
way if you wanted to.

Agreed!