JUnit et al approach - criticisms

[Timbo]

Thanks for the response. I think it's an odd distinction, myself, to say
"this is an integration test, because it depends on a 20-line class I wrote,
while this is a pure unit test because it depends only on the JDK classes,
the Apache Commons, and Xerces", but we can disagree about that.

True, it is odd, but then where is the distinction? 50 lines? 100
lines? 1000? Maybe a metric related to the data structures used?
This way it is clear cut and easy to identify, and it fits with
the strict definition of 'unit testing'.

Regarding the 3rd party classes, one doesn't test them as part of
their unit/integration/system testing, so they don't come into the
picture (i.e. they are assumed correct by the fact that you are
not trying to find faults in them).

 

[Timbo]

Timbo wrote:


Its impractical to test any class in isolation, as we'd have to wrap all
Java & 3rd party libraries in order to fake/mock them to do so (never
mind about Object.class). If your class uses a String object - you
don't stub that do you? There's no difference between those classes and
your own.

There is a difference. Those classes are assumed to be correct for
the purpose of testing (i.e. you aren't trying to find faults in
them, so you aren't testing them). Third-party classes are not
treated as units in your code, and therefore need not be stubbed.
Similarly, if you are relying on a previously written and verified
internal library, you don't include that in the testing of a
system you are developing.

maybe have missed something I said.... those command classes were
*extracted* from the single class. Every line of code within the
original single class was covered by a unit test. Every scenario for
that single class was captured as a unit test.

Sorry, I didn't miss it, I just wasn't thinking about that when I
wrote that paragraph So, as I mentioned above, the
'top-level' tests are testing outside of the unit boundary, so its
not really a unit test.

Its just that part of TDD is 'refactoring' where we look at the current
design and improve things. In this case, we saw that the single class
was not conforming to Single Responsibility Principle, so we extracted
those hidden classes from within it.
The result is several small classes, where every line of code for all of
them is still covered by tests for the original class - in fact we don't
touch those tests - they create a safety net for our refactorings.

And doing so is a great way to perform refactoring! Whether or not
you refactor your tests after that is a decision that must be
made, and if I'm to be honest, I probably wouldn't bother.

So, the choice we made was to separate the logic into discrete classes,
but we could have left them as one class - do you really think this
means the previous unit test is now an integration test?

Strictly, yes. Otherwise it becomes very difficult to make a
distinction between a unit test and an integration test.

Tim

 

[Mike Schilling]

True, it is odd, but then where is the distinction? 50 lines? 100 lines?
1000? Maybe a metric related to the data structures used? This way it is
clear cut and easy to identify, and it fits with the strict definition of
'unit testing'.

Making that definition useless, IMHO.

Regarding the 3rd party classes, one doesn't test them as part of their
unit/integration/system testing, so they don't come into the picture (i.e.
they are assumed correct by the fact that you are not trying to find
faults in them).

With Xerces, at least, that's a poor assumption.

 

[Timbo]

Making that definition useless, IMHO.

Fair enough, but nobody can ever give me a definition of unit
testing other than this, so I'm sticking with it.

With Xerces, at least, that's a poor assumption.

Do you test the Xerces libraries in your testing?

 

[Mike Schilling]

Do you test the Xerces libraries in your testing?

Implicitly, yes, and I've certainly fixed bugs in them found by my "unit"
tests.

 

[Timbo]

Implicitly, yes, and I've certainly fixed bugs in them found by my "unit"
tests.

Implicitly? From that I gather you mean that when you execute your
tests, some of the Xerces code is executed. That's not testing. By
testing, I mean select test cases, execute them, and compare
behaviour to the expected behaviour.

I think NOT assuming that the 3rd party libraries are correct
during testing is crazy. During debugging, by all means, you have
to keep an open mind, but not during testing. You'll end up with
test case explosion.