Unit testing - Breaking bad habits

[Arne Vajhøj]

Lew, I disagree here. It doesn't make any sense to test such simple
things like getters and setter or any CRUD operations on the database,
especially when you use some ORM framework. That's just a waste of
(rare) time.

I agree with Lew.

Even the most trivial stuff should be unit tested.

If it is trivial then writing the unit tests should be trivial.

And even trivial code can contain bugs. I have seen setters/getters
being copy pasted and only the method name but not the field being
accessed being changed.

You should use a test coverage tool instead to decide if
the coverage level satisfies you (and your boss).

Remember that 100% of lines tested does not even guarantee that normal
flows are working (unless all your methods has CCN=1).

Arne

 

[Arne Vajhøj]

And I stand by my characterization of methods as either being "interesting"
or not. Simple getters and setters are not, therefore I don't test them. I'm
also not going to test other simple methods that are clearly correct by
inspection.

You are so lucky to work a place with so good programmers that they
never make the classic mistake of reading what was supposed to be
there instead of what is actually there ?

I would not recommend everybody to rely on that.

Arne

 

[Arne Vajhøj]

Refactoring in TDD is used only to clean up after you add stuff. When
you add, you aren't supposed to care about duplicate code or patterns or
design, only about adding the one small feature. Then, once you have
"proved" the feature (Tests pass), do a small round of clean-up. That
doesn't mean you have to refactor every test, only the part of the
design that is affected by the code that you just added.

Also, refactoring tools make it easy to clean up large code-bases in
O(1) human-time (for example, rename method does all the search/replace
for me.) The unit tests help verify that the refactoring tool did its
job appropriately.

Often not.

There are two problems:
- doing replaces in a large number of files
- whether those files are actually under your control

The first one has been solved with smart IDE's for many years.

But the last one is a real problem in many contexts.

Arne

 

[Arne Vajhøj]

I'm interested in practicing at least some level of Test Driven
Design. The problem is that I'm so set in my ways that it's hard to
see the "right" way to do this. Particularly as I do a lot of CRUD-
style Web work, my code doesn't resemble the simple examples very
much, and Mock objects isn't where I'd prefer to start.

If it was TDD then you would have started with the tests.

Here's an example app: It comes in three parts: (This is the
simplified version)
One is a Bean that pulls XML messages off a JMS queue, parses them,
and inserts the resulting data into a DB.
The second piece is a client jar that presents a DTO. When the DTO is
populated, it can be passed to a send() routine that converts the DTO
data into XML and drops it on the JMS queue.
The third piece is a simple web-based search form that searches the DB
for matching terms and displays them.

Where should/should I not use tests?
Testing that each of the above pieces "work" seems too high level for
unit testing, but many of the components are dependent on dynamic data
and/or DB.
I can test that a given DTO will generate the correct XML, but do I
want to test that in one large pass, or do I run a different test for
each DTO data element?
Likewise how do I test my XML parsing - I don't want to test that
XPath works, but testing that the data shows up in the DB seems too
late for unit testing.
And testing the web form just confuses me. I don't want to query the
DB, I don't want to test the webserver itself, but there's very little
logic to test outside of the framework. I know I can try higher-level
testing packages to test the full process, and I eventually want to,
but I'm trying to learn TDD here.

You should unit test all 3 pieces.

* setup that create test database, create test queue and put known XML
message in it
* call code
* check that database contains what it should

* setup that create test queue and populate DTO with known values
* call send
* check that message in queue contains what it should

* setup that create test database with know data
* send HTTP POST
* check response data

If you can find a good mockup implementation for database and
queue, then it would be more pure unit tests, but if not then
just go for the real thing. I am not a purist.

Arne

 

[Arne Vajhøj]

// Test setters and getters
theCell.setX(5.0);
assertTrue("bad X coordinate", 5.0, theCell.getX());
theCell.setY(10.0);
assertTrue("bad Y coordinate", 10.0, theCell.getY());

Looks fine to me!

Now I am puzzeled.

Are you saying that you want to unit test set and get as well ?

Or are there some subtleties in your code (assertTrue, floating
point comparison) ?

Arne

 

[Arved Sandstrom]

You are so lucky to work a place with so good programmers that they
never make the classic mistake of reading what was supposed to be there
instead of what is actually there ?

I would not recommend everybody to rely on that.

Arne

I'm pretty conservative with my definition of what's "clearly correct by
inspection". In any case, methods that do get tested invariably end up
pulling in others that aren't direct testing targets.

There are also certain things that I will never consider to be correct by
inspection, such as anything to do with date/time computations.

AHS

 

[Lew]

There are also certain things that I will never consider to be correct by
inspection, such as anything to do with date/time computations.

Absolutely. I'm working on date/time stuff at work and if it weren't for
JUnit my code would be riddled with errors.

 

[Arved Sandstrom]

I agree with Lew.

Even the most trivial stuff should be unit tested.

If it is trivial then writing the unit tests should be trivial.

And even trivial code can contain bugs. I have seen setters/getters
being copy pasted and only the method name but not the field being
accessed being changed.


Remember that 100% of lines tested does not even guarantee that normal
flows are working (unless all your methods has CCN=1).

Arne

I like to keep the amount of unit tests to a necessary level. The project
that I am currently one of the maintenance programmers for has 2500+
JUnit tests, and very few of them are trivial. If the original
programmers had actually put in tests for every public and protected
method we'd be in the tens of thousands for tests.

Don't forget, unit tests are also code that has to be designed,
implemented, _tested_ and _maintained_. Just 2 weeks ago I spent a day
debugging and fixing JUnit _tests_, not the code that was being tested.
This was stopping a build that was required for regression testing, and
there was some urgency. And I have plenty of real defects on my plate as
well. So I personally am not that enthused with unnecessary tests...not
as a maintainer I'm not.

Real projects also see changes to public and protected methods. If your
test package has a test for every one of these, whenever a change request
or defect fix changes public/protected methods in the tested code, the
maintenance programmer now has to change a bunch of tests...if just to
remove them or add them.

Just my two devalued cents...

AHS

 

[Mike Schilling]

Now I am puzzeled.

Are you saying that you want to unit test set and get as well ?

Or are there some subtleties in your code (assertTrue, floating
point comparison) ?

Dammit, the assertTrue's should be assertEquals. And for
completeness;

private double x;
private double y;

public double getX() { return x; }

public void setX(double x) { this.x = x; }

public double getY() { return x; }

public void setY(double x) { this.x = x; }

 

[Andrea Francia]

Often not.

There are two problems:
- doing replaces in a large number of files
- whether those files are actually under your control

The first one has been solved with smart IDE's for many years.

But the last one is a real problem in many contexts.

The refactoring used in TDD is often limited to a small chunk of code
that you are developing and whose you don't have yet distributed the
public API.

 

[Arne Vajhøj]

The refactoring used in TDD is often limited to a small chunk of code
that you are developing and whose you don't have yet distributed the
public API.

That is often less than 1/10th of the codes lifespan.

Arne

 

[Arne Vajhøj]

I like to keep the amount of unit tests to a necessary level. The project
that I am currently one of the maintenance programmers for has 2500+
JUnit tests, and very few of them are trivial. If the original
programmers had actually put in tests for every public and protected
method we'd be in the tens of thousands for tests.

Java SE is said to have had 76000 unit tests for some
version (1.4 or 1.5 I think).

Don't forget, unit tests are also code that has to be designed,
implemented, _tested_ and _maintained_. Just 2 weeks ago I spent a day
debugging and fixing JUnit _tests_, not the code that was being tested.
This was stopping a build that was required for regression testing, and
there was some urgency. And I have plenty of real defects on my plate as
well. So I personally am not that enthused with unnecessary tests...not
as a maintainer I'm not.

Not working unit tests means some code is not unit tested. That is
a risk in my book. And the fix is not to get rid of the unit tests.

Real projects also see changes to public and protected methods. If your
test package has a test for every one of these, whenever a change request
or defect fix changes public/protected methods in the tested code, the
maintenance programmer now has to change a bunch of tests...if just to
remove them or add them.

Sure. But I consider that a good thing. Start by modifying the unit
tests and the change the code.

Arne

 

[Andrea Francia]

That is often less than 1/10th of the codes lifespan.

I don't understand what you mean? What is the remaining 9/10th? Are you
talking about the manteinance?

After I released some public APIs classes I always use the TDD to
develop new methods or new classes. Maybe that during this work I need
refactoring private or package-scoped elements, I do this without problems.

The only think that should be avoided is introduce incompatibilities
with the old public API.

With "public API" I mean all the "public" declarations and all the
"protected" declarations of classes designed to subclassed.