Need help designing some JUnit tests

[Rhino]

I'm getting more and more comfortable with JUnit after a long absence
from it but I'm still struggling with composing tests for some
situations. If anyone can help with any of these situations, I'd love to
hear your suggestions.

--------------------------------

Scenario 1 - Moving Targets

What is the best way to test a method whose output is unpredictable
because that output is a moving target? For example, I have a method that
uses Locale.getAvailableLocales() to display all of the Locales on the
current JVM. It works fine but how do I write a JUnit test, given that an
upgrade to the JVM could introduce additional Locales? I'd have the same
kind of problem if a method was returning a list of area codes or cell
phone providers. There must be some standard way of writing a JUnit test
for that but I'm drawing a blank.

--------------------------------

Scenario 2 - Constructors

Given a hypothetical class named Foo where the constructor is:

public Foo () {
//anything from no code at all to umpteen lines
}

is this adequate as a JUnit test?

public void testFoo() {

Foo myFoo = new Foo();
if (!myFoo instanceof Foo) fail ("Failed to instantiate Foo");
}

If not, what would be a better test?
--------------------------------

Scenario 3 - getInstance()

Given a hypothetical class named Fuzz where the constructors and
getInstance() methods are:

private Fuzz() {
// do something
}

private Fuzz(Locale locale) {
// do something
// initialize instance variable for locale
}

public getInstance() {

return Foo();
}

public getInstance(Locale locale) {

return Foo(locale);
}

Is it necessary to write JUnit tests for the constructors, given that
they are private?

Would the following be adequate as JUnit tests for the getInstance()
methods?

public void testGetInstance() {

Fuzz fuzz = Fuzz.getInstance();
if (!fuzz instanceof Fuzz) fail("Failed to instantiate Fuzz");
}

public void testGetInstancelocale() {

Fuzz fuzz = Fuzz.getInstance(new Locale("FR", "fr"));
if (!fuzz instanceof Fuzz) fail("Failed to instantiate Fuzz"):
}

If not, what tests would be better?
--------------------------------

Lastly - and thanks for bearing with me this far - is it normal practice
to write JUnit tests for all methods that are in parent classes of the
class being tested? For example, while writing tests for one of my
classes, the wizard in Eclipse asked if I wanted to produce test methods
to test methods of Object like equals() and wait(). Should I be writing
tests for those methods as well as for those in my own class? I'm
inclined to assume that the methods in Object, for example, have already
been thoroughly tested and further testing by me seems redundant. Even
parent classes that I myself wrote would presumably have had their own
sets of JUnit Tests. Then again, the Eclipse developers presumably put
that functionality in there for a reason so I'm left unclear about
whether I should be trying to write tests for them too.

 

[Lew]

I'm getting more and more comfortable with JUnit after a long absence
from it but I'm still struggling with composing tests for some
situations. If anyone can help with any of these situations, I'd love to
hear your suggestions.

There may be several "right" answers to your questions. I will take a stab at
some of them.

--------------------------------

Scenario 1 - Moving Targets

What is the best way to test a method whose output is unpredictable
because that output is a moving target? For example, I have a method that

When the target "moves", write new tests.

A unit test should test the behaviors of the methods given the domain of
inputs and range of desired outputs or responses. Thus, given your scenario:

uses Locale.getAvailableLocales() to display all of the Locales on the
current JVM. It works fine but how do I write a JUnit test, given that an
upgrade to the JVM could introduce additional Locales? I'd have the same

Your current JVM doesn't have new Locales, so writing for ones that don't
exist is clearly not possible. However, you can use
<http://java.sun.com/javase/6/docs/api/java/util/Locale.html#getAvailableLocales()>

to return an array of all currently available locales, and for-loop through
them in your unit test:

for ( Locale loc : Locale.getAvailableLocales() )
{
assertTrue( "insane", sanityCheck( loc ));
}

kind of problem if a method was returning a list of area codes or cell
phone providers. There must be some standard way of writing a JUnit test
for that but I'm drawing a blank.

--------------------------------

Scenario 2 - Constructors

Given a hypothetical class named Foo where the constructor is:

public Foo () {
//anything from no code at all to umpteen lines
}

is this adequate as a JUnit test?

public void testFoo() {

Foo myFoo = new Foo();
if (!myFoo instanceof Foo) fail ("Failed to instantiate Foo");
}

If not, what would be a better test?

No, it's ridiculous. It is literally impossible for 'new Foo()' to return an
instance of something that is not an instance of Foo, so checking for that is
wacky. Checking that the constructor does not throw an exception is what you
want.

--------------------------------

Scenario 3 - getInstance()

Given a hypothetical class named Fuzz where the constructors and
getInstance() methods are:

private Fuzz() {
// do something
}

private Fuzz(Locale locale) {
// do something
// initialize instance variable for locale
}

public getInstance() {

Where is your return value?

This won't even compile.

return Foo();
}

public getInstance(Locale locale) {

return Foo(locale);
}

Where is your return value?

This won't even compile.

Is it necessary to write JUnit tests for the constructors, given that
they are private?

Write a test for the method, after you get it to compile.

Normally you don't unit-test private or package-private methods, as they are
not part of the published contract. Unit tests usually test the public contract.

Would the following be adequate as JUnit tests for the getInstance()
methods?

public void testGetInstance() {

Fuzz fuzz = Fuzz.getInstance();
if (!fuzz instanceof Fuzz) fail("Failed to instantiate Fuzz");
}

If 'getInstance()', for which you did not show a return type, has a return
type of 'Fuzz', then testing that the returned value is an instance of that
type is wacky. The compiler guarantees it, so a run-time test is silly. You
want to test what happens at run time, i.e., that the returned value exists,
is not null, and there was no exception.

Actually, it's also valid to test that there is an exception thrown if initial
conditions demand it. For example, testing a constructor that takes an
argument that must not be null, you might pass a null argument and confirm the
'IllegalArgumentException' from the constructor.

It's just plain foolish to test that the type of the result matches what the
compiler already guarantees is the type of the result. Unit tests are for
run-time behaviors.

public void testGetInstancelocale() {

Fuzz fuzz = Fuzz.getInstance(new Locale("FR", "fr"));
if (!fuzz instanceof Fuzz) fail("Failed to instantiate Fuzz"):
}

If not, what tests would be better?
--------------------------------

Lastly - and thanks for bearing with me this far - is it normal practice
to write JUnit tests for all methods that are in parent classes of the
class being tested? For example, while writing tests for one of my
Depends.

classes, the wizard in Eclipse asked if I wanted to produce test methods
to test methods of Object like equals() and wait(). Should I be writing

Not sure how you'd write a test for 'wait()'. Writing a test for 'equals()'
makes sense IF the class overrides 'equals()', in which case you'd also test
that 'hashCode()' and 'toString()' match, as should 'compareTo()' if the class

tests for those methods as well as for those in my own class? I'm
inclined to assume that the methods in Object, for example, have already
been thoroughly tested and further testing by me seems redundant. Even

Depends. Did you override them?

parent classes that I myself wrote would presumably have had their own
sets of JUnit Tests. Then again, the Eclipse developers presumably put
that functionality in there for a reason so I'm left unclear about
whether I should be trying to write tests for them too.

The reason is that you routinely override methods like 'equals()' (and
'hashCode()', 'toString()' and 'compareTo()' with it).

 

[Eric Sosman]

[...]
Scenario 2 - Constructors

Given a hypothetical class named Foo where the constructor is:

public Foo () {
//anything from no code at all to umpteen lines
}

is this adequate as a JUnit test?

public void testFoo() {

Foo myFoo = new Foo();
if (!myFoo instanceof Foo) fail ("Failed to instantiate Foo");
}

This is inadequate because it won't compile. Insert the
missing parentheses and it's not inadequate, but it's pointless:
if the `new' completes without throwing an exception, `myFoo'
*will* be an instance of Foo -- in fact, it will be an instance
of Foo exactly, and not even of a Foo subclass.

For constructors, what you want to test is that they throw
exceptions when they're supposed to (e.g., InvalidArgumentException),
and that the newly-constructed object satisfies all the invariants
it's supposed to. In your no-argument Foo() constructor exceptions
seem unlikely (possible, though: think HeadlessException), but you
might check `myFoo.getHeight() * myFoo.getWidth() == myFoo.getArea()'
or whatever.

Scenario 3 - getInstance()

Given a hypothetical class named Fuzz where the constructors and
getInstance() methods are:

private Fuzz() {
// do something
}

private Fuzz(Locale locale) {
// do something
// initialize instance variable for locale
}

public getInstance() {

return Foo();
}

public getInstance(Locale locale) {

return Foo(locale);
}

Is it necessary to write JUnit tests for the constructors, given that
they are private?

No, for two reasons. First, since the constructors are private
you can't call them anyhow. Second, the code won't compile.

Would the following be adequate as JUnit tests for the getInstance()
methods?

public void testGetInstance() {

Fuzz fuzz = Fuzz.getInstance();
if (!fuzz instanceof Fuzz) fail("Failed to instantiate Fuzz");
}

Inadequate because (and this is becoming tiresome; I *wish*
people would stop posting garbage and pretending it's code samples!)
the code won't compile.

You should test that the factory method behaves as advertised.
If it can return null under some circumstances, you should check that
it does so when it's supposed to and does not when it isn't. If it
returns non-null, the thing returned will necessarily be of the type
declared for the factory method -- but not necessarily of that "exact"
type, as it might be a subclass or any arbitrary implementation of an
interface type. That may make a difference in what you test.

public void testGetInstancelocale() {

Fuzz fuzz = Fuzz.getInstance(new Locale("FR", "fr"));
if (!fuzz instanceof Fuzz) fail("Failed to instantiate Fuzz"):
}

(Broken) won't (record) compile (broken) won't (record) compile ...

If not, what tests would be better?

Something that compiles would be better. "Hello, world!" for
example, would be better -- not much of a test, I'll grant, but it
would be better.

Lastly - and thanks for bearing with me this far - is it normal practice
to write JUnit tests for all methods that are in parent classes of the
class being tested? For example, while writing tests for one of my
classes, the wizard in Eclipse asked if I wanted to produce test methods
to test methods of Object like equals() and wait(). Should I be writing
tests for those methods as well as for those in my own class? I'm
inclined to assume that the methods in Object, for example, have already
been thoroughly tested and further testing by me seems redundant. Even
parent classes that I myself wrote would presumably have had their own
sets of JUnit Tests. Then again, the Eclipse developers presumably put
that functionality in there for a reason so I'm left unclear about
whether I should be trying to write tests for them too.

I'd say it's unnecessary to test wait() and notify() and other
final methods of Object. More generally, it's probably unnecessary
to test final methods of any superclass.

But equals() is not final, and if the class being tested has its
own equals() you should test it. (Note that it's extremely rare to
inherit equals() from a superclass unless you're inheriting it all
the way from Object undisturbed.) If you were writing tests for
Integer, you might test `new Integer(42).equals(new Integer("42"))',
for example, and `! new Integer(42).equals(new Integer("-42"))'.

 

[Rhino]

There may be several "right" answers to your questions. I will take a
stab at some of them.


When the target "moves", write new tests.

A unit test should test the behaviors of the methods given the domain
of inputs and range of desired outputs or responses. Thus, given your
scenario:


Your current JVM doesn't have new Locales, so writing for ones that
don't exist is clearly not possible. However, you can use
<http://java.sun.com/javase/6/docs/api/java/util/Locale.html#getAvailab
leLocales()>

to return an array of all currently available locales, and for-loop
through them in your unit test:

for ( Locale loc : Locale.getAvailableLocales() )
{
assertTrue( "insane", sanityCheck( loc ));
}

Actually, my getLocales() method is really just a convenience method that
massages the results of Locale.getAvailableLocales() itself.

Just to be sure I'm using the term "convenience method" correctly, I'm
referring to a method I write that uses existing Java API methods but
that combines several lines of code into one or two. For example, since I
prefer my Locales list to be in alphabetical order, I've written this:

public Map<String, String> getLocales() {

Locale[] listOfLocales = Locale.getAvailableLocales();

Map<String, String> locales = new TreeMap<String, String>();
for (Locale singleLocale : listOfLocales) {
locales.put(singleLocale.toString(), singleLocale.getDisplayName
(locale));
}

return locales;
}

As such, I don't know how to do a JUnit test on it, specifically how to
generate an expected result that can be compared to my actual result. It
seems self-evident that I have to get my expected result in a different
way than I get the actual result, otherwise, I'm not proving anything.

But the same situation applies to things that aren't based on convenience
methods. A list of telephone area codes or the names of all chiropractors
in Ohio or any of a thousand other results from methods are also "moving
targets". I'm really not sure how to test those. I can certainly execute
the methods and prove that they didn't throw an exception but how can I
verify that they are giving full and complete information?

Or is it the case that such a method CAN'T have its accuracy tested in
this way and no such attempt should be made? Is it enough to prove that
the method executes without throwing an exception?

No, it's ridiculous. It is literally impossible for 'new Foo()' to
return an instance of something that is not an instance of Foo, so
checking for that is wacky. Checking that the constructor does not
throw an exception is what you want.

Ah, now it starts to make sense....

Where is your return value?

This won't even compile.

Sorry, I just hacked that together to save a minute. I probably should
have copied in a compiled example....

Where is your return value?

This won't even compile.


Write a test for the method, after you get it to compile.

Normally you don't unit-test private or package-private methods, as
they are not part of the published contract. Unit tests usually test
the public contract.

Excellent, that makes sense to me.

If 'getInstance()', for which you did not show a return type, has a
return type of 'Fuzz', then testing that the returned value is an
instance of that type is wacky. The compiler guarantees it, so a
run-time test is silly. You want to test what happens at run time,
i.e., that the returned value exists, is not null, and there was no
exception.

Actually, it's also valid to test that there is an exception thrown if
initial conditions demand it. For example, testing a constructor that
takes an argument that must not be null, you might pass a null
argument and confirm the 'IllegalArgumentException' from the
constructor.

That makes perfectly good sense to me. So, if the constructor doesn't
throw any exceptions and is public, you say that I should test that "the
returned values exists and is not null". What's my best way of doing
that? Am I right in assuming that a simple

if (Foo != null)

will cover both of those?

It's just plain foolish to test that the type of the result matches
what the compiler already guarantees is the type of the result. Unit
tests are for run-time behaviors.

Fair enough. I couldn't quite see the point of the test either but
thought I'd check it out with people that knew more than me.

Not sure how you'd write a test for 'wait()'. Writing a test for
'equals()' makes sense IF the class overrides 'equals()', in which
case you'd also test that 'hashCode()' and 'toString()' match, as


Depends. Did you override them?

Nope.

The reason is that you routinely override methods like 'equals()' (and
'hashCode()', 'toString()' and 'compareTo()' with it).

Okay, that makes sense: if I override a method inherited from the parent,
I should test the overridden method to make sure I didn't mess something
up.

 

[Rhino]

[...]
Scenario 2 - Constructors

Given a hypothetical class named Foo where the constructor is:

public Foo () {
//anything from no code at all to umpteen lines
}

is this adequate as a JUnit test?

public void testFoo() {

Foo myFoo = new Foo();
if (!myFoo instanceof Foo) fail ("Failed to instantiate Foo");
}

This is inadequate because it won't compile.

Sorry!! I just dashed that off quickly from memory and I obviously missed
a few bits.... It was really just meant to be illustrate a standard
constructor but perhaps I should have lifted one from an actual class (or
omitted the class code altogether)....

Insert the
missing parentheses and it's not inadequate, but it's pointless:
if the `new' completes without throwing an exception, `myFoo'
*will* be an instance of Foo -- in fact, it will be an instance
of Foo exactly, and not even of a Foo subclass.

For constructors, what you want to test is that they throw
exceptions when they're supposed to (e.g., InvalidArgumentException),
and that the newly-constructed object satisfies all the invariants
it's supposed to. In your no-argument Foo() constructor exceptions
seem unlikely (possible, though: think HeadlessException), but you
might check `myFoo.getHeight() * myFoo.getWidth() == myFoo.getArea()'
or whatever.

Could you expand on this last paragraph a bit. The part about verifying
that exceptions get thrown at the appropriate time are fine but I'm not
clear on what you mean by the invariants that it is supposed to satisfy.
The last sentence is almost not quite clear. Are you saying to find some
aspect of the work that is done in the constructor and verify that it
took place, so that if it is drawing a GUI component, that the component
exists and has dimensions greater that 0 x 0? What if the constructor
does very very little - maybe just a super() - or even nothing at all? In
those cases, is it sufficient to just do

if (Foo == null) fail("Constructor failed to instantiate the class");

No, for two reasons. First, since the constructors are private
you can't call them anyhow. Second, the code won't compile.


Inadequate because (and this is becoming tiresome; I *wish*
people would stop posting garbage and pretending it's code samples!)
the code won't compile.
Sorry!!

You should test that the factory method behaves as advertised.
If it can return null under some circumstances, you should check that
it does so when it's supposed to and does not when it isn't. If it
returns non-null, the thing returned will necessarily be of the type
declared for the factory method -- but not necessarily of that "exact"
type, as it might be a subclass or any arbitrary implementation of an
interface type. That may make a difference in what you test.

Can you elaborate on this a bit? Can you show me a simple example of a
constructor returning a subclass or implementation of an interface?

(Broken) won't (record) compile (broken) won't (record) compile

Again, sorry....

Something that compiles would be better. "Hello, world!" for
example, would be better -- not much of a test, I'll grant, but it
would be better.


I'd say it's unnecessary to test wait() and notify() and other
final methods of Object. More generally, it's probably unnecessary
to test final methods of any superclass.

But equals() is not final, and if the class being tested has its
own equals() you should test it. (Note that it's extremely rare to
inherit equals() from a superclass unless you're inheriting it all
the way from Object undisturbed.) If you were writing tests for
Integer, you might test `new Integer(42).equals(new Integer("42"))',
for example, and `! new Integer(42).equals(new Integer("-42"))'.

So, in a nutshell, only test the methods of the parent classes if I
overrode them; otherwise, don't worry about them. That makes sense to me!

Thanks!!

 

[Eric Sosman]

[...]
For constructors, what you want to test is that they throw
exceptions when they're supposed to (e.g., InvalidArgumentException),
and that the newly-constructed object satisfies all the invariants
it's supposed to. In your no-argument Foo() constructor exceptions
seem unlikely (possible, though: think HeadlessException), but you
might check `myFoo.getHeight() * myFoo.getWidth() == myFoo.getArea()'
or whatever.

Could you expand on this last paragraph a bit. The part about verifying
that exceptions get thrown at the appropriate time are fine but I'm not
clear on what you mean by the invariants that it is supposed to satisfy.
The last sentence is almost not quite clear. Are you saying to find some
aspect of the work that is done in the constructor and verify that it
took place, so that if it is drawing a GUI component, that the component
exists and has dimensions greater that 0 x 0? What if the constructor
does very very little - maybe just a super() - or even nothing at all? In
those cases, is it sufficient to just do

Read the Javadoc for your Foo class, and consider what
properties a newly-constructed Foo instance should satisfy.
Test that they are satisfied -- that is, test that a newly-
constructed Foo complies with its "contract."

if (Foo == null) fail("Constructor failed to instantiate the class");

Rhino, if you keep on spewing this sort of codecrap I'm going
to shove that horn of yours firmly up the orifice that spews.

Can you elaborate on this a bit? Can you show me a simple example of a
constructor returning a subclass or implementation of an interface?

No, because a constructor cannot do such a thing. But you seemed
to be talking about a factory method (although it's hard to be sure
from reading your codecrap), and a factory method -- any method, in
fact -- can return anything compatible with its declared type. Wasn't
it you who had the problem with SpinnerNumberModel recently, where the
getNumber() method sometimes returned a Short, sometimes a Long,
sometimes something else?

So, in a nutshell, only test the methods of the parent classes if I
overrode them; otherwise, don't worry about them. That makes sense to me!

There's a subtle point there, a conflict between "black box"
and "clear box" testing. The only way you can *know* that a subclass
inherits a non-final method rather than overriding it is to peek
into the subclass' implementation (either by looking at the source
or by using reflection). But what if somebody comes along next week
and decides to override a method you decided not to test, on the
grounds that it was not overridden?

One thing you might do is run some of Super's unit tests on Sub
instances. Another might be to include a "sanity check" test in your
Sub, something that reflects on Sub and verifies that the methods
you've chosen not to test are in fact inherited.

Finally, you've got to realize that unit testing, important as it
is, is not the be-all and end-all of verifying correctness.

 

[Jim Janney]

I'm getting more and more comfortable with JUnit after a long absence
from it but I'm still struggling with composing tests for some
situations. If anyone can help with any of these situations, I'd love to
hear your suggestions.

--------------------------------

Scenario 1 - Moving Targets

What is the best way to test a method whose output is unpredictable
because that output is a moving target? For example, I have a method that
uses Locale.getAvailableLocales() to display all of the Locales on the
current JVM. It works fine but how do I write a JUnit test, given that an
upgrade to the JVM could introduce additional Locales? I'd have the same
kind of problem if a method was returning a list of area codes or cell
phone providers. There must be some standard way of writing a JUnit test
for that but I'm drawing a blank.

Strictly speaking, unit testing means testing small bits of code in
isolation -- you break a program down into the smallest pieces that
can be tested independently (in Java this generally means a class) and
verify that each one satisfies its public contract. The public
contract is whatever it is that says how the class is supposed to
behave. In some languages (Eiffel and, um, uh, ...) this can be
formally specified in the language itself but more usually it's some
written description or just an idea in the programmer's mind. In Java
it may be in the Javadoc (the JRE classes are pretty good about this).

For example, the Javadoc for Locale.getAvailableLocales() specifies
that the result must contain Locale.US, so you could sensibly test for
this in a unit test. If your application depends on the presence of
other locales you may want to test for those also. Once you move past
this you're no longer doing unit testing but some sort of integration
testing. This is also valuable but the rules are different so it's
useful to remember which kind of testing you're doing.

If you have a class that finds, say, all the zip codes in a given
city, and it gives you a wrong answer, where is the problem likely to
be? In the class or in some database that it's searching? You can
verify that the class searches the data correctly but not that the
data is correct. Or perhaps you can, but that really belongs in some
other test.

 

[Lew]

Actually, my getLocales() method is really just a convenience method that
massages the results of Locale.getAvailableLocales() itself.

Just to be sure I'm using the term "convenience method" correctly, I'm
referring to a method I write that uses existing Java API methods but
that combines several lines of code into one or two. For example, since I
Yep.

prefer my Locales list to be in alphabetical order, I've written this:

public Map<String, String> getLocales() {

Locale[] listOfLocales = Locale.getAvailableLocales();

Map<String, String> locales = new TreeMap<String, String>();
for (Locale singleLocale : listOfLocales) {
locales.put(singleLocale.toString(), singleLocale.getDisplayName
(locale));

Umm, what is 'locale' in this line? I mean, it's obvious that it's a
'Locale', but what is it?

}

return locales;
}

Your 'listOfLocales' variable is, perhaps, not necessary.

That idiom also works if you want to retrieve the Locales themselves based on
name:

Map <String, Locale> locales = new TreeMap <String, Locale> ();

for( Locale loc : Locale.getAvailableLocales() )
{
locales.put( loc.getDisplayName(locale), loc );
}

or something like.

As such, I don't know how to do a JUnit test on it, specifically how to
generate an expected result that can be compared to my actual result. It
seems self-evident that I have to get my expected result in a different
way than I get the actual result, otherwise, I'm not proving anything.

Unit tests cannot do everything that the class under test does, otherwise the
unit test class would be the class under test. What unit tests do is test the
"happy path" and various corner cases to provide a high level of certainty
that the tested class will correctly handle the infinite variety of stuff
thrown at it in production. In other words, a unit test is really a sanity
check that takes care of the most likely issues. If a unit test could prevent
all possible errors, we'd never need logging.

For your case, you might test that the 'Map' has the same number of entries as
'getLocales()' has elements and that it correctly returns the right values for
some representative keys.

....

Or is it the case that such a method CAN'T have its accuracy tested in
this way and no such attempt should be made? Is it enough to prove that
the method executes without throwing an exception?

You should go farther than that.

Sorry, I just hacked that together to save a minute. I probably should
have copied in a compiled example....

Oopsie. (Giggle)

...
So, if the constructor doesn't
throw any exceptions and is public, you say that I should test that "the
returned values exists and is not null".

Non-nullity of the return value should be handled by an 'assert' in the
factory method, and therefore not necessary to test in the unit test.
Existence is already guaranteed by the successful return of the factory method.

What's my best way of doing that? Am I right in assuming that a simple

if (Foo != null)

Ummm, 'Foo' is a class, right? It better be, and therefore that line will not
compile.

will cover both of those?

How about 'assertNotNull( fuzz );'?

 

[Lew]

Rhino, if you keep on spewing this sort of codecrap I'm going
to shove that horn of yours firmly up the orifice that spews.

Rhino, the source of Eric's irritation is that despite apologizing profusely
multiple times for posting uncompilable code, you did it again anyway.

Apologies don't help much if you don't correct the behavior.

Also, never omit the curly braces in the statement bodies of 'if', 'while',
'for' and 'do...while' statements.

You'd best adhere to the discipline of providing only SSCCEs for a while
<http://sscce.org/>
until you gain more familiarity with the syntax rules for Java.

You won't know that it's an SSCCE unless you actually at least *compile* the
code you post, and really, you should run it first, too.

Doing that will help you. Failing to do that will cause the smartest and most
helpful respondents, such as Eric, not to.

 

[Rhino]

[...]
For constructors, what you want to test is that they throw
exceptions when they're supposed to (e.g.,
InvalidArgumentException), and that the newly-constructed object
satisfies all the invariants it's supposed to. In your no-argument
Foo() constructor exceptions seem unlikely (possible, though: think
HeadlessException), but you might check `myFoo.getHeight() *
myFoo.getWidth() == myFoo.getArea()' or whatever.

Could you expand on this last paragraph a bit. The part about
verifying that exceptions get thrown at the appropriate time are fine
but I'm not clear on what you mean by the invariants that it is
supposed to satisfy. The last sentence is almost not quite clear. Are
you saying to find some aspect of the work that is done in the
constructor and verify that it took place, so that if it is drawing a
GUI component, that the component exists and has dimensions greater
that 0 x 0? What if the constructor does very very little - maybe
just a super() - or even nothing at all? In those cases, is it
sufficient to just do

Read the Javadoc for your Foo class, and consider what
properties a newly-constructed Foo instance should satisfy.
Test that they are satisfied -- that is, test that a newly-
constructed Foo complies with its "contract."

Okay, you've given me a broad principle and that is all well and good but
I do a lot better with at least one concrete example. It doesn't have to
be _my_ example either, just something that resembles something I'm
doing.

Rhino, if you keep on spewing this sort of codecrap I'm going
to shove that horn of yours firmly up the orifice that spews.

I'm really not sure what you mean by "codecrap" here. I assume you're
saying that it won't compile but, in fact, this is a line from an actual
test case with the class name changed to Foo. Here is the full unit test,
WITHOUT the name change, copied and pasted directly from the test case
and I assure you that this does compile:

public void testGetInstance() {

StringUtils stringUtils = StringUtils.getInstance();
if (stringUtils == null) fail("testGetInstance() failed");
}

No, because a constructor cannot do such a thing. But you seemed
to be talking about a factory method (although it's hard to be sure
from reading your codecrap),

Yes, I do in fact mean a factory method. And here are my constructors and
getInstance() methods, copied and pasted directly from the source:

private StringUtils() {

locale = Locale.getDefault();
localizationUtils = LocalizationUtils.getInstance(locale);
locMsg = localizationUtils.getResources(locale, MSG_PREFIX);
msgFmt.setLocale(locale);
}

private StringUtils(Locale myLocale) {

locale = myLocale;
localizationUtils = LocalizationUtils.getInstance(locale);
locMsg = localizationUtils.getResources(locale, MSG_PREFIX);
msgFmt.setLocale(locale);
}

public static StringUtils getInstance() {
return new StringUtils();
}

public static StringUtils getInstance(Locale myLocale) {
return new StringUtils(myLocale);
}

and a factory method -- any method, in
fact -- can return anything compatible with its declared type. Wasn't
it you who had the problem with SpinnerNumberModel recently, where the
getNumber() method sometimes returned a Short, sometimes a Long,
sometimes something else?

Yes, that's right. Oh, so _that's_ what you meant in your previous reply!
Again, when someone answers a question only with a generality, it's never
as clear for me as when I also hear a concrete "for instance'. I'm really
not being obtuse although I imagine it must seem like it....

There's a subtle point there, a conflict between "black box"
and "clear box" testing. The only way you can *know* that a subclass
inherits a non-final method rather than overriding it is to peek
into the subclass' implementation (either by looking at the source
or by using reflection). But what if somebody comes along next week
and decides to override a method you decided not to test, on the
grounds that it was not overridden?

My theory is rather weak so I'm not really up on the meanings of "black
box" and "clear box" testing, let alone the subtle differences between
them. Also, I'm not writing the code for anyone but myself right now,
except that I _would_ like to get one project's code looking as
professional as possible so that I could present it to a prospective
client or employer as a portfolio of what I can do. (And then imitate
that in future projects as well as gradually retrofit other existing
projects with what I've learned). With that in mind, would a reasonable
employer/client likely find it acceptable that I just tested the methods
I wrote and overrode myself in my classes or are they going to see me as
the biggest idiot since the development of COBOL if I fail to observe
these subtleties?

One thing you might do is run some of Super's unit tests on Sub
instances. Another might be to include a "sanity check" test in your
Sub, something that reflects on Sub and verifies that the methods
you've chosen not to test are in fact inherited.

Finally, you've got to realize that unit testing, important as it
is, is not the be-all and end-all of verifying correctness.

I have no problem with that at all. If there are certain things I don't
need to cover in unit testing, that's perfectly fine. If you or anyone
else reading this could point me a good summary of what is and is not a
concern in unit testing, that would be very helpful. Again, I have
effectively NO formal training and what much of the on-the-job stuff I
have was learned many years ago. That means that my memory of the theory
is very incomplete at this point. In short, I don't know what the
prevailing theory is on exactly what should be covered by unit testing,
acceptance testing, regression testing, et. al. I'm not going to worry
about anything beyond unit testing for the moment but if anyone can point
me to a general - and hopefully fairly brief and example-laden -
discussion of the prevailing theories of testing, that would be very
helpful.

 

[Rhino]

Rhino, the source of Eric's irritation is that despite apologizing
profusely multiple times for posting uncompilable code, you did it
again anyway.

Apologies don't help much if you don't correct the behavior.

For what it's worth, I never imagined that anyone was actually going to
compile that code. I just provided that (sort-of) code for illustrative
purposes. I was just trying to show a very trivial constructor and/or
getInstance() so I could understand what testing was needed. Basically,
the (real) code looked so trivial that it wasn't obvious what needed to
be tested.

Also, never omit the curly braces in the statement bodies of 'if',
'while', 'for' and 'do...while' statements.

Actually, I rarely do. Again, the "code" that I posted was just to
illustrate my point.

You'd best adhere to the discipline of providing only SSCCEs for a
while <http://sscce.org/>
until you gain more familiarity with the syntax rules for Java.

You won't know that it's an SSCCE unless you actually at least
*compile* the code you post, and really, you should run it first, too.

But I will endeavour to use only real compiled code from here on.

Doing that will help you. Failing to do that will cause the smartest
and most helpful respondents, such as Eric, not to.

I'm not coming here to piss people off and I'm sorry that I have done so.
I've long been criticized for being verbose so I thought I'd keep the
length of my post down a bit by just summarizing the code rather than
copying it verbatim. That, unfortunately, backfired because it left my
meaning unclear so I will try to use "real" code exclusively from now on.

 

[Lew]

I'm really not sure what you mean by "codecrap" here. I assume you're

He's referring to the fact that the code you showed will not compile.

saying that it won't compile but, in fact, this is a line from an actual
test case with the class name changed to Foo. Here is the full unit test,

Nope. What you should have shown is the *instance* name changed to 'foo'.

WITHOUT the name change, copied and pasted directly from the test case
and I assure you that this does compile:

public void testGetInstance() {

StringUtils stringUtils = StringUtils.getInstance();
if (stringUtils == null) fail("testGetInstance() failed");
}

Rhino, Rhino, Rhino.

In the code about which Eric complained, you compared 'Foo', a *class*, to
'null'. That WILL NOT COMPILE!

In the code you assure us does compile you compare an instance to 'null'. Of
course that compiles.

Apples and oranges.

You cannot compare a class to 'null', only an instance. Classes and instances
are different things. What works for one does not necessarily work for the other.

Also, that "actual" code you show neglects the curly braces around the
statement body of the 'if'.

 

[Eric Sosman]

Rhino, the source of Eric's irritation is that despite apologizing
profusely multiple times for posting uncompilable code, you did it
again anyway.

Apologies don't help much if you don't correct the behavior.

For what it's worth, I never imagined that anyone was actually going to
compile that code. I just provided that (sort-of) code for illustrative
purposes. [...]

Nobody was probable to compile the code, not. But a fragment
in full load of the errors is not the "illustrative," it is only
offuscatorio. In order to extract the meaning from a beautiful
insignificant piece of the code, the reading of the person must
make the changes and corrections to arrive to something images that
you could have in mind and are every probability that will supply
some different thing from what has meant and that its ulterior
observations therefore will be not at all useful you.

(In other words: "Illustrate" your thoughts with garbled
utterances, and people may form opinions about your thinking.)

 

[Arved Sandstrom]

news:[email protected]:

[ SNIP ]

My theory is rather weak so I'm not really up on the meanings of "black
box" and "clear box" testing, let alone the subtle differences between
them. Also, I'm not writing the code for anyone but myself right now,
except that I _would_ like to get one project's code looking as
professional as possible so that I could present it to a prospective
client or employer as a portfolio of what I can do. (And then imitate
that in future projects as well as gradually retrofit other existing
projects with what I've learned). With that in mind, would a reasonable
employer/client likely find it acceptable that I just tested the methods
I wrote and overrode myself in my classes or are they going to see me as
the biggest idiot since the development of COBOL if I fail to observe
these subtleties?

No word of a lie, just the fact that you're writing tests already gets
you some serious points with employers or clients. And the huge majority
of people that matter will be perfectly happy with you writing tests for
a class that focus on methods that are lexically in the class. That's
what I do myself.

You perhaps misunderstood Eric - he didn't mean that there is a subtle
difference between black box and white box testing. There isn't - those
two techniques are quite different. He simply called the specific point
he was making subtle.

You don't need to be a testing god - people make FT careers out of
understanding testing. But you should know what black box and white box
testing are, for example. The Wikipedia page on Software Testing is not
a bad start, IMO. Just bear in mind that their terminology and
definitions are debatable on some specifics, but this doesn't detract
from the overall discussion (in particular the first few sentences under
Non-Functional Testing are whacked, try to gloss over them).

I have no problem with that at all. If there are certain things I don't
need to cover in unit testing, that's perfectly fine. If you or anyone
else reading this could point me a good summary of what is and is not a
concern in unit testing, that would be very helpful. Again, I have
effectively NO formal training and what much of the on-the-job stuff I
have was learned many years ago. That means that my memory of the theory
is very incomplete at this point. In short, I don't know what the
prevailing theory is on exactly what should be covered by unit testing,
acceptance testing, regression testing, et. al. I'm not going to worry
about anything beyond unit testing for the moment but if anyone can point
me to a general - and hopefully fairly brief and example-laden -
discussion of the prevailing theories of testing, that would be very
helpful.

You'll end up doing a fair bit of reading and playing with code to get a
good handle on testing overall, but it won't take all that long to get a
good grip on the basics.

I will make a few personal recommendations, which shouldn't be taken as
complete. This is based on going on ten years of J2EE work, so depending
on what kind of coding you're doing YMMV...some.

1. Code reviews/inspections and static code analysis with tools _are_ a
form of testing. Keep them in mind.

2. Integration tests are as important as unit tests. In a J2EE web
application these are indispensable - you may have hundreds or thousands
of unit tests all pass but still have integration tests not pass. You'll
hear a lot of people, myself included, refer to a specific class of
integration tests as functional tests - we mean "application functions
from the user perspective" when we use "functional" in that regard.

Examples of integration/functional tests in a J2EE web app range all the
way from ensuring that good things happen in the logic when you click
that "Transmogrify" button all the way to doing a complete pass through
a use case and making sure that your "Issue Fishing License" logic works
in Angler Administrator 2010.

Myself I use Selenium IDE/RC in conjunction with the JUnit framework to
write these kinds of tests.

3. Code coverage - Huge, IMO. How can you know that your unit tests or
integration tests (or even human-tester-driven acceptance tests) are
doing any good unless you know how much of the code is actually being
exercised? Code coverage is very simple to do, and for starters you
can't go wrong investigating Emma or Cobertura. These simply instrument
the Java bytecode, such that when the bytecode is executed (by any
mechanism) coverage counts by line/branch/method/class/package are
written to HTML or XML reports.

4. Carefully consider the issue of test data - test SQL scripts, mock
data, in-memory databases (see http://www.mikebosch.com/?p=8) etc.

5. Your tests are themselves defective. The sad truth is that if the
core source code you just wrote is riddled with defects, then so
probably are your tests. Main take-away here is, be aware that just
because all your unit tests pass, some not insignificant percentage of
those results are wrong.

As a side note, this is where the higher-level layer of integration
tests also helps - it can assist in identifying flawed unit tests.

HTH,
AHS

 

[Rhino]

Yep.

Okay, good. At least I know now that I've correctly understood what
_that_ term means. One down and a gazillion to go.... ;-)

prefer my Locales list to be in alphabetical order, I've written
this:

public Map<String, String> getLocales() {

Locale[] listOfLocales = Locale.getAvailableLocales();

Map<String, String> locales = new TreeMap<String, String>();
for (Locale singleLocale : listOfLocales) {
locales.put(singleLocale.toString(), singleLocale.getDisplayName
(locale));

Umm, what is 'locale' in this line? I mean, it's obvious that it's a
'Locale', but what is it?

That 'locale' is an instance variable that the caller can set via a
getInstance(locale) method to set the locale for the class. I'm just
trying to make my code "multilingual" so that a user who is operating in
French or German or whatever will get the output in their language.

Your 'listOfLocales' variable is, perhaps, not necessary.

That idiom also works if you want to retrieve the Locales themselves
based on name:

Map <String, Locale> locales = new TreeMap <String, Locale> ();

for( Locale loc : Locale.getAvailableLocales() )
{
locales.put( loc.getDisplayName(locale), loc );
}

or something like.

I like it I had originally tried to build the map using the locale as
the key but ran up against the fact that Locale is not Comparable so I
did it the way you see in this email. But making the DisplayName the key
and the Locale as the value works fine and is concise to boot so I'll do
it your way.

Unit tests cannot do everything that the class under test does,
otherwise the unit test class would be the class under test. What
unit tests do is test the "happy path" and various corner cases to
provide a high level of certainty that the tested class will correctly
handle the infinite variety of stuff thrown at it in production. In
other words, a unit test is really a sanity check that takes care of
the most likely issues. If a unit test could prevent all possible
errors, we'd never need logging.

Fair enough.

For your case, you might test that the 'Map' has the same number of
entries as 'getLocales()' has elements and that it correctly returns
the right values for some representative keys.

Okay, that's reasonable. Someone else suggested that I verify that the
en_US is there too, since that's part of the contract from
Locale.getAvailableLocales(). That should be enough for unit testing
then, right?

...

You should go farther than that.



Oopsie. (Giggle)


Non-nullity of the return value should be handled by an 'assert' in
the factory method, and therefore not necessary to test in the unit
test. Existence is already guaranteed by the successful return of the
factory method.

I'm not very familiar with "assert". I'll look into it on my own and add
that code.

Ummm, 'Foo' is a class, right? It better be, and therefore that line
will not compile.


How about 'assertNotNull( fuzz );'?

That would go in the test case (as opposed to the class) and wouldn't be
redundant with the 'assert' you mentioned that should go in the class
itself?

 

[Rhino]

Rhino wrote:
...

Actually, my getLocales() method is really just a convenience method
that massages the results of Locale.getAvailableLocales() itself.

Just to be sure I'm using the term "convenience method" correctly,
I'm referring to a method I write that uses existing Java API methods
but that combines several lines of code into one or two. For example,
since I prefer my Locales list to be in alphabetical order, I've
written this:

public Map<String, String> getLocales() {

Locale[] listOfLocales = Locale.getAvailableLocales();

Map<String, String> locales = new TreeMap<String, String>();
for (Locale singleLocale : listOfLocales) {
locales.put(singleLocale.toString(), singleLocale.getDisplayName
(locale));
}

return locales;
}

As such, I don't know how to do a JUnit test on it, specifically how
to generate an expected result that can be compared to my actual
result. It seems self-evident that I have to get my expected result
in a different way than I get the actual result, otherwise, I'm not
proving anything.

You seem to be assuming that a JUnit test requires an expected result.
Don't forget the assertTrue method, which lets you test arbitrary
conditions.

I'm not really familiar with assertTrue - I'm just getting back into Java
and JUnit after a gap of a few years and I was never all that fluent with
JUnit even before the gap. I'll look at the JUnit docs and see what I can
learn there about the purpose and best uses of assertTrue....

Thanks for the suggestion, I'm sure it will be helpful once I understand it
better ;-)

 

[Jim Janney]

[...]
For constructors, what you want to test is that they throw
exceptions when they're supposed to (e.g.,
InvalidArgumentException), and that the newly-constructed object
satisfies all the invariants it's supposed to. In your no-argument
Foo() constructor exceptions seem unlikely (possible, though: think
HeadlessException), but you might check `myFoo.getHeight() *
myFoo.getWidth() == myFoo.getArea()' or whatever.

Could you expand on this last paragraph a bit. The part about
verifying that exceptions get thrown at the appropriate time are fine
but I'm not clear on what you mean by the invariants that it is
supposed to satisfy. The last sentence is almost not quite clear. Are
you saying to find some aspect of the work that is done in the
constructor and verify that it took place, so that if it is drawing a
GUI component, that the component exists and has dimensions greater
that 0 x 0? What if the constructor does very very little - maybe
just a super() - or even nothing at all? In those cases, is it
sufficient to just do

Read the Javadoc for your Foo class, and consider what
properties a newly-constructed Foo instance should satisfy.
Test that they are satisfied -- that is, test that a newly-
constructed Foo complies with its "contract."

Okay, you've given me a broad principle and that is all well and good but
I do a lot better with at least one concrete example. It doesn't have to
be _my_ example either, just something that resembles something I'm
doing.

Rhino, if you keep on spewing this sort of codecrap I'm going
to shove that horn of yours firmly up the orifice that spews.

I'm really not sure what you mean by "codecrap" here. I assume you're
saying that it won't compile but, in fact, this is a line from an actual
test case with the class name changed to Foo. Here is the full unit test,
WITHOUT the name change, copied and pasted directly from the test case
and I assure you that this does compile:

public void testGetInstance() {

StringUtils stringUtils = StringUtils.getInstance();
if (stringUtils == null) fail("testGetInstance() failed");
}

No, because a constructor cannot do such a thing. But you seemed
to be talking about a factory method (although it's hard to be sure
from reading your codecrap),

Yes, I do in fact mean a factory method. And here are my constructors and
getInstance() methods, copied and pasted directly from the source:

private StringUtils() {

locale = Locale.getDefault();
localizationUtils = LocalizationUtils.getInstance(locale);
locMsg = localizationUtils.getResources(locale, MSG_PREFIX);
msgFmt.setLocale(locale);
}

private StringUtils(Locale myLocale) {

locale = myLocale;
localizationUtils = LocalizationUtils.getInstance(locale);
locMsg = localizationUtils.getResources(locale, MSG_PREFIX);
msgFmt.setLocale(locale);
}

public static StringUtils getInstance() {
return new StringUtils();
}

public static StringUtils getInstance(Locale myLocale) {
return new StringUtils(myLocale);
}

and a factory method -- any method, in
fact -- can return anything compatible with its declared type. Wasn't
it you who had the problem with SpinnerNumberModel recently, where the
getNumber() method sometimes returned a Short, sometimes a Long,
sometimes something else?

Yes, that's right. Oh, so _that's_ what you meant in your previous reply!
Again, when someone answers a question only with a generality, it's never
as clear for me as when I also hear a concrete "for instance'. I'm really
not being obtuse although I imagine it must seem like it....

There's a subtle point there, a conflict between "black box"
and "clear box" testing. The only way you can *know* that a subclass
inherits a non-final method rather than overriding it is to peek
into the subclass' implementation (either by looking at the source
or by using reflection). But what if somebody comes along next week
and decides to override a method you decided not to test, on the
grounds that it was not overridden?

My theory is rather weak so I'm not really up on the meanings of "black
box" and "clear box" testing, let alone the subtle differences between
them. Also, I'm not writing the code for anyone but myself right now,
except that I _would_ like to get one project's code looking as
professional as possible so that I could present it to a prospective
client or employer as a portfolio of what I can do. (And then imitate
that in future projects as well as gradually retrofit other existing
projects with what I've learned). With that in mind, would a reasonable
employer/client likely find it acceptable that I just tested the methods
I wrote and overrode myself in my classes or are they going to see me as
the biggest idiot since the development of COBOL if I fail to observe
these subtleties?

Speaking as someone who occasionally sits in on job interviews, I'm
delighted when we find a candidate who's written any tests at all.
Most of them haven't. We don't expect people to know more than we do
(it would be great if they did) and we all learned as we went along
and wish we understood it better.

I have no problem with that at all. If there are certain things I don't
need to cover in unit testing, that's perfectly fine. If you or anyone
else reading this could point me a good summary of what is and is not a
concern in unit testing, that would be very helpful. Again, I have
effectively NO formal training and what much of the on-the-job stuff I
have was learned many years ago. That means that my memory of the theory
is very incomplete at this point. In short, I don't know what the
prevailing theory is on exactly what should be covered by unit testing,
acceptance testing, regression testing, et. al. I'm not going to worry
about anything beyond unit testing for the moment but if anyone can point
me to a general - and hopefully fairly brief and example-laden -
discussion of the prevailing theories of testing, that would be very
helpful.

In the specific case of a utility class, what I look for is clear
Javadoc. I need to be able to decide, without wasting a lot of time,
whether a method does what I need or not, and how it handles corner
cases. For an example of how I like to see it done, go to

http://commons.apache.org/lang/api-release/org/apache/commons/lang/StringUtils.html

and look at, say, the definition for

center(java.lang.String, int)

Notice that once you have a good definition of how the method should
behave, writing a unit test for it is straightforward.

Notice also that testing a utility class is very different from
testing a program or an application framework.

 

[Tom Anderson]

3. Code coverage - Huge, IMO. How can you know that your unit tests or
integration tests (or even human-tester-driven acceptance tests) are
doing any good unless you know how much of the code is actually being
exercised? Code coverage is very simple to do, and for starters you
can't go wrong investigating Emma or Cobertura. These simply instrument
the Java bytecode, such that when the bytecode is executed (by any
mechanism) coverage counts by line/branch/method/class/package are
written to HTML or XML reports.

Out of interest, do those play well with other things which modify
bytecode, like JDO/JPA enhancers?

tom

 

[Tom Anderson]

Arved Sandstrom wrote:
...

Although it is not 100% effective, there is some value in writing, and
running, the test before implementing the feature. Start testing a
method when it is just an empty stub. At that point, a test that passes
is either useless or has a bug in it.

The true extremos work to this rule - you must always start with a failing
test, then work to make it pass.

In fact, the way we (the programmers where i work) try to work is to write
the test first, before even writing the stub of the method, then treat the
fact that the test doesn't compile as a failure, and work in from there.

tom

 

[Rhino]

I'll modify my suggestion from "Don't forget" to "Learn about".

I've already found a basic article on JUnit that talked about the
different sorts of assertXXX methods and looked at the JUnit API to get a
handle on that. It's much clearer now, thanks for the suggestion.

Here's a possibly relevant example, a method that tests an
Iterator<String> for non-empty, strictly increasing, no null elements.

/**
* Pass a non-empty String iterator with only non-null elements
* in strictly
* increasing order.
*/
private void testIteratorBehavior(Iterator<String> it) {
assertTrue(it.hasNext());
String oldElement = it.next();
assertNotNull(oldElement);
while (it.hasNext()) {
String newElement = it.next();
assertNotNull(newElement);
assertTrue(newElement.compareTo(oldElement) > 0);
oldElement = newElement;
}
}

Using this sort of technique you can test a structure for conforming
to some known rules without using an expected value.

Thanks you for the interesting suggestion. I hadn't thought of testing to
insure that the sequence was correct or that nulls were excluded and
might have struggled a bit trying to think of ways to ensure that.

Am I right in assuming that it is reasonable to bypass the sequence
testing in the case of my sorted list of Locales, given that I am using a
TreeMap which already guarantees the order? I assume you are just
including these tests to cover situations where I am generating a sorted
list out of thin air (without the benefit of the TreeXXX Collection
classes) so that I can be confident my code worked correctly?

I think I will include a test for nulls in the Set though, just in
case....