Unit Testing in C++

[earthwormgaz]

Hello,

I'm after doing some C++ unit testing, I'm using CppUnit. I like the
look of RudeMocks, but it doesn't work for Solaris/Sparc, so its no
good to me sadly.

So, I have class A, and it uses classes B and C. B and C have methods
A uses, but they're not virtual, and I don't want to hack the code so
that they are just to allow Unit Tests to work. This means that
inheriting from B and C to provide Mock classes is a non starter.

So, I've been trying to provide a mock implementation which is
basically a copy of B or C, but with the guts ripped out and swapped
for mock code. I've then tried to get the code for A, which is under
test, to include the mock versions of the classes headers it depends
on, and link to the objects built from the mock source.

The trouble is, A, B and C are all in the same directory. So, when A
does #include "B.h", it picks up the real version, even if I add the
include path to the mock header. That's because the current directory
takes precedence using VC++, it may well do with GCC and Suncc, which
I'm also using.

I've tried to get around this by not building an object file for class
A, but using A.obj which has already been built, but linking it
against my mock versions of B.obj and C.obj. This would work, except
that C.h defines C::~C(), and so A.obj ends up with a definition of
C::~C() in it, and I get multiply defined symbol link errors. I could
make sure all method definitions are in cpp files, but often having a
getter or setter inline is perfectly sensible.

How do people get around these issues of mocking out classes
effectively from a build system point of view?

 

[anon]

Hello,

I'm after doing some C++ unit testing, I'm using CppUnit. I like the
look of RudeMocks, but it doesn't work for Solaris/Sparc, so its no
good to me sadly.

So, I have class A, and it uses classes B and C. B and C have methods
A uses, but they're not virtual, and I don't want to hack the code so
that they are just to allow Unit Tests to work. This means that
inheriting from B and C to provide Mock classes is a non starter.

So, I've been trying to provide a mock implementation which is
basically a copy of B or C, but with the guts ripped out and swapped
for mock code. I've then tried to get the code for A, which is under
test, to include the mock versions of the classes headers it depends
on, and link to the objects built from the mock source.

The trouble is, A, B and C are all in the same directory. So, when A
does #include "B.h", it picks up the real version, even if I add the
include path to the mock header. That's because the current directory
takes precedence using VC++, it may well do with GCC and Suncc, which
I'm also using.

you can do this:

#ifdef USE_MOCK_CLASSES
#include "Bmode.h"
#else
#include "B.h"
#endif

Define USE_MOCK_CLASSES when you want to do the testing, that is, change
your make file to do it automatically

 

[earthwormgaz]

you can do this:

#ifdef USE_MOCK_CLASSES
#include "Bmode.h"
#else
#include "B.h"
#endif

Define USE_MOCK_CLASSES when you want to do the testing, that is, change
your make file to do it automatically

I'd rather not have to change the original source code just to
accommodate mocking for unit tests. Is that actually what most people
end up doing?

It seems a bit of an 'orrible fix to me.

 

[Puppet_Sock]

I'd rather not have to change the original source code just to
accommodate mocking for unit tests. Is that actually what most people
end up doing?

So, your plan is to do unit testing without writing any code?

Let us know how that works out.
Socks

 

[earthwormgaz]

So, your plan is to do unit testing without writing any code?

Let us know how that works out.
Socks

I'm expecting to write code, in the form of test code, and mock
objects. I'm hoping not to have to change the classes under test just
to test them.

 

[James Kanze]

[...]
And if you know the problem, you know the way to fix it.

So you end up testing something different than what you deliver?
Not a good idea, by any means.

I'd rather not have to change the original source code just to
accommodate mocking for unit tests. Is that actually what most people
end up doing?

It seems a bit of an 'orrible fix to me.

It is.

It's not too clear to me what your deliverables are, but
generally, you should organize your directory structure so that
everything in a given directory can be tested together. If you
need a mock something, then that something should normally be
from a different directory, and not part of the code being
developed in the same directory.

If you need mock object (which you should avoid if
possible---but it isn't always possible on larger projects), you
should arrange for them at the start, putting them in a separate
directory. If the mock objects don't contain any templates or
inline functions, you use the standard header for it, and link
against a locally defined mock library. If B contains templates
or inline functions (both things to be avoided as much as
possible exactly for these sort of reasons), you're probably
stuck with having to compile against a different header file as
well (again, in the mock directory), but beware that this
seriously affects the reliability of your tests, in a negative
way.

To pick up the mock objects, of course, you just add the
appropriate -L (and -I, if you can't avoid it) in the compiler
command (before the other -L or -I).

 

[earthwormgaz]

It's not too clear to me what your deliverables are, but
generally, you should organize your directory structure so that
everything in a given directory can be tested together. If you
need a mock something, then that something should normally be
from a different directory, and not part of the code being
developed in the same directory.

Hmm, I'd come to think that I wanted to mock everything around the
class under test. You seem to be saying mock around the system that
includes the class under test, unless you have to do otherwise.

This is where the unit test/integration test line blurs ...

Say then that classes B and C interface into someLib:, you'd provide
a mock of D, and use test hooks from that?

 

[Noah Roberts]

I'd rather not have to change the original source code just to
accommodate mocking for unit tests. Is that actually what most people
end up doing?

I try not to but sometimes it's the only way. It certainly points to
strong coupling when you have to trick your source into allowing a mock,
and maybe you want to do something about that, but sometimes that's just
the way it is.

You can't write perfect code. You shouldn't expect it and you shouldn't
try. Just do the best you can under the conditions you find yourself in.

 

[Noah Roberts]

Hmm, I'd come to think that I wanted to mock everything around the
class under test. You seem to be saying mock around the system that
includes the class under test, unless you have to do otherwise.

I don't think the person you are replying to fully understands the
purpose of unit tests. You definitely do want to test each unit
individually; there should be a test for each public class at the least.
This often involves mock objects.

 

[Greg Herlihy]

I've tried to get around this by not building an object file for class
A, but using A.obj which has already been built, but linking it
against my mock versions of B.obj and C.obj. This would work, except
that C.h defines C::~C(), and so A.obj ends up with a definition of
C::~C() in it, and I get multiply defined symbol link errors. I could
make sure all method definitions are in cpp files, but often having a
getter or setter inline is perfectly sensible.

How do people get around these issues of mocking out classes
effectively from a build system point of view?

It sounds like your problem could be fixed object by more carefully
controlling the visiblity of each module's symbols.

There should not be any problem with A.obj having linked in a copy of
C's destructor. The problem occurs when A.obj then -exports- C's
destructor (that it imported from C.obj) for other modules to link
against. So with A.obj and C.obj both claiming to have C's destructor,
the linker does not know which one to choose - and reports an error.

I'm assuming that each .obj file is configured to export of all of its
(global) symbols. In that case, I would reconfigure the .obj projects
to export no symbols by default. Next, I would then label (in the
source code), each of A's (public) methods with the appropriate
attribute for exporting (probably "dllexport" or similar). So, by
having A.obj, B.obj and C.obj exporting only the public functions of
their respective classes - there should be no more multiply-defined
symbols and the unit test modules and the program modules should link
together successfully.

Greg

 

[Noah Roberts]

It sounds like your problem could be fixed object by more carefully
controlling the visiblity of each module's symbols.

There should not be any problem with A.obj having linked in a copy of
C's destructor. The problem occurs when A.obj then -exports- C's
destructor (that it imported from C.obj) for other modules to link
against. So with A.obj and C.obj both claiming to have C's destructor,
the linker does not know which one to choose - and reports an error.

I'm assuming that each .obj file is configured to export of all of its
(global) symbols. In that case, I would reconfigure the .obj projects
to export no symbols by default. Next, I would then label (in the
source code), each of A's (public) methods with the appropriate
attribute for exporting (probably "dllexport" or similar). So, by
having A.obj, B.obj and C.obj exporting only the public functions of
their respective classes - there should be no more multiply-defined
symbols and the unit test modules and the program modules should link
together successfully.

The problem I see here is that A will call it's version of C::~C(). If
the OP is experiencing this problem I have a feeling they are overriding
even the destruction behavior of the C object (and they pretty much have
to if using a mock object, no?). So they could experience some
seriously undefined behavior if they go about things in the way you
describe.

 

[James Kanze]

Hmm, I'd come to think that I wanted to mock everything around
the class under test. You seem to be saying mock around the
system that includes the class under test, unless you have to
do otherwise.

It depends on the context. It's usually acceptable to use fully
tested lower level components. (You probably wouldn't want to
mock std::vector, for example.)

This is where the unit test/integration test line blurs ...

The unit test tests "units". In some cases, one class is one
unit. In other case, not. I find that putting each "unit" in a
separate directory is a good policy, so unit tests can be
implemented at the directory level.

Say then that classes B and C interface into someLib:, you'd
provide a mock of D, and use test hooks from that?

It depends. As I said, if someLib: is a lower
level---something you can more or less consider "system" at the
level you're working at---then just use it. Mocking becomes
important in two cases: when the other library is also part of
the application, and can't be considered stable, and when the
other library provides an interface to some external resources,
which either aren't present, or can't be suitably controlled to
generate all of your test cases.

From experience, I find that in well designed applications, the
first is rather rare (but the larger the application, the more
often it occurs).

 

[James Kanze]

I don't think the person you are replying to fully understands the
purpose of unit tests.

That from you?

You definitely do want to test each unit individually; there
should be a test for each public class at the least.

Hmmm, the name is "unit test", not class test. You want to test
each unit more or less individually. On the other hand, you can
generally consider lower level stuff as "part of the system".
(I never try to mock std::vector, for example. I just suppose
that it works.)

This often involves mock objects.

Not if your code is well designed and well engineered. It
sometimes involves mock objects, but I find that with good
design, it this isn't the case that frequently.

 

[Ian Collins]

Not if your code is well designed and well engineered. It
sometimes involves mock objects, but I find that with good
design, it this isn't the case that frequently.

Something is wrong here, I agree 100% with James on a unit test issue!

It looks like the OP is facing a common problem caused by adding unit
tests after the fact, rather than designing them in form day -1.

 

[Ian Collins]

I try not to but sometimes it's the only way. It certainly points to
strong coupling when you have to trick your source into allowing a mock,
and maybe you want to do something about that, but sometimes that's just
the way it is.

It's seldom if ever the only way. It might be the easy way, bit it is
definitely a design smell.

 

[James Kanze]

Something is wrong here, I agree 100% with James on a unit
test issue!

It looks like the OP is facing a common problem caused by
adding unit tests after the fact, rather than designing them
in form day -1.

I'd express it a bit differently: if adding unit tests after the
fact does cause problems, then it's often (but not always) a
symptom of poor design and unnecessary coupling. Which in the
long run, causes problems not just for testing, but for other
things as well.

There is one point that hasn't been mentionned (and I'm pretty
sure you'd agree with me on it): classes which provide low level
services should generally be designed so that higher level code
can use them directly in there tests. Thus, a class which wraps
a socket interface should have a "debug mode" in which test code
(in the client tests) can trigger all possible error conditions.
And the global operator new/operator delete you use in your
tests should provide a mechanism to force an std::bad_alloc at
will (as well as checking for memory leaks, etc.). In other
words, suppliers of low level components should expect clients
to want to unit test, and provide the necessary hooks.

 

[Ian Collins]

I'd express it a bit differently: if adding unit tests after the
fact does cause problems, then it's often (but not always) a
symptom of poor design and unnecessary coupling. Which in the
long run, causes problems not just for testing, but for other
things as well.

Good point.

There is one point that hasn't been mentionned (and I'm pretty
sure you'd agree with me on it): classes which provide low level
services should generally be designed so that higher level code
can use them directly in there tests. Thus, a class which wraps
a socket interface should have a "debug mode" in which test code
(in the client tests) can trigger all possible error conditions.

Either that, or use virtual methods to enabled test variants to be
derived from them. On one project I work we use a generic
StreamTerminal class with various variants (Serial, UDP, TCP etc..) and
we use a DummyStreamTerminal variant for testing.

And the global operator new/operator delete you use in your
tests should provide a mechanism to force an std::bad_alloc at
will (as well as checking for memory leaks, etc.). In other
words, suppliers of low level components should expect clients
to want to unit test, and provide the necessary hooks.

Agreed.

 

[Noah Roberts]

Good point.

I disagree. It isn't that good of a point. Unit testing is meant to be
done ahead of the time and guides the design process. Even if you're
not going to be building the test ahead of time you should be thinking
about how you will.

Your original point was well placed. What Kanze is saying is sort of
true, but doesn't change anything with regard to your original point and
simply places the unit test coding in the wrong place: after production
coding.

 

[Noah Roberts]

It's seldom if ever the only way. It might be the easy way, bit it is
definitely a design smell.

I never said it wasn't. In legacy code issues (where complete product
lines where developed and designed without unit testing) it can
certainly be the only way to get something under test. At times like
that you absolutely have to cut corners or you can be left refactoring
HUGE chunks of code without the support of unit tests.

 

[James Kanze]

I disagree. It isn't that good of a point. Unit testing is
meant to be done ahead of the time and guides the design
process.

And how do you know what to test if you haven't done some high
level design? How do you even know what other classes might
exist, to interact with them?

Even if you're not going to be building the test ahead of time
you should be thinking about how you will.

Testing is part of the overall quality proceedure, and
certainly, one should be thinking about it up front. But you
still need to think about what the class is supposed to do, and
how it interacts with the other components, before you can write
either the class or its tests.

Your original point was well placed. What Kanze is saying is
sort of true, but doesn't change anything with regard to your
original point and simply places the unit test coding in the
wrong place: after production coding.

When the actual code gets written is largely irrelevant. If
writing the tests first helps you to think about how to design
the class internals, fine. But it's hardly a critical issue.
(And I don't see the relationship with whether you need to hack
some sort of struts, rather than using the actual classes.)