Unit testing in C

[nroberts]

Something I've been trying to do and my bosses have wanted me to
continue is to apply unit testing to the code I'm working with. In
some cases this has been fairly straight forward but in others it has
not. Some of what I've tried to do to harness some of the code has
been discouraged and I'm not so sure much of what I'd do in C++ or
Java lends itself directly to C.

For example, unit testing often requires strong use of polymorphism,
abstract types, inheritance, dependency injection, mocks, etc... You
CAN do these things in C but if you were to that extent then you're
almost wasting your time and effort reimplementing C++ or Objective-C
(depending on the static/dynamic approach you want) for no good
reason. You do perhaps have a modicum of more control but beating
those languages at what they do by hand is a pretty lofty goal.

Perhaps unit testing isn't often done in C either, as might be
indicated by how different I used the term from everyone else in the
new job, but certainly there must be others that do it as it is a well
known way to measurably increase the dependency and maintainability of
your code. What then are some good C techniques to make code unit
testable that hopefully fit in nicer with the C paradigm than going
all OO or generic?

 

[John Gordon]

For example, unit testing often requires strong use of polymorphism,
abstract types, inheritance, dependency injection, mocks, etc... You

Unit testing is, simply, writing code that calls your functions under a
known set of conditions and examines the results to verify that the test
passed.

For example, say you have a function that accepts a list of items and a
specific item to be searched for in the list. The function returns true
if the item is found in the list, and false if it isn't.

To unit-test this function, you'll want to write (at least) two test
cases: one case where the list does contain the item to verify that
the function returns true, and another case where the list does not
contain the item to verify that the function returns false.

To be realistic, you'd probably want more than two test cases to cover
things like the list being empty or containing only one item, the target
item being at the beginining/middle/end of the list, etc.

Unit testing can involve polymorphism and all those other things, but it
certainly doesn't have to.

 

[Ian Collins]

Something I've been trying to do and my bosses have wanted me to
continue is to apply unit testing to the code I'm working with. In
some cases this has been fairly straight forward but in others it has
not. Some of what I've tried to do to harness some of the code has
been discouraged and I'm not so sure much of what I'd do in C++ or
Java lends itself directly to C.

If whatever you are doing in C++ doesn't lend itself to C, you aren't
writing unit tests.

"Unit test" is a much abused and overloaded term. I agree with John
Gordon's definition else-thread. Anything else is some form of higher
level module or functional testing.

For example, unit testing often requires strong use of polymorphism,
abstract types, inheritance, dependency injection, mocks, etc...

Mocks, yes, the rest? Probably not.

You
CAN do these things in C but if you were to that extent then you're
almost wasting your time and effort reimplementing C++ or Objective-C
(depending on the static/dynamic approach you want) for no good
reason. You do perhaps have a modicum of more control but beating
those languages at what they do by hand is a pretty lofty goal.

If the goal is to write unit tests, then function mocks for the
functions called by the unit under test and the test functions them
selves are all you require.

Perhaps unit testing isn't often done in C either, as might be
indicated by how different I used the term from everyone else in the
new job, but certainly there must be others that do it as it is a well
known way to measurably increase the dependency and maintainability of
your code. What then are some good C techniques to make code unit
testable that hopefully fit in nicer with the C paradigm than going
all OO or generic?

Unit tests are used equally in C and C++ (at least in my world). I use
the same test frameworks form both languages (Google test and CppUnit).
If your mainline code has to be aware of the tests (granting access to
private bits for example), your design is flawed.

 

[James Kuyper]

Something I've been trying to do and my bosses have wanted me to
continue is to apply unit testing to the code I'm working with. In
some cases this has been fairly straight forward but in others it has
not. Some of what I've tried to do to harness some of the code has
been discouraged and I'm not so sure much of what I'd do in C++ or
Java lends itself directly to C.

For example, unit testing often requires strong use of polymorphism,
abstract types, inheritance, dependency injection, mocks, etc... You
CAN do these things in C but if you were to that extent then you're
almost wasting your time and effort reimplementing C++ or Objective-C
(depending on the static/dynamic approach you want) for no good
reason. You do perhaps have a modicum of more control but beating
those languages at what they do by hand is a pretty lofty goal.

Perhaps unit testing isn't often done in C either, ...

Unit testing is done in C, at least in my group. I know enough about C++
to understand what each of the things you mentioned are (except "mocks"
- is that short for "mock objects"?), and I can understand how they
might be used in unit testing. However, having done almost all of my
unit testing in contexts where those concepts are meaningless, I find it
hard to understand why you might consider them essential.

... as might be
indicated by how different I used the term from everyone else in the
new job, ...

That, at least, is in agreement with my experience: there are many
different concepts of what unit testing means, with corresponding
confusion when people try to talk about it.

... What then are some good C techniques to make code unit
testable that hopefully fit in nicer with the C paradigm than going
all OO or generic?

While it is possible to do OO and generic programming in C, the design
of the language does not make it easy to do so. Trying to force C to fit
the OO/generic paradigm that you're used to would be a good way to waste
large quantities of your time; I'd recommend learning to adapt your
techniques to C, rather than trying to retrofit C to fit into the
techniques that you're familiar with.

Testability shouldn't have much affect on your code design. If it does,
you're doing something wrong, either in your design or your testing.

 

[nroberts]

On 10/03/2011 03:26 PM, nroberts wrote:

While it is possible to do OO and generic programming in C, the design
of the language does not make it easy to do so. Trying to force C to fit
the OO/generic paradigm that you're used to would be a good way to waste
large quantities of your time; I'd recommend learning to adapt your
techniques to C, rather than trying to retrofit C to fit into the
techniques that you're familiar with.

So your answer to my question is to tell me I should ask the
question. Thanks.

Testability shouldn't have much affect on your code design. If it does,
you're doing something wrong, either in your design or your testing.

I can't say I agree with that at all. One could argue of course that
sure, in fact by definition it is correct since one major bonus of
doing unit testing is that it forces good design, and one main reason
why legacy code in notoriously untestable is because it lacked this
force. On the other hand, if the practice of unit testing is not
changing the way you design your code then you're already too damn
perfect to make bugs anyway or you're doing something very
wrong...like violating 15 inviolate laws of the universe and turning
logic inside out or something.

 

[nroberts]

On 10/03/2011 03:26 PM, nroberts wrote:

And please reply to the newsgroup, not to my email.

 

[Jorgen Grahn]

I can't say I agree with that at all. One could argue of course that
sure, in fact by definition it is correct since one major bonus of
doing unit testing is that it forces good design, and one main reason
why legacy code in notoriously untestable is because it lacked this
force.

I partly agree and partly disagree with that. Partly disagree, because
there are designs which make perfect sense for the program itself, but
are impossible for unit testing.

For example, I believe doing I/O (file I/O, sockets, logging,
whatever) in the core of your code isn't necessarily bad design. If
you try to split it out, you end up with a complicated -- but
unit-testable -- design.

/Jorgen

 

[nroberts]

I partly agree and partly disagree with that. Partly disagree, because
there are designs which make perfect sense for the program itself, but
are impossible for unit testing.

For example, I believe doing I/O (file I/O, sockets, logging,
whatever) in the core of your code isn't necessarily bad design. If
you try to split it out, you end up with a complicated -- but
unit-testable -- design.

That's where mocking and DI comes in. For example with regard to file
I/O my C++ functions take streams. You can then set them up with
string streams during testing.

Unfortunately I've not seen a good way to do this in C so functions
that do file I/O...only way I've found to test them is to create files
with the appropriate content, inject the file or FILE*, and delete
after.

Those are all the hard cases though. They are testable and the
designs you create to make them so are demonstratively better, but
they are difficult problems.

 

[James Kuyper]

So your answer to my question is to tell me I should ask the
question. Thanks.

In essence, yes, and I apologize for not being able to answer your
question more directly. I couldn't figure out anything C-specific to
tell you about. All of the unit testing techniques I use regularly are
equally usable in C and C++, such as the use of stubs and test drivers;
I'd expect you to already be familiar with them. Most of the differences
between C and C++ unit testing consist of things you don't need to do in
C because you can't do them in C - for instance, there's no need for
your test drivers to catch exceptions, because the code you're testing
can't throw any. I wouldn't expect that you would need such advice.

Your comments did leave me worried that you might go to painful lengths
to give your C code the genericity which comes so easily in C++, and
then do a lot of testing to verify that the genericity is working
properly. Writing generic code is so much harder to do in C that it's
seldom worth the trouble. It requires extensive use of features like
void* that aren't type-safe.

I can't say I agree with that at all. One could argue of course that
sure, in fact by definition it is correct since one major bonus of
doing unit testing is that it forces good design, and one main reason
why legacy code in notoriously untestable is because it lacked this
force. ...

It's not been my experience that legacy code is untestable - quite
possibly the legacy code you've had to deal with was mis-designed in
ways that I've been lucky enough not to run into.

Now that I think of it, a small portion of the legacy code I've had to
deal with was so poorly modularized that it would have been quite a pain
to test. However, I was not held responsible for testing it until after
I'd been assigned the task of updating it to add several new features.
In the course of the update I broke it up into modules small enough and
sufficiently coherent in their purposes that they could reasonably be
tested.

... On the other hand, if the practice of unit testing is not
changing the way you design your code then you're already too damn
perfect to make bugs anyway or you're doing something very
wrong...like violating 15 inviolate laws of the universe and turning
logic inside out or something.

Unit testing has certainly changed the design of my code, by leading me
to fix the bugs I've uncovered. However, concern for testability has
not, in itself, had any significant effect on the design of my code.
Possibly I've always known something about design that you had to learn.
More plausibly, perhaps you've learned something about designing for
testability that I haven't - but by definition, if that's the case, I'd
have no idea what that might be.

I apologize for sending you e-mail in parallel with the message to the
group. I've been hitting the "Reply-All" button rather than the "Reply"
button a lot frequently - I'm not sure why.

 

[Jorgen Grahn]

Something I've been trying to do and my bosses have wanted me to
continue is to apply unit testing to the code I'm working with. In
some cases this has been fairly straight forward but in others it has
not. Some of what I've tried to do to harness some of the code has
been discouraged and I'm not so sure much of what I'd do in C++ or
Java lends itself directly to C.

For example, unit testing often requires strong use of polymorphism,
abstract types, inheritance, dependency injection, mocks, etc...

Personally, I've decided to reject that idea for now. When I work in
C++, I don't "fuzz" my design to make it possible to unit test -- I
feel the loss of focus on the problem (caused by adding flexibility
which the problem doesn't need) makes the code much harder to
understand.

For the parts of the code which *can* be unit tested without warping
the design, I happily write tests. That leaves me with bits and pieces
which I can trust, but I have to find other ways to trust the rest
(review, a solid design, other kinds of testing).

All this is slightly easier in C++ than in C because C++ has more
"builtin fuzz" like I/O to/from strings and generic programming.

Perhaps unit testing isn't often done in C either, as might be
indicated by how different I used the term from everyone else in the
new job

What are the two different usages? I think the only confusion I've
encountered is if testing foo() also means testing everything it
depends on, or if it means mocking everything "below it".

/Jorgen

 

[Ian Collins]

That's where mocking and DI comes in. For example with regard to file
I/O my C++ functions take streams. You can then set them up with
string streams during testing.

Unfortunately I've not seen a good way to do this in C so functions
that do file I/O...only way I've found to test them is to create files
with the appropriate content, inject the file or FILE*, and delete
after.

It is pretty simple to interpose I/O library functions in either C or
C++. It is actually more straightforward with C; in C++ if a function
*creates* a file stream to output data, you either have to mock the
whole stream class, or know which lower level functions will be called.
Higher levels of abstraction make for more complex mocks.

Those are all the hard cases though. They are testable and the
designs you create to make them so are demonstratively better, but
they are difficult problems.

No, they are quite simple once you know how!

 

[Jorgen Grahn]

But being unit testable should be a design goal of most programs. If you
can't test it to make sure it works, then how do you know if the program
does what it's supposed to do?

Lack of unit tests doesn't imply lack of testing. You have to spend
more time reviewing the code and on testing on a higher level ("system
testing", or whatever the term is).

I should add here that I think I do more unit testing than most people
around me -- so I'm not *totally* against it.

Splitting those things out doesn't require complexity, there are just fewer
ways to do it simply and elegantly (and many more ways to do it poorly).
Experience and practice will teach you the right way to do it.

Perhaps. I've certainly learned *some* ways of doing it. Some examples
which come to mind:
- make my testable function foo(FILE*) instead of foo(filename)
- or make it a state machine which you feed blocks of data one by one
But I'd rather not do such things unless there are other benefits
apart from testability.

/Jorgen

 

[Ian Collins]

Perhaps. I've certainly learned *some* ways of doing it. Some examples
which come to mind:
- make my testable function foo(FILE*) instead of foo(filename)
- or make it a state machine which you feed blocks of data one by one
But I'd rather not do such things unless there are other benefits
apart from testability.

I'd say your example is such a case. Splitting the work done on the
file from opening it (and all the accompanying error checking baggage)
is good, as well as testable, design.

Whenever I come across a case where something looks difficult to test,
the design can nearly always be improved to make it easier to test.

In other words: tricky to test == design smell.

 

[Nick Keighley]

In other words: tricky to test == design smell.

and of all the cute, folksy terms floating around "design smell" must
be the most irritating

 

[Jorgen Grahn]

and of all the cute, folksy terms floating around "design smell" must
be the most irritating

Yes. It's subtly anti-intellectual; who wants to smell?

Unfortunately there is also no better term that I can think of.
So I didn't mind Ian using it above.

/Jorgen