Testing in C++

[nw]

Hi,

I have been asked to teach a short course on testing in C++. Until now
I have used my own testing classes (which from what I've seen seem
similar to the boost unit testing classes). Considering I have a
limited amount of time what do readers of this group think would be
useful to cover in this course? Is boost the way to go?

Sorry if this is off-topic in this group.

Many Thanks.

 

[Ian Collins]

Hi,

I have been asked to teach a short course on testing in C++. Until now
I have used my own testing classes (which from what I've seen seem
similar to the boost unit testing classes). Considering I have a
limited amount of time what do readers of this group think would be
useful to cover in this course? Is boost the way to go?

Test Driven Development with CppUnit!

 

[dave_mikesell]

Hi,

I have been asked to teach a short course on testing in C++. Until now
I have used my own testing classes (which from what I've seen seem
similar to the boost unit testing classes). Considering I have a
limited amount of time what do readers of this group think would be
useful to cover in this course? Is boost the way to go?

This article is a bit dated (2004), but it compares several C++ unit
testing frameworks (including Boost):

http://www.gamesfromwithin.com/articles/0412/000061.html

 

[Phlip]

Try to get as far as Abstract Test Cases, which are super-important for
refactoring. Read /Pragmatic Unit Testing/ and/or /Unit Test Frameworks/.

And emphasize programmers write tests _while_ they write the tested code.
That saves a lot of time debugging.

This article is a bit dated (2004), but it compares several C++ unit
testing frameworks (including Boost):

http://www.gamesfromwithin.com/articles/0412/000061.html

Yeah, it pans my "NanoCppUnit", which I _told_ him was just a sketch and not
productized yet. Then he complains it's just a sketch and not productized.

And that article came out before Noel's own UnitTest++, which bears a
striking resemblance to NanoCppUnit, which was in use at the lab where I
worked with Noel.

http://unittest-cpp.sourceforge.net/

With friends like that...

But, seriously folks, UnitTest++ is my favorite C++ unit test rig. Its tests
for itself are exemplary.

 

[Roland Pibinger]

I have been asked to teach a short course on testing in C++. Until now
I have used my own testing classes (which from what I've seen seem
similar to the boost unit testing classes). Considering I have a
limited amount of time what do readers of this group think would be
useful to cover in this course?

You should at least explain the different kinds of tests: integration
test, functional test, unit test, assert.

Is boost the way to go?

No (unless you want to chase away your students). The following is a
nice unit test framework, esp. for teaching purposes:
http://www.ddj.com/dept/cpp/184401279

Best regards,
Roland Pibinger

 

[Sarath]

Hi,

I have been asked to teach a short course on testing in C++. Until now
I have used my own testing classes (which from what I've seen seem
similar to the boost unit testing classes). Considering I have a
limited amount of time what do readers of this group think would be
useful to cover in this course? Is boost the way to go?

Sorry if this is off-topic in this group.

Many Thanks.

CPP Unit is a free tool available for C++ Unit Testing.

http://cppunit.sourceforge.net

 

[Pete Becker]

You should at least explain the different kinds of tests: integration
test, functional test, unit test, assert.

And regression test. Well, maybe not, since it has become meaningless.

--

-- Pete
Roundhouse Consulting, Ltd. (www.versatilecoding.com)
Author of "The Standard C++ Library Extensions: a Tutorial and
Reference." (www.petebecker.com/tr1book)

 

[dave_mikesell]

But, seriously folks, UnitTest++ is my favorite C++ unit test rig. Its tests
for itself are exemplary.

Thanks, Phlip. I'll check it out. I've only used CPPUnit to date
because it's what I used first and it worked OK.

 

[nw]

Test Driven Development with CppUnit!

I've taken a look at the CppUnit documentation. It seems there is
quite a significant overhead in CppUnit in terms of creating classes/
methods to get a simple test running (as compared to other testing
frameworks) what advantages does this give you?

 

[nw]

Try to get as far as Abstract Test Cases, which are super-important for
refactoring. Read /Pragmatic Unit Testing/ and/or /Unit Test Frameworks/.

And emphasize programmers write tests _while_ they write the tested code.
That saves a lot of time debugging.



Yeah, it pans my "NanoCppUnit", which I _told_ him was just a sketch and not
productized yet. Then he complains it's just a sketch and not productized.

And that article came out before Noel's own UnitTest++, which bears a
striking resemblance to NanoCppUnit, which was in use at the lab where I
worked with Noel.

http://unittest-cpp.sourceforge.net/

With friends like that...

But, seriously folks, UnitTest++ is my favorite C++ unit test rig. Its tests
for itself are exemplary.

Thanks for your suggestion, after reading a bit about it I'm tending
towards this it seems quite straight forward to use.

As an aside, many testing frameworks (UnitTest++) appear to make
extensive use of macros, is there a particular reason for this? I
thought "Macros are evil"? http://www.parashift.com/c++-faq-lite/misc-technical-issues.html#faq-39.4

nope, it's currently a 404 error.

 

[Pete Becker]

Hi,

I have been asked to teach a short course on testing in C++. Until now
I have used my own testing classes (which from what I've seen seem
similar to the boost unit testing classes). Considering I have a
limited amount of time what do readers of this group think would be
useful to cover in this course? Is boost the way to go?

Talk about testing, not about tools. For example, have each member of
the class write a complete set of test cases for a program that reads
three numbers from stdin and tells you whether a triangle with sides of
those lengths is equilateral, isosceles, scalene, or impossible. Then
compare the solutions, and talk about testing principles that help
refine the set of test cases.

--

-- Pete
Roundhouse Consulting, Ltd. (www.versatilecoding.com)
Author of "The Standard C++ Library Extensions: a Tutorial and
Reference." (www.petebecker.com/tr1book)

 

[Phlip]

As an aside, many testing frameworks (UnitTest++) appear to make
extensive use of macros, is there a particular reason for this? I
thought "Macros are evil"?

That is the entry-level aphorism. The correct rule is "don't abuse
macros; they are easier to abuse than other things".

Per Bjarne, Herb, and Alex (IIRC), a macro is the only way to do an
assertion, and a unit test is nothing but a humongous nested
assertion. All test runners in the C languages use macros, and they
provide features, such as expression reflection, that other test
runners sorely miss.

nope, it's currently a 404 error. - Hide quoted text -

I need to downgrade that to http://c2.com/cgi/wiki?ZeekLand . But the
project is on hold while I do other things. Start here:

http://flea.sourceforge.net/PiglegToo_1.html

 

[Noah Roberts]

I've taken a look at the CppUnit documentation. It seems there is
quite a significant overhead in CppUnit in terms of creating classes/
methods to get a simple test running (as compared to other testing
frameworks) what advantages does this give you?

Really? Huh...I guess I've been using some other CppUnit library...

 

[Noah Roberts]

All test runners in the C languages use macros, ...

If by "C languages" you mean C++ inclusively then no, you're wrong.

http://tut-framework.sourceforge.net/

 

[Noah Roberts]

And regression test. Well, maybe not, since it has become meaningless.

Has it?

 

[Pete Becker]

Has it?

Yes.

--

-- Pete
Roundhouse Consulting, Ltd. (www.versatilecoding.com)
Author of "The Standard C++ Library Extensions: a Tutorial and
Reference." (www.petebecker.com/tr1book)

 

[Pete Becker]

Yes.

Well, I suppose a more complete answer would be appropriate.

Today, "regression test" seems to mean "run the tests you've run before
and see if anything got worse." I.e., run the test suite. Formally,
though, a regression test is a test you add to your test suite in
response to a user-reported defect, reproducing the user's conditions.
You also add specific tests cases to the general test suite, since
obviously the particular problem that caused the failure wasn't
detected. The idea behind regression tests is that you may be dealing
with a fragile area of the code, and rerunning cases that failed in the
field gives you some reassurance that you haven't missed something.
Since user-supplied code can be big and clunky, you only run regression
tests occasionally, and rely on the more focused test cases for day to
day testing.

--

-- Pete
Roundhouse Consulting, Ltd. (www.versatilecoding.com)
Author of "The Standard C++ Library Extensions: a Tutorial and
Reference." (www.petebecker.com/tr1book)

 

[Phlip]

If by "C languages" you mean C++ inclusively then no, you're wrong.

http://tut-framework.sourceforge.net/

All true Scotsmen eat their oatmeal without sugar.

Thanks for confirming my point; let's see what tut is missing:

ensure("lost ownership",ap.get()==0);

At fault time, the output cannot reflect the file, line number, or the
'ap.get()==0'.

So, all true assertions in the C languages use macros. Those who avoid
them, just to brag "we don't use macros" on their homepages, are not
as useful.

BTW the "significant overhead" in CppUnit is all the test cases,
suites, and registration methods you must set up just to get anything
done. Never ignore the effect that redundant code, and poor assertion
diagnostics, have on project velocity.

 

[dave_mikesell]

But, seriously folks, UnitTest++ is my favorite C++ unit test rig. Its tests
for itself are exemplary.

Hey Phlip - I checked out UnitTest++ and it's really cool. One thing
I tried to do, though, that wasn't successful...I defined member
functions in my fixture to do certain checks that are performed across
several tests, but the CHECK* macros wouldn't compile (errors below).
When I moved the CHECKs to TEST_FIXTURE definitions they work fine,
but now I have redundant macro calls across several tests. Is this a
feature?

error C2065: 'testResults_' : undeclared identifier
error C2228: left of '.OnTestFailure' must have class/struct/union
type
error C2065: 'm_details' : undeclared identifier

 

[Noah Roberts]

All true Scotsmen eat their oatmeal without sugar.

So you realize the fallacy but just choose to use it anyway. Whatever;
I have no need to continue this conversation anyway.