A simple unit test framework

J

James Kanze

What you are up against here is Kanze is one of the most aggressive and
competent reviewers out there, and he leads by reviewing. This explains his
devastating accuracy on C++ questions. So by claiming two dumb people can
get by with pairing and TDD, instead of submitting their code to him for
review, you are taking him out of his Comfort Zone.

Thanks for the complement, but it's not personal, and it's not
about me. The important aspect of code review (one of them,
anyway) is the fresh, unjaundiced viewpoint; the person looking
at the code has not seen it before. Another, perhaps even more
important, is that the programmers write their code to be
reviewed. Someone who has never seen the code before must be
capable of understanding it quickly and without undo effort, and
it must be "obvious" why it works, and that it cannot fail. If,
during review, the author has to explain why the code works, the
code fails review.
 
M

Michael DOUBEZ

anon a écrit :
James said:
[snip]
Pete, I think you are missing the point of TDD.

It's easy for those unfamiliar with the process to focus on the "T" and
ignore the "DD". TDD is a tool for delivering better code, the tests
drive the design, they are not driven by it.

Which, of course, is entirely backwards.

It is, but you get better code

That greatly depend on the quality of tests. At the end of the day a
clumsy programmer make clumsy tests and take it down the whole chain.

TDD Anti-Pattern do exist.

[snip]
So your log function only has to produce correct results for the
limited set of values you use to test it? I hope I never have
to use a library you wrote.

If you take some random numbers (for example 0.0, 0.007, 0.29, 0.999,
1.0, 1.0001, 1.9900, 555.0, 999999.0) and your log function for these
numbers gives correct results (with small enough error) you can be sure
your log function is good

Well your example is a good one:
0.0 will give nan
0.007, .... 999999.0 will eventually give correct result

But what of the values between 0.007 and 0.0 where the derivative tend
to infinite and error epsilon may grow accordingly.

You can be confident the function is good provided the test values (or
the repartition of randomly generated ones) are chosen carefully.

Michael
 
J

James Kanze

It is, but you get better code

Better code than what?

In practice, if a coder wants to write the unit tests first, no
one will stop him; his deliverable consists of both the code and
the unit tests. But there's absolutly no advantage in doing so,
and most people I've worked with feel more comfortable writing
the actual code first.
If you were to write a log() function, would you test it
against all floats?

That's precisely my point: you can't. So you have to analyse
the code, to determine where the critical boundaries are, and
test the limit cases. And of course, you have to document that
analysis, because the code reviewers will definitly want to
review it.
If you take some random numbers (for example 0.0, 0.007, 0.29, 0.999,
1.0, 1.0001, 1.9900, 555.0, 999999.0) and your log function for these
numbers gives correct results (with small enough error) you can be sure
your log function is good

Obviously, you don't know how floating point arithmetic works,
or you wouldn't make such a stupid statement.
 
A

anon

James said:
Better code than what?

In practice, if a coder wants to write the unit tests first, no
one will stop him; his deliverable consists of both the code and
the unit tests. But there's absolutly no advantage in doing so,
and most people I've worked with feel more comfortable writing
the actual code first.

I can see you are extremely against TDD, and thats your personal preference.
As long as you have good unit tests, and high coverage, I can say your
code is good.
That's precisely my point: you can't. So you have to analyse
the code, to determine where the critical boundaries are, and
test the limit cases. And of course, you have to document that
analysis, because the code reviewers will definitly want to
review it.



Obviously, you don't know how floating point arithmetic works,
or you wouldn't make such a stupid statement.

Maybe I don't know. So, why is it a stupid statement?

How would you test your log() function?
 
G

Gianni Mariani

James Kanze wrote:
....
So you exclude multithreading and floating point. Since neither
can really be tested.

Certainly not.

Try reading that again. The whole purpose of the diatribe was for
testing MT code.

I still maintain that if you can't test your code to an acceptable level
of confidence then your code is no good - needs to be
re-architected/re-written/wahetever.
So you write bad code to begin with. Or simply prefer to ignore
real problems that don't show up trivially.

That statement reminds me of a question I sent to support at a software
vendor for a protocol stack in '86. My question was basically "how does
your stack handle corrupt packets". The question came back, "why would
you want to send it corrupt packets ?" .... :-(
 
J

James Kanze

James said:
James Kanze wrote:
Gianni Mariani wrote:
I have met very few customers that know what a spec is even if it
smacked them up the side of the head.
Welcome to the club!
Sad. Inevitably it leads to pissed off customer.
Any agile process (XP, Scrum or whatever) is ideal for this situation.
If your goal is to rip off the customer, yes.
[First, a meta-comment: I am, of course, exagerating to make
a point, and I don't really suspect Ian of trying to use any
technique to rip off his customers.]
So by helping them to get what they really wanted, rather than forcing
them to commit to what they thought the wanted, I'm ripping them off?
How does testing help the customer to get what he really wants?
By being an integral part of the process.

Testing is, and always has been, an integral part of the
process. Just as is design, coding, code review, integration...
Obviously, having a process does help the customer get what he
wants. My question was more along the lines of: how does
involving the customer in all this (most of which he doesn't
care about, nor understand) help him get what he wants?
Not every customer is in a position to provide specifications and those
that think they can often change their minds once the development has
started.

You need some sort of specification to begin coding. If the
customer can't provide it, you provide it for him, but you do
get him to sign it off.

And of course, specifications have never been cast in stone.
They evolve. But it's not open ended, either. If the customer
suddenly decides that he wants the program to do ten times more
than was originally considered, that will have an impact on the
cost and the delay. The customer asks for the change, I work
out how much it will cost, in time and money, and he decides
whether he wants it or not.

I've seen companies who never accepted any changes. They very
soon find themselves without any customers. I've also seen
companies who accepted all changes, without consideration of the
additional time or cost. They very soon go bankrupt. It's a
two way street.

But I'm sure you know all this. What I want to know is what
relationship this has to testing, per se? These have been
self-evident truths since my first contact with computer
science, back in the 1960's.
If you were to take the time to look into XP or Scrum, you would see
that you are to some extent preaching to the choir.

I wonder if part of our differences don't have to do with the
type of applications we write. In another posting, you
mentionned that you did a program for a Web. In other words,
most of your requirements are visible behavior, and the
requirements for reliability and longivity aren't really very
high. I'm not too sure what you understand by "tests" here,
because I consider unit tests something that runs automatically,
with no visible output unless there is an error, but certainly,
in such a context, showing the customer a more or less working
program very early in the process, and getting feed back from
it, is important.

I work mainly on large scale servers. The programs run 24 hours
a day, 7 days a week. Deploying a new version may mean stopping
the system, at considerable cost. Nothing that the program does
is immediately visible (but if it doesn't work, a lot of other
things suddenly stop working, and that IS visible). An error in
the code costs real money.
That's why there are (web driven) test suites like FIT or
Selenium that are specifically designed for the customer to
understand and in some cases, use them selves. Don't forget
the customer acceptance tests are system behavioral tests, not
code unit tests (unplug module A and module A missing trap is
sent kind of tests). How the application behaves is what the
customer wants to see.

Or doesn't want to see:). (Much of my work in the past has
been involved with network routing. Which normally should
happen silently, behind the scenes.)
 
G

Gianni Mariani

James said:
I have exactly opposite experience. Multi threading program failures
show up only with specially crafted input.
That is I have to look into code first and then make tests to
specifically target places in code .
But, such errors are usually most simplistic ones, those are common
threading errors, and can be usually catched by eye.

Even when you know exactly where the error is, it's sometimes
impossible to write code which reliably triggers it. Consider
std::string, in g++: if you have an std::string object shared by
two threads, and one thread copies it, and the other does [] or
grabs an iterator at exactly the same time, you can end up with
a dangling pointer---an std::string object whose implementation
memory has been freed. The probability of doing so, however, is
very small, and even knowing exactly where the error is, I've
yet to be able to write a program which will reliably trigger
it.

Are you sure. I ran into that problem in an earlier version of gcc's
std::string. I have never seen it since and I did review the
std::string code at 3.0 and I was satisfied that all was well.

std::string is not thread safe but this should work:

std::string global1( "A" );

std::string global2( global1 );

void thread1()
{
global1 += "1";
}

void thread2()
{
global2 += "A";
}

Consider some variations on DCL. In at least one case, you only
get into trouble if one of the processors has read memory in the
same cache line as the pointer just before executing the
critical code. Which means that the function can work perfectly
in one application, and fail when you link it into another
application.

Yes, but it's exactly this type of test that finds bugs like that.
As I mentionned in a response to another poster, it may just
depend on what you consider acceptable quality. I've worked on
critical systems a lot in the past. With contractual penalties
for downtime. So I tend to set my standards high.
Interestingly enough, however, it turns out that developing code
to such high standards is actually cheaper than just churning it
out.

Yes. Better tested code is easier to develop with.
... As a rough, back of the envelope figure: correcting an
error found in code review costs one tenth of correcting the
same error found in unit tests,

What are you smoking ? Sure you can find some obvious bugs in a code
review, but I would have already run the unit tests before I do the code
review.
... which costs one tenth of
correcting the same error found in integration tests, which
costs one tenth of correcting the same error found in the field.

We can haggle on how many orders of magnitude field errors cost but I'll
agree it's at least 2 orders above unit tests.
Not to mention the advantages of the transfer of knowledge which
takes place in code review. The result is that any time an
integration test finds an error, it is considered first and
foremost an error in the process; we analyse the process to find
out where it failed. And there is even a tendancy to do this
for unit tests, although perhaps not as systematically.

[...]
Not just mp system, but *all* possible mp systems, which is
of course impossible.

It's perhaps worth pointing out that most current low-end MP
systems use a more or less synchronized memory model. This is
not true, however, for Alpha processors, nor the Itanium, nor, I
think, top of the line Sparcs, nor, probably future desktop
processors. That means that the fact that code works on a
current four processor system based on Intel 32 bits means
nothing with regards to future processors.

Only in low level code. Code written above the iron layer should work
just fine.
 
J

James Kanze

James Kanze wrote:

[...]
If you write tests according to the requirements, you can be 100% sure
that statement is correct

A requirement for the function log() is that it return the
correct value, at 1 LSB, for all positive floating point values.
If you write a test directly according to that, it will take
something over 3000 centuries to run, supposing that you can
test one value per microsecond (which is beyond the capabilities
of many modern machines).

If the requirements involve thread safety, it may not even be
possible to theoretically guarantee coverage, since a number of
aleas are involved that you cannot control. The same thing is
true about any number of other types of applciations: real time
process control, network management, etc.
 
A

anon

James said:
James Kanze wrote:
[...]
In this case you will not
have to do a coverage, but it is a plus. This way, the code you write
will be minimal and easier to understand and maintain.
And will not necessarily meet requirements, or even be useful.
If you write tests according to the requirements, you can be 100% sure
that statement is correct

A requirement for the function log() is that it return the
correct value, at 1 LSB, for all positive floating point values.
If you write a test directly according to that, it will take
something over 3000 centuries to run, supposing that you can
test one value per microsecond (which is beyond the capabilities
of many modern machines).

So, what you are saying here is that every math function is tested the
way you suggested?

If the requirements involve thread safety, it may not even be
possible to theoretically guarantee coverage, since a number of
aleas are involved that you cannot control. The same thing is
true about any number of other types of applciations: real time
process control, network management, etc.

There are things which are not possible to be unit tested, but if a code
can be unit tested, then there are no reasons not to do so.
Off course, I am talking about serious projects - not examples and hello
worlds.
 
I

Ian Collins

James said:
Which is typical for most process improvements:). At least at
the beginning; one of the problems in the initial process (or
lack of it) is that it doesn't generate figures.


Comprehensive unit tests are important for many reasons. I'm
not arguing against them. I'm saying that they aren't
sufficient, if they are the only measure. For that matter, how
do you know the tests are comprehensive? The only means I know
is to review them at the same time you review the code.
Because there were written first. We've been there before!

The code was well reviewed there were six or eight on the team, we
rotated pairs and everyone worked on every bit of code.
Comprehensive unit tests are most important for maintenance.
New code should always be reviewed. But a review is not without
costs, and reviewing an entire module because someone has
changed just a couple of characters in just one line isn't
really cost effective. Comprehensive unit tests, on the other
hand, are very effective at catching slips of the finger in the
editor.
Again, that's where paring and continuous integration come in, the code
is under constant scrutiny from many pairs of eyes.
And why weren't they being tracked? 6 is, IMHO, a lot, and can
certainly be reduced. Find out why the bug crept in there to
begin with, and modify the process to eliminate it. (I've
released code where the only bug, for the lifetime of the
product, was a spelling error in a log message. That happened
to be the product where there weren't any unit tests, but
admittedly, it was a very small project---not even 100 KLoc in
all.)
The cause was too obvious, a couple of spelling mistakes and a couple of
bugs in a bit of code that crept in without tests. The latter was an
object lesson in following the process, the former a prime demonstration
of engineer's collective inability to spell! The remainder were in an
obscure protocol implementation we lacked the equipment to test. We had
a perfect implementation of our misunderstanding of the specification!
We tried that, but found that it's no where near as cost
effective as good code review. The code review brings in an
"outside" view---someone who's not implicated in the code. In
practice, it turns out that it's this external viewpoint which
really finds the most bugs. And of course, code review costs
less than pairing (although not by a large amount).
Pairing is not just about review, but it brings a unique energy into the
programming process. Rotating pairs gives the "outside" view, in our
case, there wasn't anyone out side the team to review the code.
One team used full XP for about six months. We stopped the
effort when we saw their error rate shooting up significantly.
(We were at about 1 error per 100 KLoc, going into integration,
at that time. The team using full XP ended up at about ten
times that.)
Did they hire an experienced coach? That is usually the result of not
doing thing correctly.
 
J

James Kanze

[...]
Even when you know exactly where the error is, it's sometimes
impossible to write code which reliably triggers it. Consider
std::string, in g++: if you have an std::string object shared by
two threads, and one thread copies it, and the other does [] or
grabs an iterator at exactly the same time, you can end up with
a dangling pointer---an std::string object whose implementation
memory has been freed. The probability of doing so, however, is
very small, and even knowing exactly where the error is, I've
yet to be able to write a program which will reliably trigger
it.
Are you sure. I ran into that problem in an earlier version of gcc's
std::string. I have never seen it since and I did review the
std::string code at 3.0 and I was satisfied that all was well.

I'm sure. I wouldn't swear that it hasn't been corrected in the
latest version, but it was definitely present in 3.4.3. The bug
report is still open; from what I understand, a totally new
string class is in the works, so the current one will not be
corrected.
std::string is not thread safe

Everything in the standard library in g++, from 3.0 on, is
supposed to be thread safe. There is some uncertainty, for some
classes, as to how this is defined, and I hesitated to post the
bug, because I wasn't sure that std::string was supposed to meet
the Posix requirements (although all of the other g++ containers
meet them). That is, however, a bit irrelevant to the
discussion here. Posix compliant thread safety is a reasonable
choice, the current implementation of std::string in g++ doesn't
meet it, and I cannot imagine any possible test which would
display this defect in a reliable fashion.
but this should work:
std::string global1( "A" );
std::string global2( global1 );
void thread1()
{
global1 += "1";
}
void thread2()
{
global2 += "A";
}

That definitly should (and as far as I know, does) work. The
problem is more along the lines of:

std::string global( "a" ) ;

void thread1()
{
std::string s1( global ) ;
}

void thread2()
{
std::string s2( global.begin(), global.end() ) ;
}

Since I'm not modifying global, Posix says that the above should
work (or rather, that it should work if the involved types were
known to Posix). Code review shows that it can fail in the
current implementation in G++. I would very much like to see a
test program where it reliably fails, however.
Yes, but it's exactly this type of test that finds bugs like that.

What type of test? You mean that for each test, you wrap the
code in a different environment, recompile, and relink?
Yes. Better tested code is easier to develop with.

The issue hasn't been raised to date, but...

Good code is easy to understand. Code that isn't easy to
understand fails code review. How does testing verify this?

(In some ways, easy to understand is more important than an
absence of errors.)
What are you smoking ? Sure you can find some obvious bugs in
a code review, but I would have already run the unit tests
before I do the code review.

That's doing things the hard way, and is hardly cost effective.
In code review, you are told that you forgot to initialize
variable i in line 1234 of abc.cc. The unit test tells you that
tests 25 through 30 all fail. Which information makes it easier
to find and fix the error?

And a well run code review doesn't find just the obvious bugs.
It also finds those which no test will find. (I found the bug
in the g++ implementation of std::string by reviewing the code.)
We can haggle on how many orders of magnitude field errors
cost but I'll agree it's at least 2 orders above unit tests.

My figures are "back of the envelope". The actual values will
vary enormously. The goal is only to give a rough idea.
[...]
It does require a true multi processor system to test adequately.
Not just mp system, but *all* possible mp systems, which is
of course impossible.
It's perhaps worth pointing out that most current low-end MP
systems use a more or less synchronized memory model. This is
not true, however, for Alpha processors, nor the Itanium, nor, I
think, top of the line Sparcs, nor, probably future desktop
processors. That means that the fact that code works on a
current four processor system based on Intel 32 bits means
nothing with regards to future processors.
Only in low level code. Code written above the iron layer should work
just fine.

Dream on.
 
I

Ian Collins

James said:
I wonder if part of our differences don't have to do with the
type of applications we write. In another posting, you
mentionned that you did a program for a Web. In other words,
most of your requirements are visible behavior, and the
requirements for reliability and longivity aren't really very
high. I'm not too sure what you understand by "tests" here,
because I consider unit tests something that runs automatically,
with no visible output unless there is an error, but certainly,
in such a context, showing the customer a more or less working
program very early in the process, and getting feed back from
it, is important.
I work in two differing domains, AJAX web applications (JavaScript
/PHP/MySQL) and embedded C++, mainly DC power system controllers. If
the web application goes down, just blame IE. If the power system
fails, the customer (Telcos) can loose millions of dollars in lost call
revenue.

Very different reliability criteria and run times, but I use the same
process to ensure quality for both.

Ian.
 
M

Michael DOUBEZ

anon a écrit :
James said:
James Kanze wrote:
[...]
In this case you will not
have to do a coverage, but it is a plus. This way, the code you write
will be minimal and easier to understand and maintain.
And will not necessarily meet requirements, or even be useful.
If you write tests according to the requirements, you can be 100% sure
that statement is correct

A requirement for the function log() is that it return the
correct value, at 1 LSB, for all positive floating point values.
If you write a test directly according to that, it will take
something over 3000 centuries to run, supposing that you can
test one value per microsecond (which is beyond the capabilities
of many modern machines).

So, what you are saying here is that every math function is tested the
way you suggested?

Please, lookup "numerical analysis". The stability of an algorithm can
be proven and TDD is useless in this case.
Test are then only validation tests (test of special value, overflow
....) which are well defined.
There are things which are not possible to be unit tested, but if a code
can be unit tested, then there are no reasons not to do so.
Off course, I am talking about serious projects - not examples and hello
worlds.

That doesn't bring you a "100% sure that statement is correct" as you
said. The only thing you can do is increase correctness and robustness.

If the "100% sure" existed, security systems such as fail-back (recovery
upon unexpected error) or comity (same algorithm implement by 3
different teams - the correct value is chosen at the majority) wouldn't
exists in critical systems.

Michael
 
A

anon

Michael said:
anon a écrit :
James said:
James Kanze wrote:

[...]
In this case you will not
have to do a coverage, but it is a plus. This way, the code you write
will be minimal and easier to understand and maintain.

And will not necessarily meet requirements, or even be useful.

If you write tests according to the requirements, you can be 100% sure
that statement is correct

A requirement for the function log() is that it return the
correct value, at 1 LSB, for all positive floating point values.
If you write a test directly according to that, it will take
something over 3000 centuries to run, supposing that you can
test one value per microsecond (which is beyond the capabilities
of many modern machines).

So, what you are saying here is that every math function is tested the
way you suggested?

Please, lookup "numerical analysis". The stability of an algorithm can
be proven and TDD is useless in this case.
Test are then only validation tests (test of special value, overflow
...) which are well defined.

Thanks. I will take a look

Steps I would follow in this explicit case of a log algorithm are:
1) find a good numerical mathematical algorithm which estimates good
enough result of a log function for passed value. This result would have
to be good enough for its use.
2) implement it
3) write several tests (maybe 10 tests) only to validate

I thought the stability of a numerical algorithm should be proven in
mathematical theory.
That doesn't bring you a "100% sure that statement is correct" as you
said. The only thing you can do is increase correctness and robustness.

If the "100% sure" existed, security systems such as fail-back (recovery
upon unexpected error) or comity (same algorithm implement by 3
different teams - the correct value is chosen at the majority) wouldn't
exists in critical systems.

I never said it is possible to cover every possible case. But having a
good test is much better then having no tests at all
 
P

Pete Becker

Ian said:
We had
a perfect implementation of our misunderstanding of the specification!

That's a typical developer's attitude. A tester's attitude is that
software is never perfect. If it's passing all the tests then the tests
need more work. Now, that's entirely about testing. The development
process has to balance the cost of additional testing against the
likelihood of finding defects serious enough to warrant it.
Nevertheless, a good tester regards "we had a perfect implementation" as
a challenge.

--

-- Pete
Roundhouse Consulting, Ltd. (www.versatilecoding.com)
Author of "The Standard C++ Library Extensions: a Tutorial and
Reference." (www.petebecker.com/tr1book)
 
P

Phlip

Pete said:
That's a typical developer's attitude. A tester's attitude is that
software is never perfect.

Ah, that's why I never code review, deploy weekly, iterate monthly, never
share my code with others, never pair program, and never add new tests to
it.
 
P

Phlip

James said:
A requirement for the function log() is that it return the correct value,
at 1 LSB, for all positive floating point values. If you write a test
directly according to that, it will take something over 3000 centuries to
run, supposing that you can test one value per microsecond (which is
beyond the capabilities of many modern machines).

That is a soak test. TDD doesn't imply that. It implies that for each known
branch in the target code there's at least one simple test. Just enough to
fail if you refactored the target code and made it grossly wrong.

There are many kinds of tests, all more mundane than soaking, which you
shouldn't write first.
 
P

Phlip

What you are up against here is Kanze is one of the most aggressive and
competent reviewers out there, and he leads by reviewing. This explains
his
devastating accuracy on C++ questions. So by claiming two dumb people can
get by with pairing and TDD, instead of submitting their code to him for
review, you are taking him out of his Comfort Zone.
Thanks for the complement, but it's not personal, and it's not about me.

Oh, you had me Googling for your answer to a GOTW, in the 30s, demanding
that the code to pass the GOTW would have a complete test harness or you'd
reject it for review.
The important aspect of code review (one of them, anyway) is the fresh,
unjaundiced viewpoint; the person looking at the code has not seen it
before.

I don't know why the founding fathers of XP don't specifically specify
those; maybe they thought that shared code ownership and common workplaces
would imply them.

But we do them at work. Pair programming and _then_ the review with a third
party.
Another, perhaps even more important, is that the programmers write their
code to be reviewed.

I need a process that does all these things as a side-effect:

- make refactoring safer than hunt-n-peck
- force code to decouple
- make future changes easier
- self-document the code

I don't want to go back and retrofit those things, and I want the
self-documentation and decoupling to support code reviews.

So we use TDD, and it provides all those and more as a side-effect of rapid
development without bugs.
Someone who has never seen the code before must be capable of
understanding it quickly and without undo effort, and it must be "obvious"
why it works, and that it cannot fail. If, during review, the author has
to explain why the code works, the
code fails review.

We accept the presence of valid "TODO" comments; deploying the feature is
more important, at this juncture, than the last step of the adversarial
system. That's because the overall code quality always goes up over time,
and because we have good culture for fixing those TODOs early and often.
 
P

Phlip

Firstly, XP specifies many other systems to avoid ripping customers off. TDD
is only one of them, and its weaknesses match other systems' strengths...

James said:
How does testing help the customer to get what he really wants?

In the ideal situation, the customer liaison writes test directly into a
custom user interface, such as http://fitnesse.org

In a compliance situation, you write a page named after the regulation, and
you fill out a table of inputs and expected outputs matching that
regulation. Some XP spokesmodels have proposed that regulations should be
expressed in test-ready format. This would make compliance trivial to
inspect.
Customer focused means talking to the customer in his language, not in
yours.

That is also why the Agile movement advocates working with the customer to
develop an "Ubiquitous Language" (formerly a "System Metaphor").
A test suite doesn't do this.

"Ubiquitous" means you use the same name for the same thing _everywhere_. In
your conversations, in the story cards, in the customer-facing tests, in the
developer tests, in the code, in the database, and in the documentation.
Getting a customer to sign off a project on the basis of a test that he's
not capable of understanding is not, IMHO, showing him much respect.

That's why the Agile Manifesto recommends collaboration over contract
negotiation. If you do the features in order of business priority, and
deliver weekly, then the customer pays as they go, and they can stop at any
time with a working system that at least does something.

So a team working with a customer with no contractual obligation to keep
paying is _highly_ motivated to not rip them off, in any iteration.
To come back to an obvious case: for certain types of thread safety
problems, I don't know how to write a test which is guaranteed to fail.

For any hard algorithm, you don't write a test that checks the entire
algorithm is correct. You write a test which might fail if individual
details of the algorithm are not implemented. That's only enough to write
code to pass the test. Diabolical code could easily pass the test and fail
the algorithm - at TDD time you don't care. You only need enough testage to
get started, and maybe as the feature grows the team will decide to add soak
tests.

And developer tests are not customer tests. The customer didn't request
threading, they requested "handle loads with these profiles". I have, at
times, written customer tests which return little charts of profiles of the
system load...
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,482
Members
44,901
Latest member
Noble71S45

Latest Threads

Top