Seriously struggling with C

[Herbert Rosenau]

Herbert Rosenau wrote:


Not if you're running a system with (hard) real time requirements.
Conditional breakpoints slow things down horribly, and almost
invariably give you a system that just simply does not work, as it
breaks all the timing constraints.

Then you are using a miserble debugger and should use a better one
that knows how to handle conditional breakpoints better. It should no
problem for a debugger in time critical environments to check simple
condition on a breakpoint withoul loosing too much time.

It's much better to add a condition
to the code itself, and when it hits N-th execution stop and dump out
any system state you're interested in.

That needs multiple whole edit, compile, link cycles with the risk of
falsifying the code in an unwanted manner.

When you can point exactly to the statement that is the cause for the
flaw you would have no need to set a breakpoint anymore because you
have the bug in hand and can fix it. The problem is to get real cause,
not the point of the break.

--
Tschau/Bye
Herbert

Visit http://www.ecomstation.de the home of german eComStation
eComStation 1.2 Deutsch ist da!

 

[Flash Gordon]

Of course, this all has to be judged on a cost/benefit basis. It
is difficult to use the software to test itself if the CPU is not
even executing the boot instructions stored in the boot ROM, for
instance.

We had a simple solution to that on one project, which required some HW.
There was a "BIT" (Built In Test) display which was designed to power up
showing a processor failure. The SW tested the processor then changed
the display it indicate a ROM failure. The SW then did a checksum on the
ROM and if that passed changed the display to show a RAM failure. It
then did a RAM test and if that passed changed it to show the base
system was OK. These initial tests were all written in assembler.

This could lead it to showing a processor failure instead of a ROM
failure, but the chances of getting an system OK display if anything was
faulty was minimal.

I often find it useful to check for "impossible" conditions in
low-level code, but I have to trade that against the lack (at that
point) of a strategy for dealing with such conditions, and the
slowdown effect of performing an "unnecessary" test.

It is, indeed, all a matter of balance. In the case of the test
equipment I had most involvement with (by coincidence I also think it
the best done ;-) ) the SW was actually running on an HP workstation,
not an embedded system. We got the SW to do a basic check of the test
rig when started, and then ensured that all exceptions where handled.
When it came to writing the tests (which were all done at system level)
we even managed to trigger some of these traps for "impossible"
conditions by simulating conditions it was reasonable for the SW to come
across! There was no requirement for it to be fast, so if the exception
handling slowed things down it was not a problem. One other benefit,
when a customer phoned us up saying, "it's not working!" I was able to
identify on the basis of the error message that a specific card was
faulty, removed or, most likely, a switch on it had been moved. I was
assured that although they had been using the workstation for other
things there was no reason and no way this switch would have been
changed. Later on I was told that switch *had* been changed. An
"impossible" condition occurred, the HW problems was identified, and the
solution given, all from my desk with the kit several thousand miles
away and me not having to check through the code.

Only optimise out unnecessary tests if you have seen you have a
performance problem ;-)

I continue to endeavour to trap all the exceptional conditions even
impossible ones somewhere in SW that I write (maybe in a higher or lower
level function than where the problem occurs), and sometimes I am
surprised when one of those traps is triggered, so I consider it a good
investment in time.

I've also found that my customers are always happier of the SW comes up
with an error message that looks like it might mean something than if
the software crashes, continues "working" but produces erroneous
results, or comes up with a generic error such as "Segmentation
violation". Of course, the customer is even happier when everything
works perfectly ;-)

I've just about finished a two month project developing some significant
functionality for one of our pieces of server SW. This new functionality
makes heavy use of third-party libraries. I can across lots of things
where I though, oh, that can't possibly go wrong, but the traps are
still in there in the code the customer is testing and will remain in
the SW throughout it's life. If there is a bug in the third-party
library that makes one of these "impossible" conditions happen this
means I will be able to look at the logs and identify accurately where
the failure is. So will a maintenance programmer 10 years down the line,
because each trap will put a unique message in the log.
--
Flash Gordon
Living in interesting times.
Web site - http://home.flash-gordon.me.uk/
comp.lang.c posting guidlines and intro -
http://clc-wiki.net/wiki/Intro_to_clc

 

[Joe Wright]

What *is* an ICE? In my 30+ years of programming experience I never
have seen the acronym.


The program I write have nothing to do with interrupts and timing at all,
so what are you talking about?


What system are you talking about? I am doing a simple combinatorial
problem. A simple back-tracking problem, it goes wrong when your
back-tracking goes wrong.

BTW, changing memory lay-out can be beneficial when finding bugs. This
was one of the ways I detected an off-by-one error in the garbage
collector of the Algol 68 system we used quite a long time ago.

ICE is In Circuit Emulator. It is computerized hardware which lets you
write and test programs which will eventually be burned into ROM on the
target device. The ICE has, at the end of a cable, the pin-base of the
device you are programming for. You plug this in to your project
breadboard as if it were the final device. When you are happy with how
it works, the system will create a file that you can use to burn the
code into the target device. Unplug the ICE and plug in the device and
off you go.

 

[Chris Hills]

Not if you're running a system with (hard) real time requirements.
Conditional breakpoints slow things down horribly, a

only if you are using a cheap and nasty ICE. A decent ICE has
(virtually) no effect on the circuit.

nd almost
invariably give you a system that just simply does not work, as it
breaks all the timing constraints.

The only time I have ever found that to happen was when the hardware was
running on such close tolerances it would not have worked anyway in
production with the normal spread of tolerances on parts.

It's much better to add a condition
to the code itself, and when it hits N-th execution stop and dump out
any system state you're interested in.

What crap! you are changing the system in order to look at it. So the
system you are looking at is NOT the same as the original system you
wanted to test.

I have used ICE where the system you test IS the one you ship with no
test code permitted in the released system. This is for high integrity
systems with 20 years MTBF where no bug fixing is possible.

Your methods would have you removed from the project very quickly.

 

[Flash Gordon]

I've found never such a situation.

You may not, but I have.

> It was always at lest
- knowing how to use the debug

I know how. I even use it when appropriate.

- exact knowledge of how the system works

Being one of the authors of one of the systems and the maintainer of the
main library it uses I have a pretty good idea.

(what does an API do excactly on which condition and what side
effects
are prior, during and after the system api gets called are active

So tell me, how am I to know from the documentation of the other
application that it can't cope with 5 simultaneous connections when even
the company selling it doesn't know that?

I generally know the interfaces to the extent they are documented, and
over time I learn a lot that is not documented. It does not change the
reality of the situation that I and others sometimes find it *far*
easier to go through logs.

Oh, for such remote environment we had setup our CORBA applications
running on the same mashine as CORBA is designed to work local or
remote even under another OS.

<fx: Phones up Microsoft> Excuse me, can you please add a CORMA
interface to SQL Server 2000?

<fx: Phones up Oracle> Excuse me, can you please add a CORBA interface
to Oracle Financials?

Hmm. That's strange. They don't seem to be prepared to make the required
changes this month.

> Setting up an test environment had not
even a single change of the code needed - but a bit defferent
environment. So debugging server and client in parallel was more nice
as to fiund some technices to setup debug logfiles.

Se tell me how I am going to debug Oracle Financials, or MS SQL Server,
or Sun Financials, or any of the many other systems my SW has to
interface with in parallel with mine.

Not to mention that if I break my application sometimes the application
on the far end will time out the connection forcing me to start again.

I had an similar situation. One of our customers reported constantly
an abnormal end of one application - but nobody and nothing was able
to reproduce that until we crippled a mashine to exact the same
hardware environment the user had.

<fx: me phones up Boss> Hello, can you authorise a purchase order for
half a million pounds cost of kit so I can replicate the customer
environment? No, I can't get it second hand because it is brand new
current kit.

Even renting kit can be rather expensive.

You see, our customers are much bigger than us and buy much more
expensive HW than we can afford.

Not to mention needing licenses for all the applications it is being
interfaced to, and not having a couple of hundred people I can sit down
at PCs entering data (or the 200 PCs for them to sit at) or...

> So it was easy in 9 from 10 runs to
produce the abort - but it was impossible to find the cause for even
then. Using a debugger to catch it was resulting in unable to get the
abort reproduced! Using the same debugger then more advanced and it
was absolutely clear what the cause was and using the brain it was
easy to fix by manually syncronise the threads at that critical point
against the stand rules. The next version of the OS brought a new
system API to make that syncronising inside the kernel to make that
problem going away.

<snip>

Just because it worked for you does not mean it is always the best approach.

I'll use logging where appropriate, debuggers when appropriate, and
sitting down with a printout and a pen when appropriate. All methods
have problems and all have advantages.
--
Flash Gordon
Living in interesting times.
Web site - http://home.flash-gordon.me.uk/
comp.lang.c posting guidlines and intro -
http://clc-wiki.net/wiki/Intro_to_clc

 

[Flash Gordon]

I think there's a misunderstanding on which side of the table you and
Ian are assuming you're sitting at. I believe Ian meant /he/ was a
customer running acceptance tests on, say, a piece of 3rd party
software. You seem to be looking at it from the 3rd party POV.

I was not the third party. I was one of the main developers and one of
the main testers. I was also involved several time in going through the
customer acceptance for this kit (we did several updates to cope with
changes in the kit it was designed to test).

IMHO, both of you are correct. When we sell a device we run it through a
battery of standardised tests to prove to the customer it works. When
we buy in a piece of software to use in our device, we test it in all
sorts of nasty ways looking to break it.

Obviously, internally, we also have tests for /our/ code that try to
break it before it gets to the customer.

Yes, that is my point. When writing test for your SW, whether system
level, function level, or somewhere in between, do everything in your
power to break it.

Also we successfully achieved this with the sole use of system level
tests written after the SW (after we thought we had finished developer
testing and debugging), so it is possible to do with tests written after
the SW if the testing is done with the right attitude and the SW is
otherwise developed well.

I'm not saying other methods might not have been more efficient.

Although you and, I'm sure, most people here do try to break SW when
testing, I've come across far to many people in my career who when
writing internal tests at whatever level write test to prove the kit
works instead and thus fail to find the problems. Once to the extent
that they failed to find a problem I had reported to the designer on
reviewing the design & code as I found out a lot later when another
project tried to reuse the code and hit one of the problems I had raised.
--
Flash Gordon
Living in interesting times.
Web site - http://home.flash-gordon.me.uk/
comp.lang.c posting guidlines and intro -
http://clc-wiki.net/wiki/Intro_to_clc

 

[CBFalconer]

.... snip ...

Yes, that is my point. When writing test for your SW, whether
system level, function level, or somewhere in between, do
everything in your power to break it.

When I had something ready to test, I used to look for the least
computer ready personnel, usually a sweet young thing, or a
near-retirement secretary, and get them to try it out. I would
tell them not to worry about breaking anything, just do whatever
comes into your mind. This served both to reduce their fear of the
evil machines, and to uncover my goofs.

--
Some informative links:
http://www.geocities.com/nnqweb/
http://www.catb.org/~esr/faqs/smart-questions.html
http://www.caliburn.nl/topposting.html
http://www.netmeister.org/news/learn2quote.html

 

[Ian Collins]

Although you and, I'm sure, most people here do try to break SW when
testing, I've come across far to many people in my career who when
writing internal tests at whatever level write test to prove the kit
works instead and thus fail to find the problems. Once to the extent
that they failed to find a problem I had reported to the designer on
reviewing the design & code as I found out a lot later when another
project tried to reuse the code and hit one of the problems I had raised.

That's one of the reasons test first is a good thing, the tests are the
design and the code is written to pass the tests, thus implementing the
design.

I agree wholeheartedly that system level tests must be written with the
objective of breaking the code. This is hard for the original
developers to do and in my opinion, best left to dedicated testers. I'm
not saying it can't be done by the developers, but it takes a lot of
discipline.

 

[Vladimir S. Oka]

only if you are using a cheap and nasty ICE. A decent ICE has
(virtually) no effect on the circuit.

Yes, if the condition is simple "Nth pass" type, or of similar
complexity. I doubt that anything more complex than that is so light,
and I really had that in mind.

The only time I have ever found that to happen was when the hardware
was running on such close tolerances it would not have worked anyway
in production with the normal spread of tolerances on parts.


What crap! you are changing the system in order to look at it. So the
system you are looking at is NOT the same as the original system you
wanted to test.

If you're so strict in this, then even the (virtually) no effect of the
ICE is no good, as the system won't be the same either. If you /really/
don't want to change system timing, then maybe adding debug code that
stays in even in production is the way to ensure timing is always the
same? (In what I do it's quite feasible, as the debug output is simply
not wired out for the end user.)

(I'll have it known that I'd rather not mess with either conditional
breakpoints or debug output, in the first place.)

I have used ICE where the system you test IS the one you ship with no
test code permitted in the released system. This is for high integrity
systems with 20 years MTBF where no bug fixing is possible.

I admit I haven't been involved in anything even close to this level
(it's still hard real time, but consumer electronics, and with 9-18
months before thrown put for the latest and greatest)...

Your methods would have you removed from the project very quickly.

....but you'd be surprised how quickly I can get into line once I'm told
the specs. ;-)

--
BR, Vladimir

You probably wouldn't worry about what people think of you if you could
know how seldom they do.
-- Olin Miller.

 

[CBFalconer]

.... snip ...

If you're so strict in this, then even the (virtually) no effect
of the ICE is no good, as the system won't be the same either. If
you /really/ don't want to change system timing, then maybe adding
debug code that stays in even in production is the way to ensure
timing is always the same? (In what I do it's quite feasible, as
the debug output is simply not wired out for the end user.)

All this foofaraw assumes the system has hardware debugging
assistance in the first place. This is not guaranteed by the C
standard. In fact, even the ability to set and use breakpoints is
not guaranteed (which will generally foul up the timing). What is
guaranteed is the ability to execute printf and similar statements.

Even systems with hardware debug assistance usually have sharp
limits, such as in the count of breakpoints settable. Even their
use can alter timing, since enabling them almost certainly alters a
microcode path during instruction fetch.

--
"If you want to post a followup via groups.google.com, don't use
the broken "Reply" link at the bottom of the article. Click on
"show options" at the top of the article, then click on the
"Reply" at the bottom of the article headers." - Keith Thompson
More details at: <http://cfaj.freeshell.org/google/>
Also see <http://www.safalra.com/special/googlegroupsreply/>

 

[Chris Hills]

Yes, if the condition is simple "Nth pass" type, or of similar
complexity. I doubt that anything more complex than that is so light,
and I really had that in mind.

No, even filtered trace with multiple (nested) conditions and watch
points make no difference on a decent ICE.

If you're so strict in this, then even the (virtually) no effect of the
ICE is no good, as the system won't be the same either.

The effect a good ICE has will be less than the normal manufacturing
tolerances on a circuit. I have had this confirmed by more than one ICE
user who looked into this.

If you /really/
don't want to change system timing, then maybe adding debug code that
stays in even in production is the way to ensure timing is always the
same? (In what I do it's quite feasible, as the debug output is simply
not wired out for the end user.)

This is the case with some systems. However systems at this level
usually demand that an ICE is used.

(I'll have it known that I'd rather not mess with either conditional
breakpoints or debug output, in the first place.)


I admit I haven't been involved in anything even close to this level
(it's still hard real time, but consumer electronics, and with 9-18
months before thrown put for the latest and greatest)...

try telecoms systems and not just the deep or sub-marine cables. then
there are satellites, smart cards not to mention cars (some 100 MCU in
each these days)

...but you'd be surprised how quickly I can get into line once I'm told
the specs. ;-)

Fair enough.

 

[Chris Hills]

All this foofaraw assumes the system has hardware debugging
assistance in the first place. This is not guaranteed by the C
standard.

The C standard is irrelevant to this discussion. The Sw could be C. C++,
Assembler PLm, pascal etc an ICE is a hardware debugger. s simulator
is a software debugger.

In fact, even the ability to set and use breakpoints is
not guaranteed (which will generally foul up the timing).

Breakpoints will change the timing because the stop the flow but watch
points do not neither do conditional breakpoints, trace etc

What is
guaranteed is the ability to execute printf and similar statements.

but these change the timing and memory map. They do not give any certain
indication of the state of the program under test.

Even systems with hardware debug assistance usually have sharp
limits, such as in the count of breakpoints settable.

an ICE is a hardware system. Al the ICE I have used do have a limit on
break points. you can only set one per address in memory. on an 8 bit
system this is in execcs of 64K breakpoints.

For JTAG systems it is only 2 HW breakpoints but unlimited Sw
breakpoints. However the number of breakpoints actually needed is
usually quite small

Even their
use can alter timing, since enabling them almost certainly alters a
microcode path during instruction fetch.

As the breakpoints are done in the ICE not on the target this is not
correct. The effect a good ice has on the target HW is less than the
fluctuations in production tolerances. In any event it is MANY orders
of magnitude less than the disruption caused by using printf etc

 

[CBFalconer]

The C standard is irrelevant to this discussion. The Sw could be C.
C++, Assembler PLm, pascal etc an ICE is a hardware debugger. s
simulator is a software debugger.

In case you failed to notice, this newsgroup is c.l.c, on which the
C language, as described by the various standards, is discussed.
BTW Pascal is always capitalized.

Breakpoints will change the timing because the stop the flow but
watch points do not neither do conditional breakpoints, trace etc

You are extremely naive. Without hardware support watchpoints also
alter the timing. Probably much more so than conditional
breakpoints, since they require intercepting every instruction.

but these change the timing and memory map. They do not give any
certain indication of the state of the program under test.

Not the memory map. You seem determined to find a free lunch.
TANSTAAFL.

 

[Flash Gordon]

That's one of the reasons test first is a good thing, the tests are the
design and the code is written to pass the tests, thus implementing the
design.

I'm not, and never have, argued that writing the tests first is wrong.
In fact, I agree with it.

You still have to design your tests to try and make the code fail rather
than assuming that the code will be written sensibly.

I agree wholeheartedly that system level tests must be written with the
objective of breaking the code. This is hard for the original
developers to do and in my opinion, best left to dedicated testers. I'm
not saying it can't be done by the developers, but it takes a lot of
discipline.

Agreed. Some of us have had the required level of discipline, and also
the required time, a lot don't.

BTW, I've also been one of an independent team of testers in the past.

 

[Flash Gordon]

Flash Gordon wrote:
... snip ...

When I had something ready to test, I used to look for the least
computer ready personnel, usually a sweet young thing, or a
near-retirement secretary, and get them to try it out. I would
tell them not to worry about breaking anything, just do whatever
comes into your mind. This served both to reduce their fear of the
evil machines, and to uncover my goofs.

I agree that is a good thing.

Something else I like to do when a bug is discovered is to make sure the
rest of the code handles the problems caused by the bug as well as
possible and sort that our before fixing the bug. E.g. making sure that
the SW correctly reports to the user that the SW is bust (with a message
that will give me some clue) and then aborts without corrupting the
database.

 

[Chris Hills]

In case you failed to notice, this newsgroup is c.l.c, on which the
C language, as described by the various standards, is discussed.
BTW Pascal is always capitalized.

However the C standard has no relevance to the debugging in question.

You are extremely naive. Without hardware support watchpoints also
alter the timing.

All the way though this thread I have been referring to ICE and
Emulators which are hardware.

Probably much more so than conditional
breakpoints, since they require intercepting every instruction.


Not the memory map. You seem determined to find a free lunch.
TANSTAAFL.

Using printf to debug will change the memory map. You have to include it
and therefore it will not be the same as when you don't include it/

 

[CBFalconer]

.... snip ...

All the way though this thread I have been referring to ICE and
Emulators which are hardware.

No you haven't. Firstly, such things are OT for c.l.c. Secondly,
this disagreement began with your objection to one line conditional
statements, as being hard to set breakpoints on, to which I
responded that printfs were a more useful debug mechanism to me.
ICE and emulators have nothing to do with that. In fact an ICE is
almost impossible to implement for modern systems where one inch is
a long transmission line. I have yet to see an emulator that
preserves the timing of the hardware.

Hint: The name of a newsgroup is a clue to the matters discussed.

--
"If you want to post a followup via groups.google.com, don't use
the broken "Reply" link at the bottom of the article. Click on
"show options" at the top of the article, then click on the
"Reply" at the bottom of the article headers." - Keith Thompson
More details at: <http://cfaj.freeshell.org/google/>
Also see <http://www.safalra.com/special/googlegroupsreply/>

 

[Keith Thompson]

No you haven't. Firstly, such things are OT for c.l.c. Secondly,
this disagreement began with your objection to one line conditional
statements, as being hard to set breakpoints on, to which I
responded that printfs were a more useful debug mechanism to me.

[snip]

I don't think it was Chris who objected to the one-line conditional
statement. I *think* I remember who it was, but I'm too lazy to track
it down.

 

[Richard G. Riley]

No you haven't. Firstly, such things are OT for c.l.c. Secondly,
this disagreement began with your objection to one line conditional
statements, as being hard to set breakpoints on, to which I
responded that printfs were a more useful debug mechanism to me.

[snip]

I don't think it was Chris who objected to the one-line conditional
statement. I *think* I remember who it was, but I'm too lazy to track
it down.

It was me : for the simple reason it makes it nigh on impossible to
set break points on condition branching.

printfs only work in the most trivial environment and certainly are
useless on anything running in a multiprocess server like environment.

But I think thats all been covered.

 

[CBFalconer]

No you haven't. Firstly, such things are OT for c.l.c. Secondly,
this disagreement began with your objection to one line conditional
statements, as being hard to set breakpoints on, to which I
responded that printfs were a more useful debug mechanism to me.

[snip]

I don't think it was Chris who objected to the one-line conditional
statement. I *think* I remember who it was, but I'm too lazy to track
it down.

Oh, maybe it was that objectionable joker who refused to stay on
topic, and got plonked by many (including me). Riley or
something. It still doesn't really matter, ICEs are hardly on
topic here.

--
"If you want to post a followup via groups.google.com, don't use
the broken "Reply" link at the bottom of the article. Click on
"show options" at the top of the article, then click on the
"Reply" at the bottom of the article headers." - Keith Thompson
More details at: <http://cfaj.freeshell.org/google/>
Also see <http://www.safalra.com/special/googlegroupsreply/>