analyzing C code?

[Zach]

Hi,

I am making changes to a few modules of code and wish to test to see
if my changes have made any increases in memory utilization, execution
speed etc... The code is part of a server so lots of other stuff has
to be run AFAIK to test my code (all the server modules). Anyone know
exactly how I can test my code? Ideally I can make a writeup which
says something like, "My changes resulted in functions foo and bar
showed a N% reduction in memory usage and ran M% faster."

Thanks,
Zach

 

[artifact.one]

Hi,

I am making changes to a few modules of code and wish to test to see
if my changes have made any increases in memory utilization, execution
speed etc... The code is part of a server so lots of other stuff has
to be run AFAIK to test my code (all the server modules). Anyone know
exactly how I can test my code? Ideally I can make a writeup which
says something like, "My changes resulted in functions foo and bar
showed a N% reduction in memory usage and ran M% faster."

Thanks,
Zach

Personally, every single module of code I write gets a small custom
program
to test the module for correctness (no less than 100% code coverage)
and a small custom program to test for speed (the program just churns
through a ton of essentially random input and prints execution time
statistics). I don't actually test for memory usage, purely because I
try
to program in an extremely explicit style to make it obvious how much
memory is being used.

Combine this with a revision control system and you can essentially
track performance (and correctness) statistics of every code revision.

I feel the style of program shouldn't ever affect ability to test it.
If the
program is too complex to test properly, re-architecture the program
into smaller, individually testable parts.

I'm not sure I've really answered your question.

MC

 

[Keith Thompson]

I am making changes to a few modules of code and wish to test to see
if my changes have made any increases in memory utilization, execution
speed etc... The code is part of a server so lots of other stuff has
to be run AFAIK to test my code (all the server modules). Anyone know
exactly how I can test my code? Ideally I can make a writeup which
says something like, "My changes resulted in functions foo and bar
showed a N% reduction in memory usage and ran M% faster."

Standard C doesn't provide a way to measure total memory usage, and
only relatively crude ways to measure speed (see the clock()
function). But many systems provide various kinds of profilers that
will perform these measurements for you. Try a newsgroup that deals
with your operating system, or search for "profiler" in your system's
documentation.

 

[Malcolm McLean]

I am making changes to a few modules of code and wish to test to see
if my changes have made any increases in memory utilization, execution
speed etc... The code is part of a server so lots of other stuff has
to be run AFAIK to test my code (all the server modules). Anyone know
exactly how I can test my code? Ideally I can make a writeup which
says something like, "My changes resulted in functions foo and bar
showed a N% reduction in memory usage and ran M% faster."

There is no easy way of doing it. As Zach said, it may be possible to
isolate your modules and test them. However you really want to know how it
performs in the server. Beyond a certain level of complexity, it becomes
impossible to control the other allocations a big program makes, and so it
is not sensible to simply run with the new and the old versions and compare.

 

[CBFalconer]

I am making changes to a few modules of code and wish to test to
see if my changes have made any increases in memory utilization,
execution speed etc... The code is part of a server so lots of
other stuff has to be run AFAIK to test my code (all the server
modules). Anyone know exactly how I can test my code? Ideally I
can make a writeup which says something like, "My changes
resulted in functions foo and bar showed a N% reduction in
memory usage and ran M% faster."

Just examine the assembly language output of the compiler. This is
system specific and thus OT here. A newsgroup that deals with your
system can give more accurate answers.

 

[Keith Thompson]

Just examine the assembly language output of the compiler. This is
system specific and thus OT here. A newsgroup that deals with your
system can give more accurate answers.

I would think that measuring the actual performance of the program
would be much more definitive than examining the assembly language,
unless you happen to know the timings of all the machine's
instructions *including* memory caching effets.

 

[Christopher Layne]

I feel the style of program shouldn't ever affect ability to test it.
If the
program is too complex to test properly, re-architecture the program
into smaller, individually testable parts.

I'm not sure I've really answered your question.

MC

I think your answer was a great one.

 

[Richard]

Hi,

I am making changes to a few modules of code and wish to test to see
if my changes have made any increases in memory utilization, execution
speed etc... The code is part of a server so lots of other stuff has
to be run AFAIK to test my code (all the server modules). Anyone know
exactly how I can test my code? Ideally I can make a writeup which
says something like, "My changes resulted in functions foo and bar
showed a N% reduction in memory usage and ran M% faster."

Thanks,
Zach

You need to use a profiler. Google will help.

 

[Zach]

Personally, every single module of code I write gets a small custom
program
to test the module for correctness (no less than 100% code coverage)
and a small custom program to test for speed (the program just churns
through a ton of essentially random input and prints execution time
statistics)

Hi,

Could you include some sample code and some sample test code you'd use
to test it. Thanks.

Zach

 

[artifact.one]

Hi,

Could you include some sample code and some sample test code you'd use
to test it. Thanks.

Zach

Well, as I said, the programs are custom programs and are therefore
specific to whatever code is being tested. Here is a contrived example
to test the standard strlen() function:

#include <string.h>
#include <stdio.h>

static const struct {
const char *str;
unsigned long exp;
} tests[] = {
{ "string1", 7 },
{ "string2", 7 },
{ "str\0ing", 3 },
};
static const unsigned long size = sizeof(tests) / sizeof(tests[0]);

int main()
{
unsigned long ind;
unsigned long len;
unsigned long exp;

for (ind = 0; ind < size; ++ind) {
len = strlen(tests[ind].str);
exp = tests[ind].exp;
if (len != exp) {
printf("[%lu] %lu != %lu\n", ind, len, exp);
return 1;
}
printf("[%lu] %lu\n", ind, len);
}
return 0;
}

As you can see it's generally a case of passing input to
code and making sure the code generates the expected
output. I usually use a mix of random and maliciously
crafted input to try to get code to misbehave. Keeping
input in a list as shown makes it easy to work out exactly
where code is misbehaving as, for example, one can say
"'t_strlen:2' fails on Solaris" and easily find out exactly
what the offending input was.

(No, test 2 doesn't fail on Solaris, it's just an example).

In production code, I have one of these programs per
module and the tests are run from a Makefile target
as part of the build.

MC

 

[Zach]

On Feb 3, 8:21 am, (e-mail address removed) wrote:
[...]

Hi MC,

I see. Thanks for the example. Should one do this only for complex
code and/or code they think could have problems or should it be done
even for more trivial programs one thinks they have inspected
rigorously already (visual auditing no test cases etc..)?

Zach

 

[Ian Collins]

On Feb 3, 8:21 am, (e-mail address removed) wrote:
[...]

Hi MC,

I see. Thanks for the example. Should one do this only for complex
code and/or code they think could have problems or should it be done
even for more trivial programs one thinks they have inspected
rigorously already (visual auditing no test cases etc..)?

Always have tests. If you get into the habit of witting the tests
first, you will save your self a lot of pain.

 

[Zach]

Always have tests. If you get into the habit of witting the tests
first, you will save your self a lot of pain.

Ok, will do.

Zach