Test coverage checking when test driver is separate from the tested program.

[Tomasz Wegrzanowski]

Hello,

I have a program compiler.rb, that compiles something.
The test driver is a separate program, that runs compiler.rb, then feeds
the results to a virtual machine, and checks if it gets expected results.

Now I'd like to verify that every single line of compiler.rb is
covered by tests.
How can I make rcov merge information from multiple runs of compiler.rb ?

 

[Mauricio Fernandez]

I have a program compiler.rb, that compiles something.
The test driver is a separate program, that runs compiler.rb, then feeds
the results to a virtual machine, and checks if it gets expected results.

Now I'd like to verify that every single line of compiler.rb is
covered by tests.
How can I make rcov merge information from multiple runs of compiler.rb ?

You can use the --aggregate option, which was added in 0.7.0.
Essentially, you just have to run your program using rcov with the
--aggregate FILE option:
rm -f coverage.data
rcov -t --no-html --aggregate coverage.data bin/compiler.rb -- -args --to --compiler.rb bar.src
rcov --aggregate coverage.data bin/compiler.rb -- -some --other --args foo.src

You can skip HTML report generation in all but the last run with --no-html;
the -t (text summary) option stops rcov from complaining about the lack of
files to analyze.

There's some additional information about how to do it both manually and with a
Rake task at
http://eigenclass.org/hiki.rb?rcov+0.7.0

Note that --aggregate is fairly slow since all the execution count and
coverage information must be saved in the specified file, as well as the code
that was loaded (since in a later execution different files could be
loaded/parsed).

 

[Mauricio Fernandez]

You can use the --aggregate option, which was added in 0.7.0.

[...]

Sorry, I misread the first paragraph and answered somewhat aside of the
question.

In the case you describe, you could create a small wrapper that would look
more or less like this:

#!/usr/bin/env ruby

system("rcov", "--aggregate", "coverage.data", "--no-html", "-t",
"compiler.rb", "--", *ARGV)
exit $?.exitstatus

Then you could select which script (the wrapper or the actual compiler.rb) is
to be executed with a symlink.
Alternatively, you could change the driver to select the script or execute
rcov with the appropriate arguments directly, but at that point you'd have to
implement some way to specify whether rcov is to be used or not, if you want
to allow both.

Hope this helps.