Automatically finding gaps in unit test coverage ?

  • Thread starter Tomasz Wegrzanowski
  • Start date
T

Tomasz Wegrzanowski

Hello,

Is it possible to somehow automatically check which parts of code
are not covered by unit tests ?
 
L

Logan Capaldo

Hello,

Is it possible to somehow automatically check which parts of code
are not covered by unit tests ?

Check out rcov. This is exactly what rcov was created for. It makes
these awesome HTML coverage reports.
 
R

Robert Klemme

Tomasz said:
Hello,

Is it possible to somehow automatically check which parts of code
are not covered by unit tests ?

I don't know of any mechanism built into RubyUnit or a code coverage
tool. But this is something you could do: use set_trace_func to trace
all methods invoked, then at the end of a single class's test case or
when all tests are done go through your classes and check their
#instance_methods to verify that all of them were called. But that
still does not give you info whether you had a proper test case for the
method or the method was called by some other method etc. Maybe you can
extend the tracing to record only methods called from a unit test.
Still you would not know whether the method was simply called or whether
return values were checked or whether they were checked properly.

Kind regards

robert
 
T

Tomasz Wegrzanowski

Check out rcov. This is exactly what rcov was created for. It makes
these awesome HTML coverage reports.

Thanks, it was exactly what I needed. :)

By the way how are people usually testing the convenience functions
that print something to STDOUT.
 
T

Tomasz Wegrzanowski

I don't know of any mechanism built into RubyUnit or a code coverage
tool. But this is something you could do: use set_trace_func to trace
all methods invoked, then at the end of a single class's test case or
when all tests are done go through your classes and check their
#instance_methods to verify that all of them were called. But that
still does not give you info whether you had a proper test case for the
method or the method was called by some other method etc. Maybe you can
extend the tracing to record only methods called from a unit test.
Still you would not know whether the method was simply called or whether
return values were checked or whether they were checked properly.

Well, one cannot have everything :). rcov is pretty useful for finding
missing tests. My only problem with it so far is its inability to test
coverage of metaprogrammed code, but I guess that can't be helped.
 
M

Mauricio Fernandez

[...]

You can do that in rcov with the --test-unit-only option.
Well, one cannot have everything :). rcov is pretty useful for finding
missing tests. My only problem with it so far is its inability to test
coverage of metaprogrammed code, but I guess that can't be helped.

Are you referring to this?

$ rcov --no-html --text-coverage --no-color a.rb
================================================================================
a.rb
================================================================================

module StupidMeta
def ugly_make_meth(meth)
module_eval <<-EOF, __FILE__, __LINE__+1
def #{meth}(a,b)
a + #{meth.to_s.inspect} + b
end
EOF
end

def make_meth(meth)
define_method(meth) do |a,b|
!! a + meth.to_s + b
!! end
end

end

class X
extend StupidMeta
ugly_make_meth :foo
make_meth :bar
end

# we don't actually call them, so they should both be uncovered
#x = X.new
#x.foo("method ", " here")
#x.bar("method ", " here")


As you can see, Module#define_method will work better, if you can use it at
all (that is, if you don't need a block argument, under 1.8, and don't care
about the closure and the speed hit). Another advantage is that it will tell
you if the code is well formed (syntactically correct) much earlier.

The problem with the string forms of eval is that I've had to go to great
lengths to make sure rcov marks heredocs correctly, and the above annotation
*is* correct.

Indeed, in the above snippet the heredoc in ugly_make_meth has been
"executed", in some sense (this interpretation gains some weight due to the
interpolation going on; there's actually something being evaluated there).

But there are two interpretations for that string: as a mere heredoc, and as
executable code that defines another method. There are thus also two ways to
think about it as far as coverage is concerned.

I'm thinking that maybe some sort of pragma would make sense:

def ugly_make_meth(meth)
module_eval <<-EOF, __FILE__, __LINE+1__ # rcov:noheredoc
def #{meth}(a,b)
a + #{meth.to_s.inspect} + b
end
EOF
end


As for other limitations, I think I know how to handle generated code that
cannot be meaningfully associated to your sources, but I feel it's not worth
the effort.

Is there anything else you'd like to see addressed?
 
R

Robert Klemme

Mauricio said:
You can do that in rcov with the --test-unit-only option.

[snip]

Mauricio, thanks for the detailed explanation! Learn something new
every day...

Kind regards

robert
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads


Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top