Code Coverage in ModelSim

[Matthias Alles]

Hi!

I'm currently performing code coverage with ModelSim on a design.
However I observe a problem regarding the branch coverage with clocked
processes:

process(clk,rst) is
begin
if rst = '1' then

elsif clk'event and clk = '1' then

end if;
end process;

The problem I have is that the elsif-branch for the clock is not
considered to be tested, so in the statistic the branch coverage will
never reach the 100%. The detail to this line is:

File: test.vhd
Line: 70
Branch Coverage for:
elsif clk = '1' and clk'event then
Branch if: True: 140036 False: 0

But in case of the falling edge the condition should be false, only the
clk'event will always be true, since clk is part of the sensitivity list.

What am I missing??

Thanks in advance!
Matthias

 

[Mike Treseler]

File: test.vhd
Line: 70
Branch Coverage for:
elsif clk = '1' and clk'event then
Branch if: True: 140036 False: 0

But in case of the falling edge the condition should be false, only the
clk'event will always be true, since clk is part of the sensitivity list.

Coverage concerns the testbench test.vhd not the synthesis code.
Sounds like modelsim is saying that that the testbench
never tests the case of no reset and a falling edge clock.
That is true for every testbench I have ever seen,
and I might ignore this warning rather than cover it.

-- Mike Treseler

 

[Andy]

Code coverage on your RTL is what tells you to what extent you have
under- (or over-) exercised the unit under test (the RTL), and is
highly recommended, particularly in constrained-random testing. It
also points out code that may not be used, but the resulting circuitry
is not likely to be optiminzed away either.

Code coverage of your testbench, assuming requirements are traceable
to testbench code, can tell you when your testbench has executed all
the requirements (at least those that are known/traceable).

I suppose this boils down to black-box vs white-box vs grey-box
verification, which is a whole 'nother issue on its own.

We've seen this exact issue in modelsim, and it occurs because
modelsim has an optimization where positive-edge-only processes are
not even run on negative edges of the clock. IIRC, the optimisation
can be disabled, with an accompanying increase in run-time, to get rid
of the coverage issue. Or you can ignore it, but that often means
having to massage the overal coverage metrics.

I'm in a constant battle with management regarding code coverage
metrics, and this is just one of many reasons why having requirements
(or even goals) for N% code coverage is useless. Code coverage is an
excellent tool to tell you, in general, how well your testbench is
exercising the RTL and testbench code, but the percentage coverage
score is marketing hype directed at managers who see it as a perfect
score of when you are done simulating.

Thus code coverage is neither necessary nor sufficient to tell you
when you are done with verification. It is just a tool (one of many)
that helps you to decide when enough is enough.

Andy

 

[Matthias Alles]

Coverage concerns the testbench test.vhd not the synthesis code.
Sounds like modelsim is saying that that the testbench
never tests the case of no reset and a falling edge clock.
That is true for every testbench I have ever seen,
and I might ignore this warning rather than cover it.

Well the name test.vhd is maybe misleading. Actually I'm talking about
synthesizable RTL code.

Matthias

 

[Matthias Alles]

Add these cases to your set of exclusions and be done with it.

That is what I thought about first. But the thing is that if I have
hundred clocked processes in my design I have to exclude each single
test for the rising edge by hand (giving the line number in the VHDL
code), as far as I could see from the ModelSim manual. Furthermore,
every time when I change something in a source file such that the line
numbers change, I have to adjust the exclusion stuff.

Isn't there a simpler solution for exclusion?

Matthias

 

[Matthias Alles]

Hi Andy,

We've seen this exact issue in modelsim, and it occurs because
modelsim has an optimization where positive-edge-only processes are
not even run on negative edges of the clock. IIRC, the optimisation
can be disabled, with an accompanying increase in run-time, to get rid
of the coverage issue. Or you can ignore it, but that often means
having to massage the overal coverage metrics.

I had a look into the ModelSim manual but couldn't find something about
this kind of optimization. I tried to simulate with

vsim -cover -novopt work.testbench

but the problem is still there. Do you know how exactly this can be
disabled?

Matthias

 

[KJ]

That is what I thought about first. But the thing is that if I have
hundred clocked processes in my design I have to exclude each single test
for the rising edge by hand (giving the line number in the VHDL code), as
far as I could see from the ModelSim manual. Furthermore, every time when
I change something in a source file such that the line numbers change, I
have to adjust the exclusion stuff.

Isn't there a simpler solution for exclusion?

Use the -- coverage off/on pragmas

process(clk,rst) is
begin
if rst = '1' then

-- coverage off
elsif clk'event and clk = '1' then

-- coverage on
end if;
end process;

KJ

 

[HT-Lab]

Hi!

I'm currently performing code coverage with ModelSim on a design. However I
observe a problem regarding the branch coverage with clocked processes:

process(clk,rst) is
begin
if rst = '1' then

elsif clk'event and clk = '1' then

end if;
end process;

The problem I have is that the elsif-branch for the clock is not considered to
be tested, so in the statistic the branch coverage will never reach the 100%.
The detail to this line is:

File: test.vhd
Line: 70
Branch Coverage for:
elsif clk = '1' and clk'event then
Branch if: True: 140036 False: 0

But in case of the falling edge the condition should be false, only the
clk'event will always be true, since clk is part of the sensitivity list.

What am I missing??

Try vcom -coveropt=1, that should fix it,

Hans
www.ht-lab.com

 

[HT-Lab]

Try vcom -coveropt=1, that should fix it,

lets try again,

vcom -coveropt 1

Hans
www.ht-lab.com

 

[Matthias Alles]

Close... ;-)

CoverOpt is a parameter in modelsim.ini and cannot be overwritten by
vcom. But setting this parameter in the modelsim.ini to 1 and then start
the simulation with

vsim -cover -novopt dut

works!

Thanks Hans!

Matthias

 

[HT-Lab]

Close... ;-)

CoverOpt is a parameter in modelsim.ini and cannot be overwritten by vcom. But
setting this parameter in the modelsim.ini to 1 and then start the simulation
with

Seems to work fine in 6.5b,

vcom -help
# Model Technology ModelSim PE vcom 6.5b Compiler 2009.05 May 21 2009
# Usage: vcom [options] files
# Options:
# -coveropt <i> Specify a digit for code coverage optimization level: 1
through 4.

Regards,
Hans.
www.ht-lab.com

 

[Matthias Alles]

Seems to work fine in 6.5b,

Ah, OK. I am using 6.3g at the moment. It's not supported for this version.

Matthias

 

[NigelE]

Ah, OK. I am using 6.3g at the moment. It's not supported for this version.

Matthias

Take a look at the 'allfalse' exclude option.

This excludes the implicit branches of 'if' statements without an
'else'.
It's available as part of the 'coverage exclude' command and the
coverage exlclusion pragmas

- Nigel

 

[NigelE]

Take a look at the 'allfalse' exclude option.

This excludes the implicit branches of 'if' statements without an
'else'.
It's available as part of the 'coverage exclude' command and the
coverage exlclusion pragmas

- Nigel

I should have added that this requires 6.5a or later