verification strategy with no specs

[alb]

Dear all,

I've been appointed to review and verify a vhdl project of about 25k
lines of code, *without* a specification document!

There are various scattered notes/docs which describes somehow some
details (*not all*), but there's no description of what the individual
parts should do, even though there are only 4 types of FPGA in the system.

It seems unbelievable to me that they got there without any spec but
this is something I cannot change. My main question here is to
understand if there exist strategies to face such type of situations and
which one is more effective.

I've started looking at the craziness at some implementation level (all
code is practically uncommented!), but I'm at the level of firing a
question to the designers for each line of code just to understand the
reason behind!

I know it sounds like a 'rescue' plan, but if anyone can point me to
some - preferably documented - direction I would greatly appreciate.

Al

 

[John Speth]

Dear all,

I've been appointed to review and verify a vhdl project of about 25k
lines of code, *without* a specification document!

There are various scattered notes/docs which describes somehow some
details (*not all*), but there's no description of what the individual
parts should do, even though there are only 4 types of FPGA in the system.

It seems unbelievable to me that they got there without any spec but
this is something I cannot change. My main question here is to
understand if there exist strategies to face such type of situations and
which one is more effective.

I've started looking at the craziness at some implementation level (all
code is practically uncommented!), but I'm at the level of firing a
question to the designers for each line of code just to understand the
reason behind!

I know it sounds like a 'rescue' plan, but if anyone can point me to
some - preferably documented - direction I would greatly appreciate.

Here's what you do: Get an audience, scratch your head and rub your
beard while looking pensive. Then declare with authority "It appears
the product meets all stated specifications".

Seriously, you'll need to write the specs before you can test it. It
sounds like that's what you're doing anyway. I sympathize with you. It
must be a huge undertaking if you actually expect to successfully
complete it.

JJS

 

[alb]

Hi John,

On 08/05/2013 18:50, John Speth wrote:
[]

I've started looking at the craziness at some implementation level (all
code is practically uncommented!), but I'm at the level of firing a
question to the designers for each line of code just to understand the
reason behind!

[]
Here's what you do: Get an audience, scratch your head and rub your
beard while looking pensive. Then declare with authority "It appears
the product meets all stated specifications".

I wish I had the kind of beard to allow me such a statement!

Seriously, you'll need to write the specs before you can test it. It
sounds like that's what you're doing anyway. I sympathize with you. It
must be a huge undertaking if you actually expect to successfully
complete it.

So your suggestion is to sit down with the current designer(s) and try
to get an higher level description of the various components in order to
define interfaces, functionality and performances.

Maybe I should size the effort and come with a proposal to the group in
order to make it effective. I'm not sure how long will it take to write
the specs from scratch and it certainly adds time to the already tight
schedule (of course!).

A very diffused sentiment in my environment comes from the false belief
that every piece of hardware or software is buggy and in the end you can
still live with it, so why bother with all these specs and
verifications? They seem to see a 'verification plan' as another
bureaucratic wall to tear down or to dodge quickly filling a couple of
formal tests required by the funding agencies.

Sometimes, I must say, I feel like tilting at windmills...

 

[Andy]

I agree, without a specification, you cannot verify function.

However, there are established design standards, regardless of function, that are prescribed to avoid certain, most often unintended, consequences andhazards.

The design can be reviewed per those standards with little or no knowledge of what the circuit is supposed to do.

Also, revising the circuit in response to the review findings will likely require some knowledge of what it was supposed to do.

I gather that more is known about what the system of FPGAs is supposed to do, than is known about what the individual FPGAs are supposed to do. If this is the case, you may have to devise some tests to exercise the intended system behavior, while instrumenting the interfaces between FPGAs to gather information on what is happening. Together with reviewing the code, one could presumably come up with a "spec" for what all of the interfaces of each FPGA are supposed to do. And from that, a functional spec can be written for each FPGA.

Such a specification will be much tighter than necessary, since you only have visibility to what they actually did, not what they really needed to do.Slight variations to what they did might still work, but it is impossible to know if you don't know what 'works' means.

Andy

 

[Rob Anderson]

Dear all,



I've been appointed to review and verify a vhdl project of about 25k

lines of code, *without* a specification document!



There are various scattered notes/docs which describes somehow some

details (*not all*), but there's no description of what the individual

parts should do, even though there are only 4 types of FPGA in the system.



It seems unbelievable to me that they got there without any spec but

this is something I cannot change. My main question here is to

understand if there exist strategies to face such type of situations and

which one is more effective.



I've started looking at the craziness at some implementation level (all

code is practically uncommented!), but I'm at the level of firing a

question to the designers for each line of code just to understand the

reason behind!



I know it sounds like a 'rescue' plan, but if anyone can point me to

some - preferably documented - direction I would greatly appreciate.



Al



--

A: Because it fouls the order in which people normally read text.

Q: Why is top-posting such a bad thing?

A: Top-posting.

Q: What is the most annoying thing on usenet and in e-mail?

Sounds like you are working with some "IP". I had a IRDA block like that I think it was written by a summer student.
The thing to do is run the test code, capture the output and assume it is a litmus test. Use DIFF on it.
The problem is that if you change things, eg. reset or clocking, the output will not be identical. So change things slowly and keep it identical.