verification strategy with no specs

Discussion in 'VHDL' started by alb, May 8, 2013.

  1. alb

    alb Guest

    Dear all,

    I've been appointed to review and verify a vhdl project of about 25k
    lines of code, *without* a specification document!

    There are various scattered notes/docs which describes somehow some
    details (*not all*), but there's no description of what the individual
    parts should do, even though there are only 4 types of FPGA in the system.

    It seems unbelievable to me that they got there without any spec but
    this is something I cannot change. My main question here is to
    understand if there exist strategies to face such type of situations and
    which one is more effective.

    I've started looking at the craziness at some implementation level (all
    code is practically uncommented!), but I'm at the level of firing a
    question to the designers for each line of code just to understand the
    reason behind!

    I know it sounds like a 'rescue' plan, but if anyone can point me to
    some - preferably documented - direction I would greatly appreciate.

    Al

    --
    A: Because it fouls the order in which people normally read text.
    Q: Why is top-posting such a bad thing?
    A: Top-posting.
    Q: What is the most annoying thing on usenet and in e-mail?
    alb, May 8, 2013
    #1
    1. Advertising

  2. alb

    John Speth Guest

    On 5/8/2013 2:13 AM, alb wrote:
    > Dear all,
    >
    > I've been appointed to review and verify a vhdl project of about 25k
    > lines of code, *without* a specification document!
    >
    > There are various scattered notes/docs which describes somehow some
    > details (*not all*), but there's no description of what the individual
    > parts should do, even though there are only 4 types of FPGA in the system.
    >
    > It seems unbelievable to me that they got there without any spec but
    > this is something I cannot change. My main question here is to
    > understand if there exist strategies to face such type of situations and
    > which one is more effective.
    >
    > I've started looking at the craziness at some implementation level (all
    > code is practically uncommented!), but I'm at the level of firing a
    > question to the designers for each line of code just to understand the
    > reason behind!
    >
    > I know it sounds like a 'rescue' plan, but if anyone can point me to
    > some - preferably documented - direction I would greatly appreciate.


    Here's what you do: Get an audience, scratch your head and rub your
    beard while looking pensive. Then declare with authority "It appears
    the product meets all stated specifications".

    Seriously, you'll need to write the specs before you can test it. It
    sounds like that's what you're doing anyway. I sympathize with you. It
    must be a huge undertaking if you actually expect to successfully
    complete it.

    JJS
    John Speth, May 8, 2013
    #2
    1. Advertising

  3. alb

    alb Guest

    Hi John,

    On 08/05/2013 18:50, John Speth wrote:
    []
    >> I've started looking at the craziness at some implementation level (all
    >> code is practically uncommented!), but I'm at the level of firing a
    >> question to the designers for each line of code just to understand the
    >> reason behind!

    []
    > Here's what you do: Get an audience, scratch your head and rub your
    > beard while looking pensive. Then declare with authority "It appears
    > the product meets all stated specifications".


    I wish I had the kind of beard to allow me such a statement!
    >
    > Seriously, you'll need to write the specs before you can test it. It
    > sounds like that's what you're doing anyway. I sympathize with you. It
    > must be a huge undertaking if you actually expect to successfully
    > complete it.


    So your suggestion is to sit down with the current designer(s) and try
    to get an higher level description of the various components in order to
    define interfaces, functionality and performances.

    Maybe I should size the effort and come with a proposal to the group in
    order to make it effective. I'm not sure how long will it take to write
    the specs from scratch and it certainly adds time to the already tight
    schedule (of course!).

    A very diffused sentiment in my environment comes from the false belief
    that every piece of hardware or software is buggy and in the end you can
    still live with it, so why bother with all these specs and
    verifications? They seem to see a 'verification plan' as another
    bureaucratic wall to tear down or to dodge quickly filling a couple of
    formal tests required by the funding agencies.

    Sometimes, I must say, I feel like tilting at windmills...
    alb, May 8, 2013
    #3
  4. alb

    Andy Guest

    I agree, without a specification, you cannot verify function.

    However, there are established design standards, regardless of function, that are prescribed to avoid certain, most often unintended, consequences andhazards.

    The design can be reviewed per those standards with little or no knowledge of what the circuit is supposed to do.

    Also, revising the circuit in response to the review findings will likely require some knowledge of what it was supposed to do.

    I gather that more is known about what the system of FPGAs is supposed to do, than is known about what the individual FPGAs are supposed to do. If this is the case, you may have to devise some tests to exercise the intended system behavior, while instrumenting the interfaces between FPGAs to gather information on what is happening. Together with reviewing the code, one could presumably come up with a "spec" for what all of the interfaces of each FPGA are supposed to do. And from that, a functional spec can be written for each FPGA.

    Such a specification will be much tighter than necessary, since you only have visibility to what they actually did, not what they really needed to do.Slight variations to what they did might still work, but it is impossible to know if you don't know what 'works' means.

    Andy
    Andy, May 9, 2013
    #4
  5. alb

    Rob Anderson Guest

    On Wednesday, May 8, 2013 2:13:40 AM UTC-7, alb wrote:
    > Dear all,
    >
    >
    >
    > I've been appointed to review and verify a vhdl project of about 25k
    >
    > lines of code, *without* a specification document!
    >
    >
    >
    > There are various scattered notes/docs which describes somehow some
    >
    > details (*not all*), but there's no description of what the individual
    >
    > parts should do, even though there are only 4 types of FPGA in the system.
    >
    >
    >
    > It seems unbelievable to me that they got there without any spec but
    >
    > this is something I cannot change. My main question here is to
    >
    > understand if there exist strategies to face such type of situations and
    >
    > which one is more effective.
    >
    >
    >
    > I've started looking at the craziness at some implementation level (all
    >
    > code is practically uncommented!), but I'm at the level of firing a
    >
    > question to the designers for each line of code just to understand the
    >
    > reason behind!
    >
    >
    >
    > I know it sounds like a 'rescue' plan, but if anyone can point me to
    >
    > some - preferably documented - direction I would greatly appreciate.
    >
    >
    >
    > Al
    >
    >
    >
    > --
    >
    > A: Because it fouls the order in which people normally read text.
    >
    > Q: Why is top-posting such a bad thing?
    >
    > A: Top-posting.
    >
    > Q: What is the most annoying thing on usenet and in e-mail?


    Sounds like you are working with some "IP". I had a IRDA block like that I think it was written by a summer student.
    The thing to do is run the test code, capture the output and assume it is a litmus test. Use DIFF on it.
    The problem is that if you change things, eg. reset or clocking, the output will not be identical. So change things slowly and keep it identical.
    Rob Anderson, Jul 1, 2013
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Aaron

    change server specs

    Aaron, Aug 28, 2004, in forum: ASP .Net
    Replies:
    3
    Views:
    364
    Scott Allen
    Aug 30, 2004
  2. Replies:
    6
    Views:
    699
    Patrick.O.Ige
    Nov 24, 2005
  3. Drew Volpe
    Replies:
    0
    Views:
    356
    Drew Volpe
    Nov 25, 2003
  4. Ilkka Huotari

    W3C Specs reformatted

    Ilkka Huotari, May 15, 2004, in forum: HTML
    Replies:
    7
    Views:
    662
    B r ia n
    May 15, 2004
  5. VK
    Replies:
    1
    Views:
    432
    Richard Tobin
    May 18, 2006
Loading...

Share This Page