Survey: simulator usage

Discussion in 'VHDL' started by Evan Lavelle, Oct 27, 2006.

  1. Evan Lavelle

    Evan Lavelle Guest

    Apologies for the (slightly) commercial nature of this post.

    Those of you with long memories may recall coding devices in ABEL. One
    nice thing about ABEL was that you could write very simple vector
    files to simulate your device, where a vector was something like

    [C,1,0,1,etc] -> [1,1,0,1,HHHH,01AB,etc]

    ie. setup some inputs, apply a clock pulse, and check some outputs.
    I've written some software over the years that lets me do the same
    thing in VHDL, with various extensions, and I use it to test most of
    my RTL code. It's simple, you don't need to write or know *any* VHDL
    to use it, and it gives you a pass/fail very quickly, for a module or
    a whole device.

    I'm thinking about brushing this up a bit, adding Verilog support, and
    flogging it for maybe 100 - 300 USD a go. To use it, you obviously
    still need a simulator - the software currently produces VHDL-only
    output, and uses your simulator to simulate your chip using the
    auto-generated verification code.

    This brings me to my problem. I can make the software a lot more
    sophisticated if I can generate C code, as well as (or instead of) the
    VHDL or Verilog. There are some testbenchy things which are just very
    difficult to do in pure VHDL or Verilog. *But*, most of the potential
    users of this software will be FPGA coders with a cheap simulator that
    doesn't support a C-language interface (ModelSim PE/VHDL on Windows,
    for example, doesn't do this, and presumably the FPGA-specific
    simulators don't do this either).

    What I'd really like to find out, if you can spare the time and this
    might be of interest to you, is:

    * What simulator do you use?

    * Is your RTL code in Verilog/VHDL/both?

    * Does your simulator have a C-language interface? From Verilog, or
    VHDL, or both?

    * If your simulator doesn't support C, would you be willing to upgrade
    it to use a product of this sort? Or would you prefer to get pure VHDL
    or Verilog out of this software, even if it means reduced vector file
    functionality?

    As a bonus, if you add the line "this is a great idea and I claim my
    50% discount", then you can have 50% off the (initial) purchase price,
    if I ever get around to doing this.

    You can reply here or directly to me at 'unet+50' 'at'
    'riverside-machines' 'dot' 'com'.

    Thanks -

    Evan
     
    Evan Lavelle, Oct 27, 2006
    #1
    1. Advertising

  2. Evan Lavelle

    Hans Guest

    "Evan Lavelle" <> wrote in message
    news:...
    ...

    > This brings me to my problem. I can make the software a lot more
    > sophisticated if I can generate C code, as well as (or instead of) the
    > VHDL or Verilog.


    What about supporting Tcl, most if not all simulators support this language.
    You can force and examine signal using Modelsim's Tcl.

    > There are some testbenchy things which are just very
    > difficult to do in pure VHDL or Verilog.


    Such as?

    > *But*, most of the potential
    > users of this software will be FPGA coders with a cheap simulator that
    > doesn't support a C-language interface (ModelSim PE/VHDL on Windows,
    > for example, doesn't do this, and presumably the FPGA-specific
    > simulators don't do this either).


    You can get a SystemC license for Modelsim PE which IMHO is much much easier
    to use than the FLI/PLI.

    >
    > What I'd really like to find out, if you can spare the time and this
    > might be of interest to you, is:
    >
    > * What simulator do you use?


    Modelsim SE.

    >
    > * Is your RTL code in Verilog/VHDL/both?


    VHDL.

    >
    > * Does your simulator have a C-language interface? From Verilog, or
    > VHDL, or both?


    Yes, SE comes with the FLI which I have used in the past until I discovered
    SystemC :)

    >
    > * If your simulator doesn't support C, would you be willing to upgrade
    > it to use a product of this sort?


    No since I don't understand the advantages of your product, also changing PE
    to SE or adding a SystemC license to PE is not particular cheap and it might
    be more cost effective to spend some more time on your testbench :)

    Hans
    www.ht-lab.com
     
    Hans, Oct 27, 2006
    #2
    1. Advertising

  3. -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA1

    Evan Lavelle wrote:

    > This brings me to my problem. I can make the software a lot more
    > sophisticated if I can generate C code, as well as (or instead of) the
    > VHDL or Verilog. There are some testbenchy things which are just very
    > difficult to do in pure VHDL or Verilog. *But*, most of the potential
    > users of this software will be FPGA coders with a cheap simulator that
    > doesn't support a C-language interface (ModelSim PE/VHDL on Windows,
    > for example, doesn't do this, and presumably the FPGA-specific
    > simulators don't do this either).


    Cheap/free simulators *should* support some sort of C interface.
    Icarus Verilog supports VPI and I'd be astonished if the bundled
    Verilog compilers w/ FPGA vendor tools doesn't also support VPI.

    - --
    Steve Williams "The woods are lovely, dark and deep.
    steve at icarus.com But I have promises to keep,
    http://www.icarus.com and lines to code before I sleep,
    http://www.picturel.com And lines to code before I sleep."
    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.2 (GNU/Linux)
    Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org

    iD8DBQFFQjburPt1Sc2b3ikRAlWSAJ4z5s8Hn9mQ4B0U8IevKZzmOsCkDQCg500z
    EziQS9sPImvMVTE3FP0T3f0=
    =tv0x
    -----END PGP SIGNATURE-----
     
    Stephen Williams, Oct 27, 2006
    #3
  4. Evan Lavelle wrote:
    > Apologies for the (slightly) commercial nature of this post.
    >
    > Those of you with long memories may recall coding devices in ABEL. One
    > nice thing about ABEL was that you could write very simple vector
    > files to simulate your device, where a vector was something like
    >
    > [C,1,0,1,etc] -> [1,1,0,1,HHHH,01AB,etc]
    >
    > ie. setup some inputs, apply a clock pulse, and check some outputs.
    > I've written some software over the years that lets me do the same
    > thing in VHDL, with various extensions, and I use it to test most of
    > my RTL code. It's simple, you don't need to write or know *any* VHDL
    > to use it, and it gives you a pass/fail very quickly, for a module or
    > a whole device.


    So this requires you generate all outputs ?
    If the sim does fail, what format is the failure report, and
    how does the user relate that back to a given line of test source code ?

    >
    > I'm thinking about brushing this up a bit, adding Verilog support, and
    > flogging it for maybe 100 - 300 USD a go. To use it, you obviously
    > still need a simulator - the software currently produces VHDL-only
    > output, and uses your simulator to simulate your chip using the
    > auto-generated verification code.
    >
    > This brings me to my problem. I can make the software a lot more
    > sophisticated if I can generate C code, as well as (or instead of) the
    > VHDL or Verilog. There are some testbenchy things which are just very
    > difficult to do in pure VHDL or Verilog. *But*, most of the potential
    > users of this software will be FPGA coders with a cheap simulator that
    > doesn't support a C-language interface (ModelSim PE/VHDL on Windows,
    > for example, doesn't do this, and presumably the FPGA-specific
    > simulators don't do this either).
    >
    > What I'd really like to find out, if you can spare the time and this
    > might be of interest to you, is:


    Interesting idea.
    What about doing a simple VHDL/Verilog/Demo-C version that is free, and
    a smarter version that is $$ ?

    A problem with this type of tool, is explaining to each user how
    its feature set can offer more to a given task, so the free/simple
    versions ( and good examples) do that.

    For those who want it, I think most? FPGA vendors have (free) waveform
    entry for simulation entry, so that is one competition point.

    There is also this work by Jan Decaluwe, that uses Python for
    both verilog generation, and testbench generation.
    http://myhdl.jandecaluwe.com/doku.php/start
    http://myhdl.jandecaluwe.com/doku.php/cookbook:stopwatch

    -jg
     
    Jim Granville, Oct 27, 2006
    #4
  5. Evan Lavelle <> writes:

    > Apologies for the (slightly) commercial nature of this post.
    >
    > Those of you with long memories may recall coding devices in ABEL. One
    > nice thing about ABEL was that you could write very simple vector
    > files to simulate your device, where a vector was something like
    >
    > [C,1,0,1,etc] -> [1,1,0,1,HHHH,01AB,etc]
    >
    > ie. setup some inputs, apply a clock pulse, and check some outputs.
    > I've written some software over the years that lets me do the same
    > thing in VHDL, with various extensions, and I use it to test most of
    > my RTL code. It's simple, you don't need to write or know *any* VHDL
    > to use it, and it gives you a pass/fail very quickly, for a module or
    > a whole device.


    If you really mean such a simple format, my answer is: don't bother,
    that way is obsolete for anything but the most trivial designs. Given
    the amount of testing and the complexity of many interfaces/busses
    today, I need to write my test vectors at a higher level than applying
    a bitvector per clock cycle.

    In order to do that, I normally create a library of high-level
    functions in a scripting language like Perl or Tcl, which would do
    "the dirty work" for me. Normally I would make the script library
    interface directly to a VHDL/Verilog testbench through TEXTIO files.
    With your extention, I would still have to make the script library,
    but just have a "backend" to that emits files compatible with your
    format. All I'd save is to write my VHDL testbench for the interface
    (which may could have serious performance impacts, since you make the
    simulator stop for each clock cycle and is a real no-no for
    performance), and frankly, writing the VHDL testbench isn't the
    problem.

    > doesn't support a C-language interface (ModelSim PE/VHDL on Windows,
    > for example, doesn't do this, and presumably the FPGA-specific
    > simulators don't do this either).


    Affirmative (at least for the XE version).

    > * What simulator do you use?


    Modelsim PE / XE

    > * Is your RTL code in Verilog/VHDL/both?


    VHDL, but we have to deal with Verilog as well due to backannotated GL
    sims.

    > * If your simulator doesn't support C, would you be willing to upgrade
    > it to use a product of this sort?


    Considering that the list price of Modelsim SE is 170K DKK vs 70K DKK
    for a PE license (both floating licenses), not bloody likely.

    > Or would you prefer to get pure VHDL
    > or Verilog out of this software, even if it means reduced vector file
    > functionality?


    With a little imagination, you can do a surprising amount intelligent
    work with "dumb" TEXTIO vector files, by carefully dividing the work
    between script library (vector generation) text files, and the
    testbench itself.


    Kai
    --
    Kai Harrekilde-Petersen <khp(at)harrekilde(dot)dk>
     
    Kai Harrekilde-Petersen, Oct 27, 2006
    #5
  6. Evan Lavelle

    Andy Peters Guest

    Evan Lavelle wrote:
    > Those of you with long memories may recall coding devices in ABEL. One
    > nice thing about ABEL was that you could write very simple vector
    > files to simulate your device, where a vector was something like
    >
    > [C,1,0,1,etc] -> [1,1,0,1,HHHH,01AB,etc]
    >
    > ie. setup some inputs, apply a clock pulse, and check some outputs.
    > I've written some software over the years that lets me do the same
    > thing in VHDL, with various extensions, and I use it to test most of
    > my RTL code. It's simple, you don't need to write or know *any* VHDL
    > to use it, and it gives you a pass/fail very quickly, for a module or
    > a whole device.


    This sort of test-vector entry is basically unmaintainable, and to be
    honest, I haven't written this type of go/no-go test in ages (maybe
    since the last time I used ABEL).

    I prefer to write test benches at a higher level. I like to have
    models of the things to which the DUT connects drive the DUT. For
    example, if my FPGA is a NAND flash controller that sits between a
    microprocessor bus and the NAND flash devices, I'll use a
    bus-functional model of the micro and the flash, and my test bench will
    mostly consist of MicroWriteBus() and MicroReadBus() "commands."

    I can't go back to the simple drive-a-vector test.

    -a
     
    Andy Peters, Oct 27, 2006
    #6
  7. Evan Lavelle

    bronzefury Guest


    >
    > What I'd really like to find out, if you can spare the time and this
    > might be of interest to you, is:
    >
    > * What simulator do you use?


    NC-Verilog

    >
    > * Is your RTL code in Verilog/VHDL/both?


    Verilog

    >
    > * Does your simulator have a C-language interface? From Verilog, or
    > VHDL, or both?


    I'm sure it does

    >
    > * If your simulator doesn't support C, would you be willing to upgrade
    > it to use a product of this sort? Or would you prefer to get pure VHDL
    > or Verilog out of this software, even if it means reduced vector file
    > functionality?


    Frankly, as an ASIC designer. I probably wouldn't use this language. In
    the future, we plan to use System Verilog.

    >
    > As a bonus, if you add the line "this is a great idea and I claim my
    > 50% discount", then you can have 50% off the (initial) purchase price,
    > if I ever get around to doing this.
    >
    > You can reply here or directly to me at 'unet+50' 'at'
    > 'riverside-machines' 'dot' 'com'.
    >
    > Thanks -
    >
    > Evan
     
    bronzefury, Oct 28, 2006
    #7
  8. Evan Lavelle

    Evan Lavelle Guest

    On Sat, 28 Oct 2006 08:18:51 +1300, Jim Granville
    <> wrote:

    >So this requires you generate all outputs ?


    Only if you actually need to test them. If a given vector file wants
    to check a specific port, but that port is a don't care for most of
    the vectors, then you just enter a '-' for that port in the expected
    outputs. You could also specify the expected outputs algorithmically,
    if possible: you can declare variables and do arithmetic on them, and
    use them as module inputs or test values.

    >If the sim does fail, what format is the failure report, and
    >how does the user relate that back to a given line of test source code ?


    Here's an example from a chip I've just done. Line 392 of one of the
    vector files looks like (note that I've put in a line break to try to
    avoid wrapping problems on your newsreader):

    [11C 1 0011 0 5A5A5A5A xxxxxxx xxxxxxx zzzzzzzz zzzzzzzz] ->
    [-0 -------- ---- EE FF 011 5A5A5A5A 011 5A5A5A5A ------- -- --]

    If I change the last '5A5A5A5A' on the expected outputs to '5B5A5A5A',
    then ModelSim reports:

    # ** Note: L2HDB error: line 392 (expected 0x5b5a5a5a, got 0x5a5a5a5a)
    # Time: 4381 ns Iteration: 0 Instance: /testbench
    # ** Note: simulation complete: 305 out of 306 vectors passed.
    # Time: 7007400 ps Iteration: 0 Instance: /testbench

    The pin's identified, the line of the vector file, and the time.

    >Interesting idea.
    > What about doing a simple VHDL/Verilog/Demo-C version that is free, and
    >a smarter version that is $$ ?
    >
    > A problem with this type of tool, is explaining to each user how
    >its feature set can offer more to a given task, so the free/simple
    >versions ( and good examples) do that.


    Hmmm... maybe a 'linear' version (ie. no smarts) for free, and
    looping/variables/procedures/macros etc. for $$? My recollection is
    that ABEL just gave you the dumb version, but it seemed useful at the
    time.

    Yes, explaining that this is (hopefully) useful is difficult. I went
    through this with a friend recently, who never writes testbenches, and
    who spent years on ABEL. He wasn't impressed... :(

    > For those who want it, I think most? FPGA vendors have (free) waveform
    >entry for simulation entry, so that is one competition point.


    I've never got this. Does anyone actually do that?

    > There is also this work by Jan Decaluwe, that uses Python for
    >both verilog generation, and testbench generation.
    >http://myhdl.jandecaluwe.com/doku.php/start
    >http://myhdl.jandecaluwe.com/doku.php/cookbook:stopwatch


    Yes, I know of Jan's stuff, but hadn't realised he also does TB
    generation - I'll have a look at that.

    Evan
     
    Evan Lavelle, Oct 29, 2006
    #8
  9. Evan Lavelle

    Evan Lavelle Guest

    On 27 Oct 2006 15:53:42 -0700, "Andy Peters" <>
    wrote:

    >This sort of test-vector entry is basically unmaintainable, and to be
    >honest, I haven't written this type of go/no-go test in ages (maybe
    >since the last time I used ABEL).
    >
    >I prefer to write test benches at a higher level. I like to have
    >models of the things to which the DUT connects drive the DUT. For
    >example, if my FPGA is a NAND flash controller that sits between a
    >microprocessor bus and the NAND flash devices, I'll use a
    >bus-functional model of the micro and the flash, and my test bench will
    >mostly consist of MicroWriteBus() and MicroReadBus() "commands."
    >
    >I can't go back to the simple drive-a-vector test.


    Yes, makes sense; I've got no problem with high-level testbenches. If
    a device or module has lots of buried state, and it's not obvious when
    any outputs or observable points should change as a result of an input
    change, or even *what* that change should be, then a test vector
    approach is pretty useless. If your problem is, for example, apply an
    opcode to a CPU and check that a memory location changes at some
    arbitrary later time, then you need to be a lot smarter. But, horses
    for courses. Consider:

    - Your complex DUT is probably built of simpler modules. If a module
    was just an FSM, a register block, a decoder, or a bus interface, for
    example, and you could test the module simply by writing
    20/50/100/whatever vectors, would it be worthwhile? Or do you just
    stitch the whole thing together and test the top level?

    - Many complete chips are surprisingly simple (seen from the outside,
    anyway). I first used this code for a (large) DSP chain for baseband
    processing. I got MATLAB data for the inputs, with expected data for
    the outputs. Once I'd automated the test, verification was trivial.
    Seven years on, and I've just finished part of a Spartan-3. Nothing
    fancy - 25K marketing-gates - it just translates accesses between two
    different buses connected to 15 external chips, handles bursts, has a
    register block with control and status ops, does some address mapping
    between the various chips, and so on. I did a basic test in a day by
    writing 200-odd vectors, and I had a complete-ish test of 752 vectors
    with a bit more work. Whatever way I'd tested it, these 752 vectors
    would have existed somewhere, if only implicitly in the VHDL code, or
    even in the documentation. Why not make the vectors themselves the
    test? The rest of it is just unnecessary verbiage that you have to
    repeat again, and again, for each new module and device you do.

    - What are MicroWriteBus() and MicroReadBus()? I can do macros and
    pass parameters to the macros; you can call the macros from wherever
    you want in the vector file. I can also do basic C-like control
    structures - looping, branching based on tested values, and so on.

    Ok, is it worth any more than $0 now? :)

    Evan
     
    Evan Lavelle, Oct 29, 2006
    #9
  10. Evan Lavelle

    Evan Lavelle Guest

    On Fri, 27 Oct 2006 15:39:39 GMT, "Hans" <> wrote:

    >No since I don't understand the advantages of your product, also changing PE
    >to SE or adding a SystemC license to PE is not particular cheap and it might
    >be more cost effective to spend some more time on your testbench :)


    Ok, here's a really trivial example. You've just coded a D-type F/F -
    how do you test it?

    - Option 1: code a testbench module, instantiate your D-type, provide
    a clock and some way to apply inputs (in this case, probably just
    explicitly driving the pins sequentially as required), check that the
    output is as required, inform the user that the test passed or failed.
    If it failed, tell the user where. Not difficult, but tedious. Also
    much more tedious if you need to do a timing sim, and you have to
    ensure that the inputs are driven only for the required setup and hold
    times, and the outputs only change between tCOmin and tCOmax. It's
    also not productive; this is all code you've written a thousand times
    before. And it may have a bug in it, and the testbench itself must be
    tested before you can use it to test your DUT.

    - Option 2:

    ([CLK, RST, D] -> [Q, QB])
    [x, 0, x] -> [0, 1]; async reset
    [C, 1, 1] -> [1, 0]; clock 1
    [C, 1, 0] -> [0, 1]; clock 0

    As I said, it's trivial. You don't need to know any VHDL or Verilog,
    and you don't have to write a test program. All that tedious stuff has
    been done for you. Of course, You're not going to test a D-type, but
    there are certainly far more complex cases where this is useful. And
    there are more complex cases where this isn't useful; see my reply to
    Andy.

    Evan
     
    Evan Lavelle, Oct 29, 2006
    #10
  11. Evan Lavelle

    Frank Buss Guest

    Evan Lavelle wrote:

    > I'm thinking about brushing this up a bit, adding Verilog support, and
    > flogging it for maybe 100 - 300 USD a go. To use it, you obviously
    > still need a simulator - the software currently produces VHDL-only
    > output, and uses your simulator to simulate your chip using the
    > auto-generated verification code.


    Why using a code generator when you can write such bit vector tests within
    VHDL itself?

    constant rom_test: unsigned(63 downto 0) := x"cf000000e4ec2933";

    rom := rom_test;
    for j in 1 to 64 loop
    wait until ds_wire = '0';
    wait until ds_wire = 'Z';
    if rom(0) = '0' then
    ds_wire <= '0';
    wait for 30 us;
    ds_wire <= 'Z';
    wait for 1 us;
    end if;
    rom := shift_right(rom, 1);
    end loop;

    -- wait until latched
    wait until data_valid = '1';

    -- check, if read process was successful
    assert rom_test(55 downto 8) = ds_wire_rom report "ROM read error" severity
    failure;

    You can refactor the for-loop to a function or procedure, if you need it
    multiple times.

    --
    Frank Buss,
    http://www.frank-buss.de, http://www.it4-systems.de
     
    Frank Buss, Oct 29, 2006
    #11
  12. Evan Lavelle wrote:
    > On Sat, 28 Oct 2006 08:18:51 +1300, Jim Granville
    >>Interesting idea.
    >> What about doing a simple VHDL/Verilog/Demo-C version that is free, and
    >>a smarter version that is $$ ?
    >>
    >> A problem with this type of tool, is explaining to each user how
    >>its feature set can offer more to a given task, so the free/simple
    >>versions ( and good examples) do that.

    >
    >
    > Hmmm... maybe a 'linear' version (ie. no smarts) for free, and
    > looping/variables/procedures/macros etc. for $$? My recollection is
    > that ABEL just gave you the dumb version, but it seemed useful at the
    > time.


    ABEL is still supplied with some tool flows.
    CUPL is also still usefull for CPLD end of the design, and it is both
    free, and has some looping/macros in the functional Sim engine.

    CUPL is functional-sim only, so has no timing smarts itself, but you CAN
    take the fitter (verilog/vhdl) output, and run a testbench on that.

    CPLD tend to be less of a timing puzzle than FPGAs so this extra flow is
    not used much.

    What CUPL _does_ offer, is test vectors into the JED file, which is
    good for smaller devices that go via device programmers.

    CUPL also allows a '*' in the vector input, which means CUPL
    generates (fills-in) the H.L.Z in the output, and that can
    speed the test process.

    If you then want to lock in all test values, you can paste the
    SimOut values back into the SimIN file, but I tend to scan the
    SimOUT carefully, and then rely on the same results being generated
    from the same stimulus. (These are stable tools.)

    Here is a simple example:
    SimIN
    0C1* ** **** *** **** 111111 ****** ****** ***** C *
    SimOUT
    0C1L LL LHHH LHL ZZZZ 111111 LLLLLL LLLLLL LLHHH C L


    >
    > Yes, explaining that this is (hopefully) useful is difficult. I went
    > through this with a friend recently, who never writes testbenches, and
    > who spent years on ABEL. He wasn't impressed... :(
    >
    >
    >> For those who want it, I think most? FPGA vendors have (free) waveform
    >>entry for simulation entry, so that is one competition point.

    >
    >
    > I've never got this. Does anyone actually do that?


    I have a friend who thinks this is great. He does a lot of BUS
    interface work.
    Personally, I prefer using a text editor, and command line tools.
    I can see the appeal of waveform entry, but the custom-file risk of
    that outweighs the benefits, for me.

    -jg
     
    Jim Granville, Oct 29, 2006
    #12
  13. Evan Lavelle

    Andy Peters Guest

    Evan Lavelle wrote:

    > - What are MicroWriteBus() and MicroReadBus()? I can do macros and
    > pass parameters to the macros; you can call the macros from wherever
    > you want in the vector file. I can also do basic C-like control
    > structures - looping, branching based on tested values, and so on.


    Shorthand notation for "generic test-bench procedure/task call that
    performs a read or write on my microprocessor's external bus, from the
    micro's point-of-view."

    -a
     
    Andy Peters, Oct 30, 2006
    #13
  14. Evan Lavelle

    Andy Guest

    Evan Lavelle wrote:
    > Ok, is it worth any more than $0 now? :)


    In a word, no.

    Why go to the trouble of learning a new language to try to do things
    like macros, loops, random stimulus, etc. when you have the power of
    the VHDL language at your disposal in a VHDL testbench? Now, if you
    have vectors from an external model/simulation, those can be applied
    with text-io relatively easily from within a vhdl testbench that will
    run on any vhdl simulator.

    My "unit level tests" are usually at a high enough level that I need a
    lot more capability than is available in any vector based scripting
    language.

    Andy
     
    Andy, Oct 30, 2006
    #14
  15. Evan Lavelle

    Thomas Guest

    Yet Another Survey: What are code generators worth?(was: Re: Survey: simulator usage)

    Generalising the question of this discussion I would like to know what
    does the group here think that code generators are worth?

    Say Someone comes out with a code generator which requires some input
    data say, D_in and generates a bunch of output code say, C_out.

    Given that C_out is something that you already write for your design
    wether the code generator exists or not, when will the code generator
    look like a fesible option?

    Given the following man days(Including coding, verfying and debugging
    time) for writing D_in and generating C_out VS manual writing of
    C_out, what is the price you will be willing to pay for the code
    generator in each case?
    ---------------------------------------------
    Development time in Mandays
    ---------------------------------------------
    D_in : Manual C_out
    ---------------------------------------------
    1 : 10
    1 : 100
    1 : 1000

    If such a code generator appears on the market will your management
    wait until a bigname EDA comes out with a me too product or will they
    buy it from a noname developer?
    Regards
    Thomas

    Andy wrote:
    > Evan Lavelle wrote:
    > > Ok, is it worth any more than $0 now? :)

    >
    > In a word, no.
    >
    > Why go to the trouble of learning a new language to try to do things
    > like macros, loops, random stimulus, etc. when you have the power of
    > the VHDL language at your disposal in a VHDL testbench? Now, if you
    > have vectors from an external model/simulation, those can be applied
    > with text-io relatively easily from within a vhdl testbench that will
    > run on any vhdl simulator.
    >
    > My "unit level tests" are usually at a high enough level that I need a
    > lot more capability than is available in any vector based scripting
    > language.
    >
    > Andy
     
    Thomas, Nov 1, 2006
    #15
  16. Evan Lavelle

    bronzefury Guest

    Re: Yet Another Survey: What are code generators worth?(was: Re: Survey: simulator usage)

    Personally, I would avoid code generators to generate RTL for design.
    Coding RTL is not a bottleneck for me, even on multi-million gate designs.
    Some IP is provided by 3rd-party, other IP our developers code. It is
    fairly easy to write Verilog once you have tried and true coding style. Why
    would I want to learn a proprietary language to generate a standard
    language? (and get locked-in?). In addition, I know exactly what hardware
    I'll generate when I code. For logic circuits that are used often, the HDL
    addresses that by allowing designers to create modules or primitives. Who
    knows what a code generator will output. I also wouldn't want to be locked
    in to a proprietary.

    System Verilog addresses many of the my department's needs.

    Even if a big name EDA company were to come out with one, I doubt this type
    of tool would be purchased.

    I think if you want to attack something in the area of chip design, you've
    got to address the parts in the design flow that addresses a company's
    development bottlenecks such as verification or physical design.


    "Thomas" <> wrote in message
    news:...
    > Generalising the question of this discussion I would like to know what
    > does the group here think that code generators are worth?
    >
    > Say Someone comes out with a code generator which requires some input
    > data say, D_in and generates a bunch of output code say, C_out.
    >
    > Given that C_out is something that you already write for your design
    > wether the code generator exists or not, when will the code generator
    > look like a fesible option?
    >
    > Given the following man days(Including coding, verfying and debugging
    > time) for writing D_in and generating C_out VS manual writing of
    > C_out, what is the price you will be willing to pay for the code
    > generator in each case?
    > ---------------------------------------------
    > Development time in Mandays
    > ---------------------------------------------
    > D_in : Manual C_out
    > ---------------------------------------------
    > 1 : 10
    > 1 : 100
    > 1 : 1000
    >
    > If such a code generator appears on the market will your management
    > wait until a bigname EDA comes out with a me too product or will they
    > buy it from a noname developer?
    > Regards
    > Thomas
    >
    > Andy wrote:
    >> Evan Lavelle wrote:
    >> > Ok, is it worth any more than $0 now? :)

    >>
    >> In a word, no.
    >>
    >> Why go to the trouble of learning a new language to try to do things
    >> like macros, loops, random stimulus, etc. when you have the power of
    >> the VHDL language at your disposal in a VHDL testbench? Now, if you
    >> have vectors from an external model/simulation, those can be applied
    >> with text-io relatively easily from within a vhdl testbench that will
    >> run on any vhdl simulator.
    >>
    >> My "unit level tests" are usually at a high enough level that I need a
    >> lot more capability than is available in any vector based scripting
    >> language.
    >>
    >> Andy

    >
     
    bronzefury, Nov 2, 2006
    #16
  17. Evan Lavelle

    Evan Lavelle Guest

    Re: Yet Another Survey: What are code generators worth?(was: Re: Survey: simulator usage)

    On 1 Nov 2006 08:10:07 -0800, "Thomas" <> wrote:

    >If such a code generator appears on the market will your management
    >wait until a bigname EDA comes out with a me too product or will they
    >buy it from a noname developer?


    I think you may be in even more trouble than me...

    You need to be specific about what you're generating. A human is a
    code generator; are you replacing humans? Are you automating something
    that's already done? How many people do it? How much time do they
    spend on it? Can you do it better than a human? Or just faster? Or
    just cheaper? Or is it too difficult for a human to do at all?

    Are you generating IP? If so, you're in trouble. People don't like
    buying IP, and they'll only buy it from someone they already trust.
    The IP itself is unimportant, anyway: what matters is how well it has
    been verified, and whether you can prove it. You're selling a
    verification plan and coverage metrics, not HDL code.

    Who are you targetting? Is it FPGA users? If so, you're shafted. If
    not, you've got no market. Google for anyone who's tried to sell IP
    for FPGAs; I can't believe anyone's ever made money doing this. If the
    users like it, the FPGA vendors will do it themselves and sell it for
    *zero*. Remember, the vendors are selling silicon; they need to bundle
    software and IP for free to get market share. If you don't understand
    this, think Microsoft and Netscape, WMP, etc.

    Finally, remember that the EDA market is tiny, and 3 mid-sized fish
    are desperately trying to make a living in it, primarily by eating all
    the smaller fish. If the FPGA vendors don't get you, then the EDA
    vendors will. The men's underwear market is bigger than EDA. Perhaps
    you should be designing a better pair of Y-fronts...

    :)

    Evan
     
    Evan Lavelle, Nov 2, 2006
    #17
  18. Evan Lavelle

    Thomas Guest

    Re: Yet Another Survey: What are code generators worth?(was: Re: Survey: simulator usage)

    bronzefury wrote:
    > Personally, I would avoid code generators to generate RTL for design.

    There are different levels of generated code. e.g.
    1> Schematic to code generation e.g. FSM, block diagram,module
    integration.
    2> Translating code from one format to another.
    3> Template based code generators etc.
    While some are simple (have <10x improvement in total coding time)
    others are complex and can result in saving days or even weeks of
    coding time. While I personally wont care for the first case the second
    case is what interests me.
    > Coding RTL is not a bottleneck for me, even on multi-million gate designs.
    > Some IP is provided by 3rd-party, other IP our developers code. It is
    > fairly easy to write Verilog once you have tried and true coding style. Why
    > would I want to learn a proprietary language to generate a standard
    > language? (and get locked-in?).

    Applying the 80-20 rule, I would say that 80% of any code is bug free
    but identifying the other 20% can take up 80% of my development time.
    so would it not be in my interest to reduce the amount of hand written
    code by generating code wherever possible? Also quite a few bugs are
    because of copy, paste, modify errors whihc a template based code
    generator can easily fix.
    > In addition, I know exactly what hardware
    > I'll generate when I code. For logic circuits that are used often, the HDL
    > addresses that by allowing designers to create modules or primitives. Who
    > knows what a code generator will output. I also wouldn't want to be locked
    > in to a proprietary.

    If we are generating RTL the generated code will be verilog or VHDL and
    if it is a testcase it will be e,c,SystemVerilog or SystemC and if at
    any time we want to "break out" of the code generator "lock-in" we can
    always take the generated code and continue from there.

    Note: I dont have a code generator to sell. But have seen various
    reusable code generators written by my team mates which have saved
    weeks and weeks of coding+debugging effort and was wondering what will
    it cost in the market.
    Regards
    Thomas.
    >
    > System Verilog addresses many of the my department's needs.
    >
    > Even if a big name EDA company were to come out with one, I doubt this type
    > of tool would be purchased.
    >
    > I think if you want to attack something in the area of chip design, you've
    > got to address the parts in the design flow that addresses a company's
    > development bottlenecks such as verification or physical design.
    >
    >
    > "Thomas" <> wrote in message
    > news:...
    > > Generalising the question of this discussion I would like to know what
    > > does the group here think that code generators are worth?
    > >
    > > Say Someone comes out with a code generator which requires some input
    > > data say, D_in and generates a bunch of output code say, C_out.
    > >
    > > Given that C_out is something that you already write for your design
    > > wether the code generator exists or not, when will the code generator
    > > look like a fesible option?
    > >
    > > Given the following man days(Including coding, verfying and debugging
    > > time) for writing D_in and generating C_out VS manual writing of
    > > C_out, what is the price you will be willing to pay for the code
    > > generator in each case?
    > > ---------------------------------------------
    > > Development time in Mandays
    > > ---------------------------------------------
    > > D_in : Manual C_out
    > > ---------------------------------------------
    > > 1 : 10
    > > 1 : 100
    > > 1 : 1000
    > >
    > > If such a code generator appears on the market will your management
    > > wait until a bigname EDA comes out with a me too product or will they
    > > buy it from a noname developer?
    > > Regards
    > > Thomas
    > >
    > > Andy wrote:
    > >> Evan Lavelle wrote:
    > >> > Ok, is it worth any more than $0 now? :)
    > >>
    > >> In a word, no.
    > >>
    > >> Why go to the trouble of learning a new language to try to do things
    > >> like macros, loops, random stimulus, etc. when you have the power of
    > >> the VHDL language at your disposal in a VHDL testbench? Now, if you
    > >> have vectors from an external model/simulation, those can be applied
    > >> with text-io relatively easily from within a vhdl testbench that will
    > >> run on any vhdl simulator.
    > >>
    > >> My "unit level tests" are usually at a high enough level that I need a
    > >> lot more capability than is available in any vector based scripting
    > >> language.
    > >>
    > >> Andy

    > >
     
    Thomas, Nov 3, 2006
    #18
  19. Evan Lavelle

    bronzefury Guest

    Re: Yet Another Survey: What are code generators worth?(was: Re: Survey: simulator usage)

    We also had some inhouse people working on these kinds of tools. Some
    issues we ran into are that code would be written incorrectly or a bug
    found, then not only would the RTL have to change, the code generator would
    have to change as well. Which means you're debugging two things and
    requiring more people and time to develop a product. To make a long story
    short, for us, it was a big waste of time because the time they spent on it
    would have been better spent verifying the 20% that you speak of,
    understanding performance bottlenecks, planning for future product,
    training, or fine tuning an architecture.

    I really think code generators are a tough sell. I can't see the value
    proposition. Coding just isn't on the top ten list of headaches for me or
    any of the designers that I work with. Again, verifying takes up most of
    the development effort.

    From your response, it sounds like you've benefited. Maybe you know more
    than I do and your experience was much more enjoyable. If you believe in
    it... put it on the market

    "Thomas" <> wrote in message
    news:...
    > bronzefury wrote:
    >> Personally, I would avoid code generators to generate RTL for design.

    > There are different levels of generated code. e.g.
    > 1> Schematic to code generation e.g. FSM, block diagram,module
    > integration.
    > 2> Translating code from one format to another.
    > 3> Template based code generators etc.
    > While some are simple (have <10x improvement in total coding time)
    > others are complex and can result in saving days or even weeks of
    > coding time. While I personally wont care for the first case the second
    > case is what interests me.
    >> Coding RTL is not a bottleneck for me, even on multi-million gate
    >> designs.
    >> Some IP is provided by 3rd-party, other IP our developers code. It is
    >> fairly easy to write Verilog once you have tried and true coding style.
    >> Why
    >> would I want to learn a proprietary language to generate a standard
    >> language? (and get locked-in?).

    > Applying the 80-20 rule, I would say that 80% of any code is bug free
    > but identifying the other 20% can take up 80% of my development time.
    > so would it not be in my interest to reduce the amount of hand written
    > code by generating code wherever possible? Also quite a few bugs are
    > because of copy, paste, modify errors whihc a template based code
    > generator can easily fix.
    >> In addition, I know exactly what hardware
    >> I'll generate when I code. For logic circuits that are used often, the
    >> HDL
    >> addresses that by allowing designers to create modules or primitives.
    >> Who
    >> knows what a code generator will output. I also wouldn't want to be
    >> locked
    >> in to a proprietary.

    > If we are generating RTL the generated code will be verilog or VHDL and
    > if it is a testcase it will be e,c,SystemVerilog or SystemC and if at
    > any time we want to "break out" of the code generator "lock-in" we can
    > always take the generated code and continue from there.
    >
    > Note: I dont have a code generator to sell. But have seen various
    > reusable code generators written by my team mates which have saved
    > weeks and weeks of coding+debugging effort and was wondering what will
    > it cost in the market.
    > Regards
    > Thomas.
    >>
    >> System Verilog addresses many of the my department's needs.
    >>
    >> Even if a big name EDA company were to come out with one, I doubt this
    >> type
    >> of tool would be purchased.
    >>
    >> I think if you want to attack something in the area of chip design,
    >> you've
    >> got to address the parts in the design flow that addresses a company's
    >> development bottlenecks such as verification or physical design.
    >>
    >>
    >> "Thomas" <> wrote in message
    >> news:...
    >> > Generalising the question of this discussion I would like to know what
    >> > does the group here think that code generators are worth?
    >> >
    >> > Say Someone comes out with a code generator which requires some input
    >> > data say, D_in and generates a bunch of output code say, C_out.
    >> >
    >> > Given that C_out is something that you already write for your design
    >> > wether the code generator exists or not, when will the code generator
    >> > look like a fesible option?
    >> >
    >> > Given the following man days(Including coding, verfying and debugging
    >> > time) for writing D_in and generating C_out VS manual writing of
    >> > C_out, what is the price you will be willing to pay for the code
    >> > generator in each case?
    >> > ---------------------------------------------
    >> > Development time in Mandays
    >> > ---------------------------------------------
    >> > D_in : Manual C_out
    >> > ---------------------------------------------
    >> > 1 : 10
    >> > 1 : 100
    >> > 1 : 1000
    >> >
    >> > If such a code generator appears on the market will your management
    >> > wait until a bigname EDA comes out with a me too product or will they
    >> > buy it from a noname developer?
    >> > Regards
    >> > Thomas
    >> >
    >> > Andy wrote:
    >> >> Evan Lavelle wrote:
    >> >> > Ok, is it worth any more than $0 now? :)
    >> >>
    >> >> In a word, no.
    >> >>
    >> >> Why go to the trouble of learning a new language to try to do things
    >> >> like macros, loops, random stimulus, etc. when you have the power of
    >> >> the VHDL language at your disposal in a VHDL testbench? Now, if you
    >> >> have vectors from an external model/simulation, those can be applied
    >> >> with text-io relatively easily from within a vhdl testbench that will
    >> >> run on any vhdl simulator.
    >> >>
    >> >> My "unit level tests" are usually at a high enough level that I need a
    >> >> lot more capability than is available in any vector based scripting
    >> >> language.
    >> >>
    >> >> Andy
    >> >

    >
     
    bronzefury, Nov 3, 2006
    #19
  20. Evan Lavelle

    Thomas Guest

    Re: Yet Another Survey: What are code generators worth?(was: Re: Survey: simulator usage)

    Evan Lavelle wrote:
    > On 1 Nov 2006 08:10:07 -0800, "Thomas" <> wrote:
    >
    > >If such a code generator appears on the market will your management
    > >wait until a bigname EDA comes out with a me too product or will they
    > >buy it from a noname developer?

    >
    > I think you may be in even more trouble than me...
    >
    > You need to be specific about what you're generating. A human is a
    > code generator; are you replacing humans? Are you automating something
    > that's already done? How many people do it? How much time do they
    > spend on it? Can you do it better than a human? Or just faster? Or
    > just cheaper? Or is it too difficult for a human to do at all?


    I am looking at a problem set where
    We tend to do the same task over and over again for each project. After
    doing it a few times we know all the possible permutations and
    combinations and which combination to apply for which problem but it
    still consumes most of our coding time. Some of the tasks in the
    problem set may require one person to be dedicated for it throughout
    the project cycle.

    The generated code could be similar to what is written by a human and
    only reduces the development and debug time. For e.g. it may do the
    same job with the benifit of giving 10X to 1000X improvement in the
    coding+debugging time.

    As an example think about writing
    y=a+b; and using a tool for synthesis and layout. VS using a layout
    editor to design the ckt one CMOS transistor at a time followed by
    spice simulations. If this were the 70's will you pay for the current
    crop of synthesis+layout tools?

    >
    > Are you generating IP? If so, you're in trouble. People don't like
    > buying IP, and they'll only buy it from someone they already trust.

    While my company is currently working on a set of IP's for its client.
    My question was more towards the market fesibility of code generators
    beacause every year EE Times covers a couple of developers who have
    tried to sell their solution to specific problems in IC design which
    could be replicated in a a few hundred lines of perl code.
    > The IP itself is unimportant, anyway: what matters is how well it has
    > been verified, and whether you can prove it. You're selling a
    > verification plan and coverage metrics, not HDL code.

    What I am looking at is a set of tools which help in taking the project
    from Spec to samples to Product faster than the current manual process
    would allow. So It could involve
    1. The methodology.
    2. The code generators to generate verbose codes from a brief high
    level input.
    3. Resolving any other bottlenecks.

    Going with the Microsoft analogy below. I am looking at something
    similar to windows apps development using visual C++ where you fireup
    the relevent wizard click a few buttons and are presented with a
    template based code framework runnning in a few tens of thousands of
    lines in which you insert your alogrithm and other code fragments.

    Note:I know about the code bloat issue for MS VC which a tool for HW
    design cannot afford.

    >
    > Who are you targetting? Is it FPGA users? If so, you're shafted. If
    > not, you've got no market. Google for anyone who's tried to sell IP
    > for FPGAs; I can't believe anyone's ever made money doing this. If the
    > users like it, the FPGA vendors will do it themselves and sell it for
    > *zero*. Remember, the vendors are selling silicon; they need to bundle
    > software and IP for free to get market share. If you don't understand
    > this, think Microsoft and Netscape, WMP, etc.


    The audience for such tools could be both FPGA and ASIC users. I know
    about the risk any successful product will have from the established
    EDA vendors copying it and offering it for free. so my question of
    wether your manager will buy it from a no name developer or wait for a
    Known EDA Vendor to come up with a me too product.
    >
    > Finally, remember that the EDA market is tiny, and 3 mid-sized fish
    > are desperately trying to make a living in it, primarily by eating all
    > the smaller fish. If the FPGA vendors don't get you, then the EDA
    > vendors will.

    I dont think any developer will regret beig eaten by a big name EDA
    vendor if the price is right :)
    Also I think EDA vendors are currently more interested in me having a
    bigger code base to verify so that meeting coverage goals will require
    more run time or more simulators.

    > The men's underwear market is bigger than EDA. Perhaps
    > you should be designing a better pair of Y-fronts...
    >
    > :)

    After a decade in the Industry I think I am too old to change
    professions. I will make sure that my kid keeps his options open ;)
    Regards
    Thomas
    >
    > Evan
     
    Thomas, Nov 3, 2006
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. christopher diggins

    Survey: Frequency of usage of smart pointers

    christopher diggins, Nov 12, 2004, in forum: C++
    Replies:
    9
    Views:
    552
    christopher diggins
    Nov 13, 2004
  2. Axter

    Survey for Compiler usage

    Axter, Feb 8, 2006, in forum: C++
    Replies:
    1
    Views:
    276
    benben
    Feb 9, 2006
  3. Andrew Berg

    Re: Python 2.x and 3.x usage survey

    Andrew Berg, Dec 30, 2013, in forum: Python
    Replies:
    0
    Views:
    52
    Andrew Berg
    Dec 30, 2013
  4. Chris Angelico

    Re: Python 2.x and 3.x usage survey

    Chris Angelico, Dec 30, 2013, in forum: Python
    Replies:
    0
    Views:
    56
    Chris Angelico
    Dec 30, 2013
  5. Dennis Lee Bieber

    Re: Python 2.x and 3.x usage survey

    Dennis Lee Bieber, Dec 31, 2013, in forum: Python
    Replies:
    0
    Views:
    57
    Dennis Lee Bieber
    Dec 31, 2013
Loading...

Share This Page