Transaction based testbench - Effective encapsulation of the client 'transactors'?

[Andrew FPGA]

Hi Newsgroup,
I am implementing a transaction based testbench and I'm trying to find
an effective way of encapsulating the client calls into my test
harness.

I have a "Test Harness" that is a single VHDL file and contains Bus
Functional Models(BFMs) for all the DUT interfaces. This test harness
also instantiates the DUT and generates the clk/reset for the DUT.
These BFM's can be thought of as servers - they provide a high level
transaction based interface.

Following Janick Bergeron's advice(Writing Testbenches - Functional
Verification of HDL Models, Kluwer Academic Publishers) I have created
a Test Harness Package in a separate file. This contains the BFM
transaction interface definition. Each BFM has a signal record(global)
for sending transactions to the BFM and another signal record(global)
for receiving transactions back from the BFM.

I then have a "Test Case" module(its another file) that acts as a
client to the Server BFM's. The client-server protocol is based around
transactions(no shit). The server BFM implements the server end of the
protocol obviously. Now, where does the client end of the protocol get
implemented? Bergeron suggests encapsulating the client end of the
protocol using procedures located in the test harness package(pg 243).
The problem is that the client server protocol is transaction based -
the server waits on a transaction( wait on ToServer'transaction) and
the client waits on an acknowledge/complete transaction from the
server. To quote the LRM "Attributes of an actual are never passed into
a subprogram". So I can't use the wait on FromServer'transaction
contruct inside my client procedure. Is there an elegent solution to
this? Why is Bergeron suggesting something that is not possible (or is
my understanding wrong somewhere)?

One solution is to use events rather than transactions. i.e. toggle a
signal rather than simply assign to it. But I find the wait on
blah'transaction much more elegant.

Regards
Andrew

 

[ALuPin]

Hi Andrew,

Try Janick's discussion forum

http://verificationguild.com/index.php

Rgds
André

 

[Mike Treseler]

The problem is that the client server protocol is transaction based -
the server waits on a transaction( wait on ToServer'transaction) and
the client waits on an acknowledge/complete transaction from the
server. To quote the LRM "Attributes of an actual are never passed into
a subprogram". So I can't use the wait on FromServer'transaction
contruct inside my client procedure.

You don't have to pass the signal if it
is packaged or otherwise in scope.

http://groups.google.com/groups?q=vhdl+procedure+scope+process+signal+package

Why is Bergeron suggesting something that is not possible (or is
my understanding wrong somewhere)?

Isn't there a full working example in the book?

-- Mike Treseler

 

[Andrew FPGA]

Andre: Thanks, I'll post there too.

You don't have to pass the signal if it
is packaged or otherwise in scope.

Ok, I can see how that works for a single server. What if you have
multiple servers of the same type though.(e.g. the DUT has multiple
interfaces of the same type). Then I want to be able to pass into the
client procedure the signals for the particular server I want to
communicate with. Having a seperate set of client procedures for every
server instance of the same type gets messy pretty quick.

http://groups.google.com/groups?q=vhdl+procedure+scope+process+signal+package


Isn't there a full working example in the book?

Lots of code snippets but no full example for this client/server stuff.
Bergeron shows the client procedure declarations in the package file
but does not go as far as showing the procedure definitions in the
package body. In his client procedure declarations he even has the "to
server" and "from server" transaction signals in the formal paramater
list.

Thank you for your comments Mike.

Regards
Andrew

 

[Mike Treseler]

Having a seperate set of client procedures for every
server instance of the same type gets messy pretty quick.

I agree. Consider a single main test process
and keep things clean and procedural.
See: http://home.comcast.net/~mike_treseler/

Lots of code snippets but no full example for this client/server stuff.

I won't critique a book I haven't read,
but I would hesitate to take on such
a complicated test architecture
without seeing one working example.

Thank you for your comments Mike.

You are welcome.

-- Mike Treseler

 

[Andy Peters]

Andre: Thanks, I'll post there too.


Ok, I can see how that works for a single server. What if you have
multiple servers of the same type though.(e.g. the DUT has multiple
interfaces of the same type). Then I want to be able to pass into the
client procedure the signals for the particular server I want to
communicate with. Having a seperate set of client procedures for every
server instance of the same type gets messy pretty quick.

Lots of code snippets but no full example for this client/server stuff.
Bergeron shows the client procedure declarations in the package file
but does not go as far as showing the procedure definitions in the
package body. In his client procedure declarations he even has the "to
server" and "from server" transaction signals in the formal paramater
list.

The reason for this "client/server" approach to VHDL testbenches is
because of VHDL's rules for passing signals through to procedures. For
example, it'd be most excellent if your test bench could call a
DoPCIBurstWrite() procedure inside your pcimaster entity with no
parameters other than say, burst length, and have it go off and do it.
But you can't do that -- you can't call a procedure in another entity
without passing every single signal as a parameter, and if you have a
bunch of signals, things get pretty ugly pretty quickly.

So the "client/server" approach basically has your test bench process
toggling a signal (like, say, doPCIBurstWrite) that is detected by
server process in the server's entity, and then the server process
calls the procedure(s) that actually toggle the signals that need
togglin'.

It's a right royal Pain In The Ass.

This is one area where Verilog kicks VHDL's ass. Of course, Verilog
can easily kick YOUR ass here, since you can't call a function/task if
that function/task is already being executed. Verilog will happily
overwrite everything in the task and bonk you. The fixes here are to
put a "guard" around the task (as noted in Bergeron's book) or use
Verilog-2001 and mark the task as re-entrant.

-a

 

[Andrew FPGA]

For those interested, the solution I am using is to toggle a signal
when signalling between my client/server. That way I can simply wait on
an event.

Janick Bergeron had this to say:
"You are correct - and your understanding has not gone wrong: there is
an error (in fact several errors!) in the book. See
http://janick.bergeron.com/wtb/errata1.html for a list."

Mike: I really do buy into the single process style for my
synthesizable code. This style was quite a revelation to me when I
discovered it here a month or so ago. The only disadvantage I see is
that often I have FSM's with outputs that don't need to be registered
so perhaps some wasted flops with this method. But I am more than
willing to trade this for the huge improvement in code readability. For
now testbenches are a different story, I'm still coming to grips with
the style proposed in that UART example. Probably just need more time
at it and I will get it.

Andy: Its interesting that reuse is often given as a major advantage
and reason why one should persue the client/server approach. But after
reading your comment I suspect yours is the more fundamental reason
this approach is required.

Regards
Andrew

 

[Mike Treseler]

The reason for this "client/server" approach to VHDL testbenches is
because of VHDL's rules for passing signals through to procedures. For
example, it'd be most excellent if your test bench could call a
DoPCIBurstWrite() procedure inside your pcimaster entity with no
parameters other than say, burst length, and have it go off and do it.
But you can't do that -- you can't call a procedure in another entity
without passing every single signal as a parameter, and if you have a
bunch of signals, things get pretty ugly pretty quickly.

However, I can always call procedures declared in the
same process and have full access to all signals in
the testbench architecture. For packaged procedures with signal
parameters, I can overload the procedure id in the
main testbench process and enter the signal signal
identifiers only once in the overload declaration.

So the "client/server" approach basically has your test bench process
toggling a signal (like, say, doPCIBurstWrite) that is detected by
server process in the server's entity, and then the server process
calls the procedure(s) that actually toggle the signals that need
togglin'.
It's a right royal Pain In The Ass.

I agree.

-- Mike Treseler

 

[Mike Treseler]

Mike: I really do buy into the single process style for my
synthesizable code. This style was quite a revelation to me when I
discovered it here a month or so ago. The only disadvantage I see is
that often I have FSM's with outputs that don't need to be registered
so perhaps some wasted flops with this method.

If I am using an FPGA, the gates come with flops.
I can either bypass them with spaghetti code
or just let them be with the template. Note
that a variable can represent a combinational net
inside the process. The mandatory register only
applies to port outputs. If I bypass registers
on port outputs, static timing of multi-module
designs is more difficult to close and debug.

For
now testbenches are a different story, I'm still coming to grips with
the style proposed in that UART example. Probably just need more time
at it and I will get it.

Start by running the simulation and looking at the waveforms.
The main point is that the executive process delays are synchronized
by the "tic" procedure. This makes it easy to model complex stimulus
cycles. The trick is to run stimulus right up to the tick
where a known signal value is expected, and check it before
another tic; occurs.

-- Mike Treseler

 

[Klaus Falser]

Hi Newsgroup,
I am implementing a transaction based testbench and I'm trying to find
an effective way of encapsulating the client calls into my test
harness.

I have a "Test Harness" that is a single VHDL file and contains Bus
Functional Models(BFMs) for all the DUT interfaces. This test harness
also instantiates the DUT and generates the clk/reset for the DUT.
These BFM's can be thought of as servers - they provide a high level
transaction based interface.

Following Janick Bergeron's advice(Writing Testbenches - Functional
Verification of HDL Models, Kluwer Academic Publishers) I have created
a Test Harness Package in a separate file. This contains the BFM
transaction interface definition. Each BFM has a signal record(global)
for sending transactions to the BFM and another signal record(global)
for receiving transactions back from the BFM.

I then have a "Test Case" module(its another file) that acts as a
client to the Server BFM's. The client-server protocol is based around
transactions(no shit). The server BFM implements the server end of the
protocol obviously. Now, where does the client end of the protocol get
implemented? Bergeron suggests encapsulating the client end of the
protocol using procedures located in the test harness package(pg 243).
The problem is that the client server protocol is transaction based -
the server waits on a transaction( wait on ToServer'transaction) and
the client waits on an acknowledge/complete transaction from the
server. To quote the LRM "Attributes of an actual are never passed into
a subprogram". So I can't use the wait on FromServer'transaction
contruct inside my client procedure. Is there an elegent solution to
this? Why is Bergeron suggesting something that is not possible (or is
my understanding wrong somewhere)?

One solution is to use events rather than transactions. i.e. toggle a
signal rather than simply assign to it. But I find the wait on
blah'transaction much more elegant.

Regards
Andrew

Just to be curious.
Anybody knows why it's not allowed to use attributes like 'delayed or
'transaction in procedures?
The only reason I can see is that the simulator would be more
complicated since its only known at elaboration for
which signals the implicit signals behind this attributes have to be
constructed.

Regards
Klaus

 

[john Doef]

Klaus Falser a écrit :
[...]

Just to be curious.
Anybody knows why it's not allowed to use attributes like 'delayed or
'transaction in procedures?
The only reason I can see is that the simulator would be more
complicated since its only known at elaboration for
which signals the implicit signals behind this attributes have to be
constructed.

This won't be known at initial elaboration. This occurs also during
dynamic elaboration. So, the simulator has to know the past of all
signals!

JD.

 

[Jim Lewis]

Andrew,
I use a similar approach to what Janick and Ben Cohen do
with the exception that I pass records through interfaces
as inout. To make this work well, I make all record
elements based on std_logic/std_logic_vector/unsigned/signed.
While this is clumsy for some things (such as time),
it can be made to work. Then I initialize all fields to
tristate to resolve multiple drivers introduced by the
records being inout of multiple models. The VHDL-2006
effort is working on a feature called interfaces which
should make usage of any type possible.

For more details on this approach see the 2nd half of the
paper: "Accelerating Verification Through Pre-Use of
System-Level Testbench Components" that I gave at
DesignCon 2003.

I implement the transaction source in a single model I
call the test control (aka client). For each independent
interface, I create a separate process. In the process for
a given interface I make calls to the transaction procedures.
Some models require more than one transaction source to
support their interface behavior - for example a processor
model would require a process for functional transactions
and a separate process for the interrupt handler.

Handshaking between the test control (client) and the
BFM (server) can be anything you like. I use something
more like hardware handshaking just because of the
familiarity of it. The handshaking required may depend
on how the models need to work. Between some models a
level sensitive handshaking may be appropriate - for example
if the test control can issue a transaction to the BFM when
the BFM is busy, the BFM will miss the event on the signal.
Between some models edge sensitive handshaking may be
required - for example if you have a BFM that is a UART
receiver and the BFM does not buffer data, then only a
simple event from the BFM to the test control is required.
If the test control does not accept it, then the received
value is discarded. I also use a modified toggle handshaking
for the back door transaction access I added to my memory
models.

There are many ways to make handshaking work. I have found
a couple that work under different situations and have
encapsulated them into a procedure and don't bother to
think about them too much.

Cheers,
Jim
P.S.
If you use global signals for the transaction records,
then you might find creating arrays of them will help.
Then you assign each separate model an integer index into
the array by using a generic into the BFM.

I suspect that Ben Cohen has some examples of this in
his books.
--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Jim Lewis
Director of Training mailto:[email protected]
SynthWorks Design Inc. http://www.SynthWorks.com
1-503-590-4787

Expert VHDL Training for Hardware Design and Verification
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

 

[Andrew FPGA]

Hi Jim,
Thanks for your comments. My comments inline.

Andrew,
I use a similar approach to what Janick and Ben Cohen do
with the exception that I pass records through interfaces
as inout. To make this work well, I make all record
elements based on std_logic/std_logic_vector/unsigned/signed.
While this is clumsy for some things (such as time),
it can be made to work. Then I initialize all fields to
tristate to resolve multiple drivers introduced by the
records being inout of multiple models. The VHDL-2006
effort is working on a feature called interfaces which
should make usage of any type possible.

Janick Bergerons "Writing Testbenches, functional verification of HDL
models, 1st edition" recommends the use of separate in and out records
so that one does not need to write a resolution function. That had
scared me off from attempting it.

For more details on this approach see the 2nd half of the
paper: "Accelerating Verification Through Pre-Use of
System-Level Testbench Components" that I gave at
DesignCon 2003.

Thanks, actually I had already read this paper during my journey to the
"transaction based testbench" mecca. I found it well written and I
liked the clear diagrams. It did leave me with a few questions though:
1) Figure 12 shows each of the tests being a different implementation
of the TestCtl architecture. Sounds nice but practically how does one
string all these different tests together so that I can run all the
tests(sequentially) through the simulator in one hit. Do I need to
write scripts to compile Test1, load into sim, run sim, compile Test2,
load into sim, run sim?
2) Within TestCtl "each independent source of stimulous is supported by
one or more processes". The problem I see with this is that the
stimulous sources are never really independent. From the point of view
of someone writing the testcase they often want to specify the
relationship between stimulus starting on various interfaces.

I implement the transaction source in a single model I
call the test control (aka client). For each independent
interface, I create a separate process. In the process for
a given interface I make calls to the transaction procedures.
Some models require more than one transaction source to
support their interface behavior - for example a processor
model would require a process for functional transactions
and a separate process for the interrupt handler.

Hmm, not quite sure why a separate process for each independent
interface?

Handshaking between the test control (client) and the
BFM (server) can be anything you like. I use something
more like hardware handshaking just because of the
familiarity of it. The handshaking required may depend
on how the models need to work. Between some models a
level sensitive handshaking may be appropriate - for example
if the test control can issue a transaction to the BFM when
the BFM is busy, the BFM will miss the event on the signal.
Between some models edge sensitive handshaking may be
required - for example if you have a BFM that is a UART
receiver and the BFM does not buffer data, then only a
simple event from the BFM to the test control is required.
If the test control does not accept it, then the received
value is discarded. I also use a modified toggle handshaking
for the back door transaction access I added to my memory
models.

I think this is a very good point. (level sensitive vs edge sensitive
client/server protocol).

There are many ways to make handshaking work. I have found
a couple that work under different situations and have
encapsulated them into a procedure and don't bother to
think about them too much.

Cheers,
Jim
P.S.
If you use global signals for the transaction records,
then you might find creating arrays of them will help.
Then you assign each separate model an integer index into
the array by using a generic into the BFM.

I do use global signals for the transaction records. But I'm not quite
sure I see how your suggestion helps...Each server BFM already has
visibility of the global transaction records. The global transaction
records are stored in a package...Perhaps it is time for me to purchase
a more up to date book on verification(VHDL).

Thanks for your points. REgards Andrew.

 

[Mike Treseler]

...Perhaps it is time for me to purchase
a more up to date book on verification(VHDL).

You might have to write it first

-- Mike Treseler

 

[Jim Lewis]

Andrew,

Thanks, actually I had already read this paper during my journey to the
"transaction based testbench" mecca. I found it well written and I
liked the clear diagrams. It did leave me with a few questions though:
1) Figure 12 shows each of the tests being a different implementation
of the TestCtl architecture. Sounds nice but practically how does one
string all these different tests together so that I can run all the
tests(sequentially) through the simulator in one hit. Do I need to
write scripts to compile Test1, load into sim, run sim, compile Test2,
load into sim, run sim?

Yes. I have a script run each architecture independently.
Generally I compile them all and then use a configuration
to specify which is run. I put the configuration in the
bottom of TestCtrl even though it technically doesn't belong
there because it configures the top level TB, however, it
works well. I tend to run several simulations rather than
one mondo-long simulation.

2) Within TestCtl "each independent source of stimulous is supported by
one or more processes". The problem I see with this is that the
stimulous sources are never really independent. From the point of view
of someone writing the testcase they often want to specify the
relationship between stimulus starting on various interfaces.

I like to keep interfaces independent and synchronize when
needed to achieve the relationships I need. I have a
procedure just to handle synchronization between processes.

If you run all transaction procedures from one process, your
transactions must be non-blocking and operations like read will
take two interactions with the BFM - one read request and a
follow up to get the value. This complicates the test process.
It also means that when writing the test
you must have knowledge of the number of cycles an operation
takes. If a change is made, you may need to reorder your
transctions. Keep in mind that a design team may not tell a
test team about a change that does not change the big picture
functionality - like increasing the number of states a memory
access takes.

So I have found having separate processes per interface
simplifies how I think of the interactions with the models
and keeps me out of trivial design change issues.
Also with separate processes a BFM only needs a queue when
it needs to be able to execute the transactions you specify
for a test out of order.

Hmm, not quite sure why a separate process for each independent
interface?

For the interrupt handler if for all tests you know apriori
when each interrupt will happen, then you probably don't
need a separate process. This is probably true for all
simple tests. However for the system level testing, often
interrupts from the timers are handled and discarded and do
not impact the flow of the functional transactions. Even
this you could work around, by turning off timer interrupts.

However, again I use the separate processes as it is easier
for me to visualize them running independently and hence it
is easier for me to generate the code.

I think this is a very good point. (level sensitive vs edge sensitive
client/server protocol).

Going further for each class of protocols we need a
standard package that has subprograms that encapsulates
the implementation.

I do use global signals for the transaction records. But I'm not quite
sure I see how your suggestion helps...Each server BFM already has
visibility of the global transaction records. The global transaction
records are stored in a package...Perhaps it is time for me to purchase
a more up to date book on verification(VHDL).

Oops I did not put in all of the details of my stream of
conscious thought ... When I read the thread, it seemed
that someone was concerned
that they need to write a transaction support procedure for
each model - having done this for a while I thought yes this
is quite normal - unless you are writing separate procedures to
handle the case where you need to use a given model more
than once. For example, a print server may support between
1 to 4 printers. Hence, the simulation must be able to
support multiple printer BFMs instantiated. Under this
situation, an easy solution (when using global signals)
is to make the transaction record for the BFM that is used
more than once an array.

If you only have one instance of each type of BFM, then
this is not needed.

Cheers,
Jim

--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Jim Lewis
Director of Training mailto:[email protected]
SynthWorks Design Inc. http://www.SynthWorks.com
1-503-590-4787

Expert VHDL Training for Hardware Design and Verification
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~