Regression script

[jaspalbhullar]

Hello,

I need regression scripts for verification purposes, i.e. to run suit
of testcases. Preferably to launch as many testcases simultaneously as
per available licences at that time. That is, I don't want to run each
testcase serially one after the other, this will take too long.

It could be in Perl, tcl or c-shell (or for that matter in any other
language which can be run on Linux)

Thanks,
Jaspal

PS: If a suitable online resource is available, then please point it
to me. Thanks.

 

[HT-Lab]

Hello,

I need regression scripts for verification purposes, i.e. to run suit
of testcases. Preferably to launch as many testcases simultaneously as
per available licences at that time. That is, I don't want to run each
testcase serially one after the other, this will take too long.

It could be in Perl, tcl or c-shell (or for that matter in any other
language which can be run on Linux)

Thanks,
Jaspal

PS: If a suitable online resource is available, then please point it
to me. Thanks.

If you have access to Modelsim SE/Questa (and I assume other simulators have
their own equivalent) check out their JobSpy/LSF/SunGrid environment.

Hans
www.ht-lab.com

 

[Mike Treseler]

I need regression scripts for verification purposes, i.e. to run suit
of testcases. Preferably to launch as many testcases simultaneously as
per available licences at that time. That is, I don't want to run each
testcase serially one after the other, this will take too long.
It could be in Perl, tcl or c-shell (or for that matter in any other
language which can be run on Linux)

For simulation scripting I like sh style scripts using bash
and/or python (which is very good at math)

Why not csh?
http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/

In sh, I declare functions in one script - - -
....
run_tb()
{
cd $test_dir_v
pwd
make test_$entity_v
echo running short sim
source tb.sh
return 0
} ...

- - - then source the functions and run
the controlling script from another.
I collect the return code after vsim
exits, like this:
somecommand
fail="Error $?"
if [ "$fail" != "Error 0" ]
then
...

You can run parallel vsim sessions
by putting an '&' at the end of the
vsim command, but this ties up a
license per process and may or may
not save any time. It also complicates
the collection of return values.
I might consider using multiple servers.

Good luck.

-- Mike Treseler

 

[jaspalbhullar]

Thanks for the replies, I am not familiar with bash or python however
from the code it appears that the first function needs to be called
for every test case that is to be run, with 'tb.sh' containing the
vsim command and flags / options etc. So I must also have another
script, which defines the working directory for each of the testcase
and one more to check the results from each one.
However what if a licence is not currently available, will the request
for it will automatically get queued or rest of the testcases won't
run at all?
Yes I meant launching the jobs on different machines / servers.

I did a brief search on mentor's website and found about JobSpy, it
doesn't seem be a regression launcher, rather a tool to spy about you
jobs in a batch mode. Though it sounds useful, as it can tell you
about the status of your regression run.

Jaspal

 

[Mike Treseler]

it appears that the first function needs to be called
for every test case that is to be run, with 'tb.sh' containing the
vsim command and flags / options etc. So I must also have another
script, which defines the working directory for each of the testcase
and one more to check the results from each one.

I put all the functions and constants into one file.
There are lots of them.
After I say
source my_functions.sh

everything is visible to the shell,
and I can run the executable main script.

However what if a license is not currently available, will the request
for it will automatically get queued or rest of the testcases won't
run at all?

The license server queues requests, but
your script would have to handle the return codes
and timeouts somehow.

-- Mike Treseler

 

[HT-Lab]

Thanks for the replies, I am not familiar with bash or python however
from the code it appears that the first function needs to be called
for every test case that is to be run, with 'tb.sh' containing the
vsim command and flags / options etc. So I must also have another
script, which defines the working directory for each of the testcase
and one more to check the results from each one.
However what if a licence is not currently available, will the request
for it will automatically get queued or rest of the testcases won't
run at all?

As Mike said, yes, for Modelsim this is the default behaviour (disable with
the -lic_noqueue). If you want to talk to Flexlm in your script then simple
invoke "lmstat -f" (on windows lmutil lmstat -f) and parse the resulting
output.

Yes I meant launching the jobs on different machines / servers.

I did a brief search on mentor's website and found about JobSpy, it
doesn't seem be a regression launcher, rather a tool to spy about you
jobs in a batch mode. Though it sounds useful, as it can tell you
about the status of your regression run.

This appsnote might answer a few questions;

http://www.model.com/products/pdf/appnotes/GridWare.pdf

I would also look at Checkpoint&Restore, creating a seperate elaboration
file and using the -filemap_elab argument,

Hans
www.ht-lab.com