Automating VHDL Simulations in ModelSim

[Georg]

Hello everybody,
I'm still pretty new to VHDL and don't really know how to tackle the
following problem:

I want to do a whole bunch of simulations of some VHDL code I've
written, where each simulation run varies in the values of some
constants defined in the testbench. But I don't want to manually change
the values, recompile and start the whole thing over an over again.

Since I don't see a way how to do this in VHDL itself, I tried to write
a TCL script for the simulator (ModelSim). However, ModelSim won't let
the script alter the constants during the simulation, no matter whether
they are defined as 'constant' or 'generic'. I can't define them as
'variables', because the testbench is a structural description and
doesn't contain processes.

Does anyone have another idea? Is this doable with pure VHDL means?
Thanks a lot for any help,
Georg

 

[HT-Lab]

"Georg" <pontifex@sbox.tugraz.at> wrote in message news:gm7s9s$sc7$1@newsreader2.utanet.at...

Hello everybody,
I'm still pretty new to VHDL and don't really know how to tackle the
following problem:

I want to do a whole bunch of simulations of some VHDL code I've
written, where each simulation run varies in the values of some
constants defined in the testbench. But I don't want to manually change
the values, recompile and start the whole thing over an over again.

Why don't you put your constant values into an array and use a for loop to step through the different values?

Hans
www.ht-lab.com

Since I don't see a way how to do this in VHDL itself, I tried to write
a TCL script for the simulator (ModelSim). However, ModelSim won't let
the script alter the constants during the simulation, no matter whether
they are defined as 'constant' or 'generic'. I can't define them as
'variables', because the testbench is a structural description and
doesn't contain processes.

Does anyone have another idea? Is this doable with pure VHDL means?
Thanks a lot for any help,
Georg

 

[Jim Lewis]

Georg
In the TCL script, I run for at least 2 times my longest run
time (to allow for variations). Then I stop the last running
process with either:

report "test done" severity failure ; -- any version

or with VHDL-2008 and is currently implemented in recent versions
of ModelSim:
std.env.stop(0) ; -- VHDL-2008

Cheers,
Jim

--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Jim Lewis SynthWorks VHDL Training http://www.synthworks.com

A bird in the hand may be worth two in the bush,
but it sure makes it hard to type.

 

[Georg]

HT-Lab schrieb:

"Georg" <pontifex@sbox.tugraz.at <mailtoontifex@sbox.tugraz.at>> wrote
in message news:gm7s9s$sc7$1@newsreader2.utanet.at...
Hello everybody,
I'm still pretty new to VHDL and don't really know how to tackle the
following problem:

I want to do a whole bunch of simulations of some VHDL code I've
written, where each simulation run varies in the values of some
constants defined in the testbench. But I don't want to manually change
the values, recompile and start the whole thing over an over again.

Why don't you put your constant values into an array and use a for loop
to step through the different values?

Hm, isn't a for loop a sequential statement and only allowed inside
processes? As said, I have a structural description of the testbench
without processes.
Or what exactly do you mean? Bear with me, I'm a newbie...
Thanks, Georg

 

[Mike Treseler]

Georg wrote:

Hm, isn't a for loop a sequential statement and only allowed inside
processes?

Yes. Like this:

...
constant reps : natural := 8;
begin -- process main: Top level loop invokes top procedures.
init;
for i in 1 to reps loop
timed_cycle;
end loop;
for i in 1 to reps loop
handshake_cycle;
end loop;
coda;
end process main; -- that's it

-- from http://mysite.verizon.net/miketreseler/test_uart.vhd

As said, I have a structural description of the testbench
without processes.

That doesn't sound like a testbench to me.

-- Mike Treseler

 

[Jeff Cunningham]

Georg wrote:

HT-Lab schrieb:

Why don't you put your constant values into an array and use a for
loop to step through the different values?

Hm, isn't a for loop a sequential statement and only allowed inside
processes? As said, I have a structural description of the testbench
without processes.
Or what exactly do you mean? Bear with me, I'm a newbie...
Thanks, Georg

Is there a reason you are avoiding having a process inside a testbench?
A purely structural testbench seems like not much of a testbench. Where
is the test stimulus coming from? Some component contained in the
structural testbench? Simulator force commands? If the latter, you'd be
better off creating the force statement functionality using high level
behavioral code in the testbench. Doing so is flexible and powerful
since you don't have to stick to RTL synthesizable coding styles. For
instance, here are some trivial examples:

process
begin
wait for CLOCK_PERIOD/2.0;
clock <= not clock;
end process;

process
begin
some_signal <= '1';
wait for 65 ns;
some_signal <= '0';
wait for 10 ns;
some_signal <= 'X';
...

 

[HT-Lab]

"Georg" <pontifex@sbox.tugraz.at> wrote in message
news:gm7v0p$4ud$1@newsreader2.utanet.at...

HT-Lab schrieb:
"Georg" <pontifex@sbox.tugraz.at <mailtoontifex@sbox.tugraz.at>> wrote
in message news:gm7s9s$sc7$1@newsreader2.utanet.at...
Hello everybody,
I'm still pretty new to VHDL and don't really know how to tackle the
following problem:

I want to do a whole bunch of simulations of some VHDL code I've
written, where each simulation run varies in the values of some
constants defined in the testbench. But I don't want to manually
change
the values, recompile and start the whole thing over an over again.
Why don't you put your constant values into an array and use a for loop
to step through the different values?

Hm, isn't a for loop a sequential statement and only allowed inside
processes? As said, I have a structural description of the testbench
without processes.
Or what exactly do you mean? Bear with me, I'm a newbie...
Thanks, Georg

Yes, you are right, my mistake. However, I would follow the advice already
given and use a process. If you just want to have a quick test then change
your constants into a signal and use the Tcl force command to alter its
value.

Good luck,

Hans
www.ht-lab.com

 

[Jonathan Bromley]

On Mon, 02 Feb 2009 23:33:28 +0100, Georg <pontifex@sbox.tugraz.at>
wrote:

Hello everybody,
I'm still pretty new to VHDL and don't really know how to tackle the
following problem:

I want to do a whole bunch of simulations of some VHDL code I've
written, where each simulation run varies in the values of some
constants defined in the testbench. But I don't want to manually change
the values, recompile and start the whole thing over an over again.

Since I don't see a way how to do this in VHDL itself, I tried to write
a TCL script for the simulator (ModelSim). However, ModelSim won't let
the script alter the constants during the simulation, no matter whether
they are defined as 'constant' or 'generic'.

Yes it can!

When you load the simulation with the [vsim] command, you can use
the -g or -G options to patch-up the value of generics. In practice
the likely scenario is that you have a bunch of generics on the
(portless) top-level testbench, and you simply patch those.

I agree with others that it would be even nicer to use procedural
code to do it, but if you already have your TB it seems silly to
rewrite it.

One final suggestion: Instead of tweaking generics from the sim
command line, you could write your top level TB with generics...

entity TB is
generic (bus_width: positive := 8; num_ports: positive := 2);
end;
...

And then instantiate it in a configuration or (easier syntax)
another wrapper entity:

entity Simulation_Top is end;
architecture T of Simulation_Top is begin
The_TB: entity work.TB
generic map (bus_width => 16, num_ports => 4);
end;

This wrapper is so small and simple that you could easily create
a Tcl script that would rewrite the code, recompile the file, and
re-run the sim. That also has the advantage of being completely
simulator-independent.
--
Jonathan Bromley, Consultant

DOULOS - Developing Design Know-how
VHDL * Verilog * SystemC * e * Perl * Tcl/Tk * Project Services

Doulos Ltd., 22 Market Place, Ringwood, BH24 1AW, UK
jonathan.bromley@MYCOMPANY.com
http://www.MYCOMPANY.com

The contents of this message may contain personal views which
are not the views of Doulos Ltd., unless specifically stated.

 

[Georg]

Jonathan Bromley wrote:

On Mon, 02 Feb 2009 23:33:28 +0100, Georg <pontifex@sbox.tugraz.at
wrote:

Since I don't see a way how to do this in VHDL itself, I tried to write
a TCL script for the simulator (ModelSim). However, ModelSim won't let
the script alter the constants during the simulation, no matter whether
they are defined as 'constant' or 'generic'.

Yes it can!

When you load the simulation with the [vsim] command, you can use
the -g or -G options to patch-up the value of generics. In practice
the likely scenario is that you have a bunch of generics on the
(portless) top-level testbench, and you simply patch those.

Perfect! This is what I've been looking for! Thank you very much.

I agree with others that it would be even nicer to use procedural
code to do it, but if you already have your TB it seems silly to
rewrite it.

I'll consider that too, but for the moment I'm happy
Greetings, Georg

 

[Georg]

Mike Treseler wrote:

Georg wrote:

As said, I have a structural description of the testbench
without processes.

That doesn't sound like a testbench to me.

Well, I have a couple of 'devices under test', which are just entities
and I instantiate them in the testbench. Like that:

------------- Minimal Example -------------
entity test_bench is
end entity test_bench;

architecture TB of test_bench is
constant my_const : real := 1.3; -- this changes in each simulation
signal foobar : std_ulogic := '0';
-- some more signals or constants...
begin

DUT_1 : entity work.my_device(bar)
generic map (
foo => my_const)
port map(...);

DUT_2 : entity work.my_device(bar)
generic map (
foo => my_const)
port map(...);

-- some more DUTs

end TB;
------------- /Minimal Example -------------


If my entire testbench is a process, how can I instantiate my DUTs then?
Greetings, Georg

 

[Georg]

Tricky wrote:

On 3 Feb, 11:27, Georg <ponti...@sbox.tugraz.at> wrote:
Mike Treseler wrote:
Georg wrote:
As said, I have a structural description of the testbench
without processes.
That doesn't sound like a testbench to me.
Well, I have a couple of 'devices under test', which are just entities
and I instantiate them in the testbench. Like that:

------------- Minimal Example -------------
entity test_bench is
end entity test_bench;

architecture TB of test_bench is
constant my_const : real := 1.3; -- this changes in each simulation
signal foobar : std_ulogic := '0';
-- some more signals or constants...
begin

DUT_1 : entity work.my_device(bar)
generic map (
foo => my_const)
port map(...);

DUT_2 : entity work.my_device(bar)
generic map (
foo => my_const)
port map(...);

-- some more DUTs

end TB;
------------- /Minimal Example -------------

If my entire testbench is a process, how can I instantiate my DUTs then?
Greetings, Georg

Your entire testbench isnt just 1 process - you can have multiple
processes generating the stimulus at the front end that will actually
make stuff happen, and sometimes at least one capturing the output and
dumping it to a log file (or other data file). Normally the process is
a Bus Functional Model (BFM) that generates the stimulus and/or reads
data in from a file. As great as simulators are, I dont know of any
that you could connect to real hardware - it just doesnt run in real
time. So you have to model the input.

You at least have to generate the clock in its own process.

True, I do that anyway - I just skipped it for this minimal example. In
fact I have two additional entities that generate clock signals and one
process that writes the DUT output signals into a log-file.

But to modify 'my_const', I would have to define it as a variable inside
a process and then I can't use it anymore as a generic input to my DUTs,
right?

Regards, Georg

 

[Tricky]

On 3 Feb, 11:27, Georg <ponti...@sbox.tugraz.at> wrote:

Mike Treseler wrote:
Georg wrote:
As said, I have a structural description of the testbench
without processes.

That doesn't sound like a testbench to me.

Well, I have a couple of 'devices under test', which are just entities
and I instantiate them in the testbench. Like that:

------------- Minimal Example -------------
entity test_bench is
end entity test_bench;

architecture TB of test_bench is
  constant my_const  : real := 1.3;  -- this changes in each simulation
  signal foobar      : std_ulogic := '0';
  -- some more signals or constants...
begin

  DUT_1 : entity work.my_device(bar)
  generic map (
    foo       => my_const)
  port map(...);

  DUT_2 : entity work.my_device(bar)
  generic map (
    foo       => my_const)
  port map(...);

  -- some more DUTs

end TB;
------------- /Minimal Example -------------

If my entire testbench is a process, how can I instantiate my DUTs then?
Greetings, Georg

Your entire testbench isnt just 1 process - you can have multiple
processes generating the stimulus at the front end that will actually
make stuff happen, and sometimes at least one capturing the output and
dumping it to a log file (or other data file). Normally the process is
a Bus Functional Model (BFM) that generates the stimulus and/or reads
data in from a file. As great as simulators are, I dont know of any
that you could connect to real hardware - it just doesnt run in real
time. So you have to model the input.

You at least have to generate the clock in its own process.

 

[Georg]

Tricky wrote:

Why not just use a generate loop to instantiate the DUTs and processes
X times (in parrallel), that dump out to X log files?

entity my_tb is
generic (
generic_list : int_array := (0, 1, 2, 3, 4);
);
end entity my_TB

...

test_gens : for i in generic_list'range generate

DUT : my_ent
generic map (
foo => generic_list(i);
);

process
file my_log : text open WRITE_MODE is
( "logfile" & integer'image(generic_list(i)) & ".log");
begin
....
end process;

end generate test_gens;

Ah, I see, that looks clever. Well, IMO that's still a structural
testbench (if I understood the meaning of 'structural' right), but it
should do what I need. I'll try it out later, right now I'm taking
advantage of Jonathan's hint.

Thanks to you and all others who have replied!
Georg

 

[Tricky]

On 3 Feb, 12:39, Georg <ponti...@sbox.tugraz.at> wrote:

Tricky wrote:
On 3 Feb, 11:27, Georg <ponti...@sbox.tugraz.at> wrote:
Mike Treseler wrote:
Georg wrote:
As said, I have a structural description of the testbench
without processes.
That doesn't sound like a testbench to me.
Well, I have a couple of 'devices under test', which are just entities
and I instantiate them in the testbench. Like that:

------------- Minimal Example -------------
entity test_bench is
end entity test_bench;

architecture TB of test_bench is
  constant my_const  : real := 1.3;  -- this changes in each simulation
  signal foobar      : std_ulogic := '0';
  -- some more signals or constants...
begin

  DUT_1 : entity work.my_device(bar)
  generic map (
    foo       => my_const)
  port map(...);

  DUT_2 : entity work.my_device(bar)
  generic map (
    foo       => my_const)
  port map(...);

  -- some more DUTs

end TB;
------------- /Minimal Example -------------

If my entire testbench is a process, how can I instantiate my DUTs then?
Greetings, Georg

Your entire testbench isnt just 1 process - you can have multiple
processes generating the stimulus at the front end that will actually
make stuff happen, and sometimes at least one capturing the output and
dumping it to a log file (or other data file). Normally the process is
a Bus Functional Model (BFM) that generates the stimulus and/or reads
data in from a file. As great as simulators are, I dont know of any
that you could connect to real hardware - it just doesnt run in real
time. So you have to model the input.

You at least have to generate the clock in its own process.

True, I do that anyway - I just skipped it for this minimal example. In
fact I have two additional entities that generate clock signals and one
process that writes the DUT output signals into a log-file.

But to modify 'my_const', I would have to define it as a variable inside
a process and then I can't use it anymore as a generic input to my DUTs,
right?

Regards, Georg

Why not just use a generate loop to instantiate the DUTs and processes
X times (in parrallel), that dump out to X log files?

entity my_tb is
generic (
generic_list : int_array := (0, 1, 2, 3, 4);
);
end entity my_TB

....

test_gens : for i in generic_list'range generate

DUT : my_ent
generic map (
foo => generic_list(i);
);

process
file my_log : text open WRITE_MODE is
( "logfile" & integer'image(generic_list(i)) & ".log");
begin
....
end process;

end generate test_gens;

 

[Jan Decaluwe]

petrovski101@gmail.com wrote:

I wrote a tcl/tk GUI that parses the top level entity and outputs a
generic vhdl testbench. The testbench creates a clock to read data
from a synchronous textio process. A counter is loaded with the most
significant 16 bits of this array of data. This counter determines
how long to sit at each test vector. The remaining bits of the array
are stimulus.

The top level code is usually a collection of bus functional models,
address mapped to my textio stimulus, and the unit under test.

A seperate TCL program generates the test vectors and outputs expected
results. The testbench outputs it's results to a file and a bash
script compares the two. You can generate constrained random test
vectors this way in the tcl script as part of regression test.

Although textio is slow to simulate, I find it's generally fast enough
for my needs.

Pete

Can't resist, sorry

http://www.myhdl.org/doku.php/why#you_use_scripting_languages_intensively_in_the_design_flow

--
Jan Decaluwe - Resources bvba - http://www.jandecaluwe.com
Python as a hardware description language:
http://www.myhdl.org

 

I wrote a tcl/tk GUI that parses the top level entity and outputs a
generic vhdl testbench. The testbench creates a clock to read data
from a synchronous textio process. A counter is loaded with the most
significant 16 bits of this array of data. This counter determines
how long to sit at each test vector. The remaining bits of the array
are stimulus.

The top level code is usually a collection of bus functional models,
address mapped to my textio stimulus, and the unit under test.

A seperate TCL program generates the test vectors and outputs expected
results. The testbench outputs it's results to a file and a bash
script compares the two. You can generate constrained random test
vectors this way in the tcl script as part of regression test.

Although textio is slow to simulate, I find it's generally fast enough
for my needs.

Pete

 

On Feb 3, 10:08 am, Jan Decaluwe <j...@jandecaluwe.com> wrote:

petrovski...@gmail.com wrote:
I wrote a tcl/tk GUI that parses the top level entity and outputs a
generic vhdl testbench.  The testbench creates a clock to read data
from a synchronous textio process.  A counter is loaded with the most
significant 16 bits of this array of data.  This counter determines
how long to sit at each test vector.  The remaining bits of the array
are stimulus.

The top level code is usually a collection of bus functional models,
address mapped to my textio stimulus, and the unit under test.

A seperate TCL program generates the test vectors and outputs expected
results.  The testbench outputs it's results to a file and a bash
script compares the two.  You can generate constrained random test
vectors this way in the tcl script as part of regression test.

Although textio is slow to simulate, I find it's generally fast enough
for my needs.

Pete

Can't resist, sorry

http://www.myhdl.org/doku.php/why#you_use_scripting_languages_intensi...

--
Jan Decaluwe - Resources bvba -http://www.jandecaluwe.com
     Python as a hardware description language:
     http://www.myhdl.org- Hide quoted text -

- Show quoted text -

That's quite interesting. I considered something similar in the past
but decided not to take the plunge. Instead, I wanted a generic
interface that could be driven by programs other than tcl. My
testbenches are written in tcl but the output vectors are just a
string of ascii 1's and 0's. You could just as easily create testbench
stimulus with C, Perl, Python... whatever.

I'm also leaning less towards generic language and more towards
generic templates. I have templates to generate register files and
state machines. I like to describe the more common elements of a
design in as few lines of code as possible (because I'm lazy). The
templates also generate groff code for documentation. Register file table, State Machine = state diagram + equations. Change the template
and both vhdl and groff code gets updated. These get imported into my
chip documentation automatically. Docs are created as you go rather
than at the end of the project, when you're forgotten half the
details.

I do like your simulator. That's slick.