Hardware to test (FPGA-based) prototype?

[Alex Rast]

I have an FPGA-based prototype for a PCI product in development. Since the
intended product application will involve very high speed data transfers,
we have designed all the I/O and internal busses in the FPGA to work on
synchronous protocols. Now, however, I'm running into a real stumbling
block.

The problem is in testing the board. What I need to do is to be able to
generate some test data on a PC, and send it to the FPGA, simulating data
flow through the system under test. Similarly, the PC needs to be able to
receive test data from the FPGA. Because of the design of the board, I need
to use a synchronous, hardware-based protocol and interface to transfer the
data.

However, there doesn't seem to be much hardware out there that will enable
me to do this, at least not at reasonable cost. All I need to do is dump
bitstreams in either direction, synchronously, but I have met with little
success. We made an abortive effort to use LabView together with their DIO-
32HS, which seemed promising and (supposedly) offered a high-speed
synchronous protocol, but when we tried to use it, the protocol didn't
work, we couldn't make it work, and apparently nobody at National
Instruments had tried using that protocol and gotten it to work. Indeed, I
saw others posting on the Labview NG, running into the same problems! So
that's not an option.

So, what would be the easiest way to create a test interface that lets us
transfer data using a synchronous protocol at reasonable speeds (at least
10 MHz) between a PC and a device under test? Our prototype board has no
shortage of high-speed, MICTOR connectors that we can use to interface to.
We're willing to spend some dollars to do it, but if it starts escalating
into the thousands of dollars just to get a simple test interface, I think
that such a price is disproportionate relative to what we need to achieve.
So I think a fair budget limit is around $2000. Any suggestions?

--
Alex Rast
ad.rast.7@nwnotlink.NOSPAM.com
(remove d., .7, not, and .NOSPAM to reply)

 

[Hal Murray]

So, what would be the easiest way to create a test interface that lets us
transfer data using a synchronous protocol at reasonable speeds (at least
10 MHz) between a PC and a device under test? Our prototype board has no
shortage of high-speed, MICTOR connectors that we can use to interface to.
We're willing to spend some dollars to do it, but if it starts escalating
into the thousands of dollars just to get a simple test interface, I think
that such a price is disproportionate relative to what we need to achieve.
So I think a fair budget limit is around $2000. Any suggestions?

Can you cross connect two of your boards? Possibly running
totally different test code in the helper board.

If your board isn't quite right, you might browse through
the prototype/demo board list and see if you can find one
that looks good. I'm thinking of PCI, FPGA, Memory, and some
connector you can get to. Load the memory from PCI, turn the
FPGA loose to read the data out of memory and feed it to your
system. That gives you a burst for as long as the memory
has room for your pattern.

--
The suespammers.org mail server is located in California. So are all my
other mailboxes. Please do not send unsolicited bulk e-mail or unsolicited
commercial e-mail to my suespammers.org address or any of my other addresses.
These are my opinions, not necessarily my employer's. I hate spam.

 

[Hans-Bernhard Broeker]

In comp.arch.embedded Alex Rast <ad.rast.7@nwnotlink.nospam.com> wrote:

[...]

Because of the design of the board, I need
to use a synchronous, hardware-based protocol and interface to transfer the
data.

"A ... protocol" meaning exactly _what_? I'm quite sure you don't
just need some random protocol. You need a piece of hardware and
software that generates and read _exactly_ that protocol your device
under test uses. Regarding which you completely forgot to tell your
readers what that protocol actually is. From what you wrote, it might
well be some completely non-standard homegrown thing. If so, you're
obviously on your own --- oughtta have thought of that aspect, before
you designed that protocol into your hardware. Now you're stuck with
what you have.

Odds are you'll have to build your test harness yourself, in this
case. Software alone won't do (not at 10Mhz on an ordinary PC...),
and a test pulse generator inside the budget limit you gave probably
won't be flexible enough for your needs.

do is dump bitstreams in either direction, synchronously,

.... synchronous to what?

What you would need is essentially the opposite thing to a logic
analyzer or storage scope: a fully programmable logic signal
generator. Where a LA or storage scope has programmable triggers and
logs its results to an internal buffer for later retrieval, you need a
device that has programmable output triggering to output signals
stored in an internal buffer.

To build one yourself, you'll probably need an FPGA board about equally
complicated as the one you're testing.

--
Hans-Bernhard Broeker (broeker@physik.rwth-aachen.de)
Even if all the snow were burnt, ashes would remain.

 

[Nial Stewart]

So, what would be the easiest way to create a test interface that lets us
transfer data using a synchronous protocol at reasonable speeds (at least
10 MHz) between a PC and a device under test? Our prototype board has no
shortage of high-speed, MICTOR connectors that we can use to interface to.
We're willing to spend some dollars to do it, but if it starts escalating
into the thousands of dollars just to get a simple test interface, I think
that such a price is disproportionate relative to what we need to achieve.
So I think a fair budget limit is around $2000. Any suggestions?

Alex,

It sounds like you'd be best using another of your boards as a test
source as Hal suggested.

If you want to use completely independant hardware to verify the
operation of your board you could try using one of my
Cyclone based PCI evaluation boards. See the Hardware
page of my web site below.


Nial Stewart

------------------------------------------------
Nial Stewart Developments Ltd
FPGA and High Speed Digital Design
Cyclone based 'Easy PCI' eval board
www.nialstewartdevelopments.co.uk

 

[CBFalconer]

Hans-Bernhard Broeker wrote:

In comp.arch.embedded Alex Rast <ad.rast.7@nwnotlink.nospam.com> wrote:

[...]

Because of the design of the board, I need
to use a synchronous, hardware-based protocol and interface to transfer the
data.

"A ... protocol" meaning exactly _what_? I'm quite sure you don't
just need some random protocol. You need a piece of hardware and
software that generates and read _exactly_ that protocol your device
under test uses. Regarding which you completely forgot to tell your
readers what that protocol actually is. From what you wrote, it might
well be some completely non-standard homegrown thing. If so, you're
obviously on your own --- oughtta have thought of that aspect, before
you designed that protocol into your hardware. Now you're stuck with
what you have.

Odds are you'll have to build your test harness yourself, in this
case. Software alone won't do (not at 10Mhz on an ordinary PC...),
and a test pulse generator inside the budget limit you gave probably
won't be flexible enough for your needs.

do is dump bitstreams in either direction, synchronously,

... synchronous to what?

Moral: Don't design hardware without first thinking about testing
it. Or at least enough of it I learned that the hard way a
long time ago, by building a marvelous device that we could never
replicate. Each phase had been driven by a test jig, which was
replaced by the next part of the real device. The final had no
jigs, and no way to prod it.

However I don't think it is quite as bad as you say. Nearly, but
not quite. He will need some sort of hardware fifo, with dual
ports, and a way of loading it from a pc. He probably also needs
the synchronous interface module to unload the fifo, which may be
another of his systems.

He could start with something that continuously unloads one simple
pattern.

--
Chuck F (cbfalconer@yahoo.com) (cbfalconer@worldnet.att.net)
Available for consulting/temporary embedded and systems.
<http://cbfalconer.home.att.net> USE worldnet address!

 

[Rene Tschaggelar]

Alex Rast wrote:

I have an FPGA-based prototype for a PCI product in development. Since the
intended product application will involve very high speed data transfers,
we have designed all the I/O and internal busses in the FPGA to work on
synchronous protocols. Now, however, I'm running into a real stumbling
block.

The problem is in testing the board. What I need to do is to be able to
generate some test data on a PC, and send it to the FPGA, simulating data
flow through the system under test. Similarly, the PC needs to be able to
receive test data from the FPGA. Because of the design of the board, I need
to use a synchronous, hardware-based protocol and interface to transfer the
data.

However, there doesn't seem to be much hardware out there that will enable
me to do this, at least not at reasonable cost. All I need to do is dump
bitstreams in either direction, synchronously, but I have met with little
success. We made an abortive effort to use LabView together with their DIO-
32HS, which seemed promising and (supposedly) offered a high-speed
synchronous protocol, but when we tried to use it, the protocol didn't
work, we couldn't make it work, and apparently nobody at National
Instruments had tried using that protocol and gotten it to work. Indeed, I
saw others posting on the Labview NG, running into the same problems! So
that's not an option.

So, what would be the easiest way to create a test interface that lets us
transfer data using a synchronous protocol at reasonable speeds (at least
10 MHz) between a PC and a device under test? Our prototype board has no
shortage of high-speed, MICTOR connectors that we can use to interface to.
We're willing to spend some dollars to do it, but if it starts escalating
into the thousands of dollars just to get a simple test interface, I think
that such a price is disproportionate relative to what we need to achieve.
So I think a fair budget limit is around $2000. Any suggestions?

I had a National Intsruments HS32 board too.
Even though it appears to offer half a dozend protocols, none worked for us.
The protocols worked, but we were unable to use them to our hardware.
The support of NI was (well they tried but to no avail) lacking.
Their forums aren't that supportive and very slow.

I'd suggest to use a second of your boards.
You don't need its PCI interface, just reload some other firmware
into the FPGA to let it act as pattern generator.
If you don't happen to have a second board, you may let us know
what FPGA with what interface you wish to use.
One may have a spare board.

Rene
--
Ing.Buero R.Tschaggelar - http://www.ibrtses.com
& commercial newsgroups - http://www.talkto.net

 

[Alex Rast]

at Fri, 16 Jan 2004 14:53:36 GMT in <4007EF4F.16A1C673@yahoo.com>,
cbfalconer@worldnet.att.net (CBFalconer) wrote :

Hans-Bernhard Broeker wrote:

In comp.arch.embedded Alex Rast <ad.rast.7@nwnotlink.nospam.com
wrote:

[...]

Because of the design of the board, I need
to use a synchronous, hardware-based protocol and interface to
transfer the data.

"A ... protocol" meaning exactly _what_? I'm quite sure you don't
just need some random protocol. You need a piece of hardware and
software that generates and read _exactly_ that protocol your device
under test uses.

Actually, we can pretty much indeed use any protocol, as long as it's
synchronous (i.e. uses a continuous clock signal to time transactions)
That's the nice thing about an FPGA - it's reconfigurable. I designed an
internal test port that on the one side has the interface to our bus inside
the FPGA, and on the other side has an interface to whatever test fixture
we decided to use. The external interface is interchangeable, so we can
devise it to suit a wide variety of possible protocols.

Moral: Don't design hardware without first thinking about testing
it. Or at least enough of it

We did think very long and hard about testing, and had several meetings
where we really examined the design carefully, with a view to testability.
But there's also the side of that your board needs to do what it needs to
do. There's not much good designing a highly testable board that doesn't
perform the task you're designing it for. It does seem to me that the
available testing options for high-speed, synchronous interfaces are very
few and far between. The option that people have been recommending, of
putting another identical board in our system to use as a test interface,
is one I thought about and I think, with the consensus being that this is
the best way to go, is what I'll do. Is this, then, the typical way people
test high-speed cards and interfaces? I'm quite surprised that there aren't
more testing/prototype systems available for these kinds of hardware, which
must surely be extremely common.

--
Alex Rast
ad.rast.7@nwnotlink.NOSPAM.com
(remove d., .7, not, and .NOSPAM to reply)

 

[Hal Murray]

We did think very long and hard about testing, and had several meetings
where we really examined the design carefully, with a view to testability.
But there's also the side of that your board needs to do what it needs to
do. There's not much good designing a highly testable board that doesn't
perform the task you're designing it for. It does seem to me that the
available testing options for high-speed, synchronous interfaces are very
few and far between. The option that people have been recommending, of
putting another identical board in our system to use as a test interface,
is one I thought about and I think, with the consensus being that this is
the best way to go, is what I'll do. Is this, then, the typical way people
test high-speed cards and interfaces? I'm quite surprised that there aren't
more testing/prototype systems available for these kinds of hardware, which
must surely be extremely common.

I don't know if "typical" is the right word, but it seems like one
of the obvious advantages of using an FPGA. (aka you can reprogram
it to do something else)

One important consideration for using your own board: You are already
familiar with the tools and all the details of the hardware.

--
The suespammers.org mail server is located in California. So are all my
other mailboxes. Please do not send unsolicited bulk e-mail or unsolicited
commercial e-mail to my suespammers.org address or any of my other addresses.
These are my opinions, not necessarily my employer's. I hate spam.

 

[Nico Coesel]

ad.rast.7@nwnotlink.NOSPAM.com (Alex Rast) wrote:

We did think very long and hard about testing, and had several meetings
where we really examined the design carefully, with a view to testability.
But there's also the side of that your board needs to do what it needs to
do. There's not much good designing a highly testable board that doesn't
perform the task you're designing it for. It does seem to me that the
available testing options for high-speed, synchronous interfaces are very
few and far between. The option that people have been recommending, of
putting another identical board in our system to use as a test interface,
is one I thought about and I think, with the consensus being that this is
the best way to go, is what I'll do. Is this, then, the typical way people
test high-speed cards and interfaces? I'm quite surprised that there aren't
more testing/prototype systems available for these kinds of hardware, which
must surely be extremely common.

You could try to obtain a used DAS9200 or TLA510 system with a pattern
generator board (these generate patterns based on a 50MHz clock).
These systems are for sale on Ebay every now and then.

--
Reply to nico@nctdevpuntnl (punt=.)
Bedrijven en winkels vindt U op www.adresboekje.nl

 

[Mike Treseler]

Alex Rast wrote:

It does seem to me that the
available testing options for high-speed, synchronous interfaces are very
few and far between. The option that people have been recommending, of
putting another identical board in our system to use as a test interface,
is one I thought about and I think, with the consensus being that this is
the best way to go, is what I'll do. Is this, then, the typical way people
test high-speed cards and interfaces? I'm quite surprised that there aren't
more testing/prototype systems available for these kinds of hardware, which
must surely be extremely common.

One reason may be that simulation is commonly used
for design verification of synchronous logic.


-- Mike Treseler

 

[john jakson]

You could try to obtain a used DAS9200 or TLA510 system with a pattern
generator board (these generate patterns based on a 50MHz clock).
These systems are for sale on Ebay every now and then.

Or you can rent these things from companies like ElectroRent IIRC or
GE or whatever. They are not that difficult to use, we used to have a
HP PG on the network to download files into ram and that would throw
the pattern at your HW. The matching HP LA would observe the results
and get it back onto the network.

The only issue I had was the memory buffers were very small relative
to PC memory and the bus widths were too narrow, kinda dinasours by
todays std.

An FPGA just calls out to be used as an PG+LA in a box to custom spec,
but thats another project. Or just use one of the ready made boards
withs lots of IO and onboard ram.

johnjakson_usa_com

 

[Alex Rast]

at Sun, 18 Jan 2004 19:58:02 GMT in <400AE54A.3050405@flukenetworks.com>,
mike.treseler@flukenetworks.com (Mike Treseler) wrote :

Alex Rast wrote:
...The option that people have been
recommending, of putting another identical board in our system to use
as a test interface... is what I'll do. Is
this, then, the typical way people test high-speed cards and
interfaces? I'm quite surprised that there aren't more
testing/prototype systems available for these kinds of hardware, which
must surely be extremely common.

One reason may be that simulation is commonly used
for design verification of synchronous logic.

That's a nice thought, but sooner or later, simulation must give way to
actual testing. There's only so much you can confirm through simulation,
and at some point, you have to try the real hardware and see what it
actually does. I must also point out that there are certain types of
circuits that are difficult, sometimes even impossible, to simulate, not to
mention others that may be easy to set up but take forever to run even on
fast machines. Simulation is an important and valuable first step, but IMHO
it would take a rather credulous engineer to put his entire trust in the
siumulation results.


--
Alex Rast
ad.rast.7@nwnotlink.NOSPAM.com
(remove d., .7, not, and .NOSPAM to reply)

 

[Mike Treseler]

Alex Rast wrote:

interfaces? I'm quite surprised that there aren't more
testing/prototype systems available for these kinds of hardware, which
must surely be extremely common.

One reason may be that simulation is commonly used
for design verification of synchronous logic.


That's a nice thought, but sooner or later, simulation must give way to
actual testing. There's only so much you can confirm through simulation,
and at some point, you have to try the real hardware and see what it
actually does. I must also point out that there are certain types of
circuits that are difficult, sometimes even impossible, to simulate, not to
mention others that may be easy to set up but take forever to run even on
fast machines. Simulation is an important and valuable first step, but IMHO
it would take a rather credulous engineer to put his entire trust in the
siumulation results.

Of course the hardware functions and embedded firmware must
be somehow tested during alpha and beta production.

The question you asked is why few vendors cover this
requirement and I believe that design verification
using simulation and the high quality of pcb fabrication
is part of the reason.

I used to work for a start-up company named Summmation
that made a modular test system that did just what what
you are asking for. Nobody bought it.

The reason was that by the time our development was
complete, the quality of circuit boards and digital
components had improved much more than we expected.
Testing moved up to standard interfaces like
ethernet, PCI, usb, rs232 etc. Most of these
were easily covered by a PC host and the need
for an intermediate test system quickly went away.


-- Mike Treseler

 

[Alex Rast]

at Tue, 20 Jan 2004 18:58:56 GMT in <400D7A70.4090108@flukenetworks.com>,
mike.treseler@flukenetworks.com (Mike Treseler) wrote :

Alex Rast wrote:

interfaces? I'm quite surprised that there aren't more
testing/prototype systems available for these kinds of hardware,
which must surely be extremely common.

One reason may be that simulation is commonly used
for design verification of synchronous logic.

That's a nice thought, but sooner or later, simulation must give way
to actual testing. ... Simulation is an
important and valuable first step, but IMHO it would take a rather
credulous engineer to put his entire trust in the siumulation results.

Of course the hardware functions and embedded firmware must
be somehow tested during alpha and beta production.

The question you asked is why few vendors cover this
requirement and I believe that design verification
using simulation and the high quality of pcb fabrication
is part of the reason.

I see a major weakness in relying on this approach : namely, that it
assumes there should be no significant discrepancies between the
understanding you have after reading data sheets, manuals, etc. as to how
specific pieces of hardware and software work, and how they *actually*
work. Many components have all sorts of undocumented idiosyncracies that
place constraints on how you use them, and most software has both such
idiosyncracies and bugs. If you made your design assuming a certain part
would work in a certain way, or with certain timings, voltages, etc., and
in fact it actually doesn't, although there would have been no way for you
to know this in advance through the documentation that was available, you
can easily end up with a board that doesn't work in spite of it passing
simulation.

PCB fab may be high reliability these days, but I think the kinds of
problems you're trying to iron out in testing are the design flaws that you
wouldn't have been able to catch ahead of time. These are not things that
PCB fab has much to do with because the layout houses and the board
fabricators are simply going to follow your schematic.

I used to work for a start-up company named Summmation
that made a modular test system that did just what what
you are asking for. Nobody bought it.

The reason was that by the time our development was
complete, the quality of circuit boards and digital
components had improved much more than we expected.

It sounds as if your system was designed mostly to catch actual hardware
errors that crop up because of manufacturing defects, not those that happen
because your design wouldn't have worked as it was, through no fault of
your own. Is this the case? Notwithstanding, I would also be prepared to
believe that many modern engineers *are* willing to put the kind of blind
trust in their designs to assume that the kinds of problems I'm talking
about aren't going to happen.

Testing moved up to standard interfaces like
ethernet, PCI, usb, rs232 etc.

Most of the high-speed, standard interfaces are themselves complex, with
numerous subtleties as well as requirements which mean that there's a
considerable likelihood that if your system has one of these interfaces, it
may not work out of the gate on your first revision unless you were lucky
and managed to do everything perfect. I like standard interfaces,
especially during later-stage testing when the low-level details of your
board are debugged, but in the early stages of testing, you can be going
around in circles because the standard interface isn't working, perhaps
because of some obscure, undocumented behaviour of one of the components in
your design.

In addition, these interfaces take time to design and may require complex
drivers. The net result is that it can take months to get to a point where
you're able to test anything at all.

I think my central point is : It would be nice to assume your design is
perfect, and will work flawlessly, but in the real world, no engineer knows
everything, and it's more likely it won't work at first. You're going to
have to get some data about how the hardware *really* works, then refine
your design. This is where I see a gap in available testing devices.

--
Alex Rast
ad.rast.7@nwnotlink.NOSPAM.com
(remove d., .7, not, and .NOSPAM to reply)

 

[Ray Andraka]

FPGAs offer an unprecedented tool in design verification: reconfiguration. If
done carefully, one can completely isolate the board level testing from the
FPGA guts testing when the FPGA is used as some sort of data processor. For
example, if the FPGA is interfacing memory, you can make a fairly simple FPGA
build to test the memory using your FPGA memory interface a test pattern
generator and a test pattern checker. Such a tester can test the memories much
more completely and under more stringent conditions than any other method of
testing. When there are multiple memories, for example, the FPGA can test all
in parallel using worst case patterns. If the memory passes with tht test, it
will likely not fail in normal use. Likewise, the interface pieces can be
tested without the rest of the FPGA guts, and interfaces exercised much harder
than they might be in normal use. Take advantage of the FPGA's
reconfigurability to check out your board level design and your FPGA interfaces
to that design. Once those are debugged, then getting it all to play together
is not hard if you've simulated the pieces and have done your homework on the
timing. I discuss this approach with considerably more detail in my paper "An
FPGA Based Processor Yields a Real Time High Fidelity Radar Environment
Simulator", which is available for download from my website at no charge.
--
--Ray Andraka, P.E.
President, the Andraka Consulting Group, Inc.
401/884-7930 Fax 401/884-7950
email ray@andraka.com
http://www.andraka.com

"They that give up essential liberty to obtain a little
temporary safety deserve neither liberty nor safety."
-Benjamin Franklin, 1759