Handling overflow in a self-repeating frequency counter

[Philip Pemberton]

Hi guys.

I'm currently working on the HDL code for an open-source project which
images 'strange and unusual' magnetic media (mainly floppy discs but also
MFM and RLL hard disc drives).

To do this, I measure the time between incoming pulses on the RD_DATA
line. One pulse indicates a magnetic transition. The time between a given
pair of pulses encodes (in a roundabout way) a data bit. This is a gross
oversimplification, but serves the purpose for this explanation.

After measuring the time between two pulses, it re-arms and measures
between the second pulse and the one following it. Rinse, repeat.

The captured timing values are stored in an on-FPGA FIFO buffer, then
passed onto an off-chip Cypress 100MHz 512K*8bit high speed static RAM
(this thing needs A LOT of storage). The period counter's reference clock
is 100MHz. 10 nanoseconds per count. The most significant bit is reserved
for a 'flag' bit (it's used by other parts of the design), leaving a
range of 0 to 127 for the counter.

The problem is, a lot of the timing values are about 800 counts long. To
work around this, I implemented logic to store a 'special' value in the
stream if the counter overflows from 127 to 0. This value is "8'd0" -- 8
bits, all zeroes.

This raises a problem: if any two incoming pulses are measured at exactly
128 clocks apart (or a modulo thereof), the HDL stores a carry (0x00)...
immediately followed by the value of the counter, which is 0x00 (because
the count of clock pulses is a multiple of 128). This is messing up the
datastream, and I only caught it because a friend mentioned it, I added a
test case to the testbench, and I got a truckload of assertion failures.

The Verilog code is here:
http://hg.discferret.com/microcode/file/1b4ea3919380/DiscReader.v
http://hg.discferret.com/microcode/file/1b4ea3919380/
Flag_Delay1tcy_OneCycle.v
(other modules are here if you need them: http://hg.discferret.com/
microcode/file/1b4ea3919380 )

I have also rewritten DiscReader.v to use a free-running counter (which
seems like a better solution) -- that version is here:
http://pastebin.com/evMtjXWj

Feel free to offer comments on either of these... the rewrite is later
code, and as far as I can tell fully synchronous to the clock (the older
code has some other corner cases -- hence the rewrite).


Effectively the byte-stream format is:
CARRY starts at 0.
0x00 or 0x80 -- add 128 to CARRY.
Anything else -- the timing value is CARRY + (this_byte AND 0x7F).
Clear CARRY to zero afterwards.


The best idea I've come up with thus far is to scrap the overflow logic,
cause the acquisition to abort with an error state if the timer
overflows, and use a 12-bit or larger counter (which would allow for a
maximum count of 40us). This would of course mean I'd have to rewrite the
RAM controller logic to handle a 16-bit stream and chunk it into
bytes...


Can anyone think of a good way to resolve this, ideally without
introducing timing jitter (although the acquisition clock is PLL'd off a
20MHz TTL oscillator, so there's probably a bit of that already)? I'm not
overly attached to the bitstream format, so feel free to propose changes.


Thanks,
--
Phil.
philpem@philpem.me.uk
http://www.philpem.me.uk/

 

[glen herrmannsfeldt]

Philip Pemberton <philpem@philpem.me.uk> wrote:

I'm currently working on the HDL code for an open-source project which
images 'strange and unusual' magnetic media (mainly floppy discs but also
MFM and RLL hard disc drives).

To do this, I measure the time between incoming pulses on the RD_DATA
line. One pulse indicates a magnetic transition. The time between a given
pair of pulses encodes (in a roundabout way) a data bit. This is a gross
oversimplification, but serves the purpose for this explanation.

After measuring the time between two pulses, it re-arms and measures
between the second pulse and the one following it. Rinse, repeat.

The captured timing values are stored in an on-FPGA FIFO buffer, then
passed onto an off-chip Cypress 100MHz 512K*8bit high speed static RAM
(this thing needs A LOT of storage). The period counter's reference clock
is 100MHz. 10 nanoseconds per count. The most significant bit is reserved
for a 'flag' bit (it's used by other parts of the design), leaving a
range of 0 to 127 for the counter.

The bit rate for floppies is 125000b/s at the low end, and, as far as
I know, 1000000b/s at the high end. To decode FM or MFM data, you
don't need a hugh amount of resolution. There are only two different
times between transitions, at least ideally.

The GCR code used by older Apple formats has more different times, but
not all that many, and the bit rate is on the low side.

To write a floppy from data, you need to consider precompensation,
which needs a little finer timing. Note that writing a floppy from
your data stream, you will also need to consider precompensation.

Anyway, my suggestion is to use a slower clock. Then it should only
overflow at splice points, or other places where the data doesn't
matter.

The problem is, a lot of the timing values are about 800 counts long. To
work around this, I implemented logic to store a 'special' value in the
stream if the counter overflows from 127 to 0. This value is "8'd0" -- 8
bits, all zeroes.

The design of codes that can be uniquely decoded is well known.
One answer is to store one more than the overflow value.
That is, a 128 count should be X'0001', so X'00' always means overflow,
and never data. X'007f' then will be 254, and X'000001' will be 255.

The logic is a little more complicated, but shouldn't be hard to
fit into an FPGA.

-- glen

 

[Brian Drummond]

On Fri, 06 Jan 2012 02:49:54 +0000, Philip Pemberton wrote:

Hi guys.

I'm currently working on the HDL code for an open-source project which
images 'strange and unusual' magnetic media (mainly floppy discs but
also MFM and RLL hard disc drives).

To do this, I measure the time between incoming pulses on the RD_DATA
line. One pulse indicates a magnetic transition. The time between a
given pair of pulses encodes (in a roundabout way) a data bit.
....

The problem is, a lot of the timing values are about 800 counts long. To
work around this, I implemented logic to store a 'special' value in the
stream if the counter overflows from 127 to 0. This value is "8'd0" -- 8
bits, all zeroes.

Consider A-law (or mu-law) encoding instead.

- Brian

 

[glen herrmannsfeldt]

Brian Drummond <brian@shapes.demon.co.uk> wrote:

On Fri, 06 Jan 2012 02:49:54 +0000, Philip Pemberton wrote:

I'm currently working on the HDL code for an open-source project which
images 'strange and unusual' magnetic media (mainly floppy discs but
also MFM and RLL hard disc drives).

To do this, I measure the time between incoming pulses on the RD_DATA
line. One pulse indicates a magnetic transition. The time between a
given pair of pulses encodes (in a roundabout way) a data bit.

The problem is, a lot of the timing values are about 800 counts long. To
work around this, I implemented logic to store a 'special' value in the
stream if the counter overflows from 127 to 0. This value is "8'd0" -- 8
bits, all zeroes.

Consider A-law (or mu-law) encoding instead.

That should work. I think, though, that it doesn't matter much.

The gaps between sectors, and between the before/after the index address
mark, all have data bits in them as formatted.

The problem comes with write splice. (Put "write splice" floppy into
google, and some hits will be for floppy controllers. At least the
one for the 279x describes the track format.)

Each sector has a header with the track/sector/side/length bytes.
When writing, the controller finds the header for the appropriate
sector, counts off a specific number of bytes (different for FM and
MFM), then turns on the write current. It then writes some binary
zeros, an address mark, followed by the data, CRC, a few more bytes,
then turns the write current off. The point where it turns on and
off is the write splice, and I believe that is where you see the
long counts. The exact length doesn't matter much, and will vary
between drives, and with the motor speed, with a tolerance of maybe
about 10%.

The gap after the sector and before the next sector header is long
enough such that within tolerance the write splice won't overwrite
the next header. The gaps are generously large (but with data bits
in them) to allow for a variation in motor speeds and controller
clocks. (Much more motor speed than clock.)

There are enough data bits after the write splice and before the
address mark to sync up the PLL.

What is the variation in length for these long counts, and about
how many do you find on a track?

I presume you start reading on the index pulse, and stop at the
next index pulse. If you want to convert a track image back to
a floppy, you will start writing at the index pulse, and either stop
at the next (if you don't run out of data), or continue writing
binary zeros (in FM or MFM form) until the index pulse.

-- glen

 

[Tim Wescott]

On Fri, 06 Jan 2012 03:18:38 +0000, glen herrmannsfeldt wrote:

Philip Pemberton <philpem@philpem.me.uk> wrote:

I'm currently working on the HDL code for an open-source project which
images 'strange and unusual' magnetic media (mainly floppy discs but
also MFM and RLL hard disc drives).

To do this, I measure the time between incoming pulses on the RD_DATA
line. One pulse indicates a magnetic transition. The time between a
given pair of pulses encodes (in a roundabout way) a data bit. This is
a gross oversimplification, but serves the purpose for this
explanation.

After measuring the time between two pulses, it re-arms and measures
between the second pulse and the one following it. Rinse, repeat.

The captured timing values are stored in an on-FPGA FIFO buffer, then
passed onto an off-chip Cypress 100MHz 512K*8bit high speed static RAM
(this thing needs A LOT of storage). The period counter's reference
clock is 100MHz. 10 nanoseconds per count. The most significant bit is
reserved for a 'flag' bit (it's used by other parts of the design),
leaving a range of 0 to 127 for the counter.

The bit rate for floppies is 125000b/s at the low end, and, as far as I
know, 1000000b/s at the high end. To decode FM or MFM data, you don't
need a hugh amount of resolution. There are only two different times
between transitions, at least ideally.

The GCR code used by older Apple formats has more different times, but
not all that many, and the bit rate is on the low side.

To write a floppy from data, you need to consider precompensation, which
needs a little finer timing. Note that writing a floppy from your data
stream, you will also need to consider precompensation.

Anyway, my suggestion is to use a slower clock. Then it should only
overflow at splice points, or other places where the data doesn't
matter.

The problem is, a lot of the timing values are about 800 counts long.
To work around this, I implemented logic to store a 'special' value in
the stream if the counter overflows from 127 to 0. This value is "8'd0"
-- 8 bits, all zeroes.

The design of codes that can be uniquely decoded is well known. One
answer is to store one more than the overflow value. That is, a 128
count should be X'0001', so X'00' always means overflow, and never data.
X'007f' then will be 254, and X'000001' will be 255.

The logic is a little more complicated, but shouldn't be hard to fit
into an FPGA.

-- glen

If you want the data to fit neatly into seven bits, give your counter a
period of 127, and assign X'00' to overflow, with X'01' through X'7f'
denoting a bit hit at that count. That'll insure unique data, without
blowing out your bit-count budget.

You'll complicate the decoding a bit, but that's an easy task for a
computer to carry out.

--
My liberal friends think I'm a conservative kook.
My conservative friends think I'm a liberal kook.
Why am I not happy that they have found common ground?

Tim Wescott, Communications, Control, Circuits & Software
http://www.wescottdesign.com

 

[Philip Pemberton]

On Fri, 06 Jan 2012 12:18:50 -0600, Tim Wescott wrote:

If you want the data to fit neatly into seven bits, give your counter a
period of 127, and assign X'00' to overflow, with X'01' through X'7f'
denoting a bit hit at that count. That'll insure unique data, without
blowing out your bit-count budget.

You'll complicate the decoding a bit, but that's an easy task for a
computer to carry out.

That's more or less what I'm doing now.

The problem is, there's a race condition -- if there's an overflow at the
same time as an incoming data pulse, then the overflow is stored, but the
transition is ignored. In the example you gave, that's when the count is
*exactly* 127, or a modulo thereof.

What I need to do is eliminate this race condition...


Here's my code --

module DiscReader(CLOCK, CLKEN, RUN, FD_RDDATA_IN, FD_INDEX_IN, RESET,
DATA, WRITE);
parameter BITS = 8; // data bits

input
CLOCK; // counter clock
input
CLKEN; // counter clock enable
input
RUN; // enable input -- 1 to acquire
input FD_RDDATA_IN; //
read data
input FD_INDEX_IN; //
index pulse
input
RESET; // asynchronous reset

output reg [BITS-1:0] DATA; //
data output to RAM
output reg
WRITE; // write output to RAM


/////////////////////////////////////////////////////////////////////////////
// Input synchronisation
wire FD_RDDATA_IN_tcysync, FD_INDEX_IN_tcysync;
Flag_Delay1tcy_OneCycle _fcd_rddata (CLOCK, FD_RDDATA_IN,
FD_RDDATA_IN_tcysync);
Flag_Delay1tcy_OneCycle _fcd_index (CLOCK, FD_INDEX_IN,
FD_INDEX_IN_tcysync);

/////////////////////////////////////////////////////////////////////////////
// Free-running counter for frequency measurement
reg [BITS-2:0] counter, last_count;
always @(posedge CLOCK) begin
if (RESET) begin
// reset active -- clear counter
counter <= 'd0;
end else if (CLKEN) begin
// otherwise increment
counter <= counter + 'd1;
end
end

/////////////////////////////////////////////////////////////////////////////
// Frequency counter

// current counter value
wire [BITS-2:0] cur_count = (last_count <= counter) ?
counter - last_count : //
last count <= counter
('b1 << BITS) - last_count + counter; // last count >
counter

always @(posedge CLOCK) begin
// don't write to memory!
WRITE <= 1'b0;

if (RESET) begin
// reset active -- clear last-count
last_count <= 'd0;
end else begin
if (FD_RDDATA_IN_tcysync | FD_INDEX_IN_tcysync |
(cur_count == (('d1 << BITS-1)-1))) begin
// Index hit, flux transition or overflow
DATA <= {FD_INDEX_IN_tcysync ? 1'b1 :
1'b0, cur_count};

// write to memory
WRITE <= 1'b1;

// Update 'previous count value' register
last_count <= counter;
end
end
end

endmodule


module Flag_Delay1tcy_OneCycle(clk, in, out);
input clk, in;
output out;

reg[1:0] in_Delayed;
always @(posedge clk) in_Delayed <= {in_Delayed[0], in};
assign out = (in_Delayed[0] && !in_Delayed[1]);
endmodule


// Testbench -- provokes race condition in DiscReader module

// 5ns time reference with 1ns precision
`timescale 1 ns / 1 ns

module DiscReaderTB;
reg clock, run, rddata, index, reset;
wire clock_enable = 1'b1;

parameter N = 8;

wire [N-1:0] data_r;
wire ram_write;

// instantiate the disc reader
DiscReader U0 (
.CLOCK (clock),
.CLKEN (clock_enable),
.RUN (run),
.FD_RDDATA_IN (rddata),
.FD_INDEX_IN (index),
.RESET (reset),

.DATA (data_r),
.WRITE (ram_write)
);

// set initial state
initial begin
clock = 0;
run = 0;
rddata = 0;
index = 0;
reset = 0;
end

// generate a 100MHz clock
always
#1 clock = !clock;

// set up for variable dumping
initial begin
$dumpfile ("counter.vcd");
$dumpvars;
end

integer k;
// main loop
initial begin
#10
reset = 1;
#10
reset = 0;
run = 1;

for (k=0; k<=16; k=k+1) begin
rddata = 1;
#2
rddata = 0;
#252
;
end

#1000 $finish;
end

endmodule

Thanks.
--
Phil.
philpem@philpem.me.uk
http://www.philpem.me.uk/

 

[glen herrmannsfeldt]

Tim Wescott <tim@seemywebsite.com> wrote:

(snip)

The problem is, a lot of the timing values are about 800 counts long.
To work around this, I implemented logic to store a 'special' value in
the stream if the counter overflows from 127 to 0. This value is "8'd0"
-- 8 bits, all zeroes.

The design of codes that can be uniquely decoded is well known. One
answer is to store one more than the overflow value. That is, a 128
count should be X'0001', so X'00' always means overflow, and never data.
X'007f' then will be 254, and X'000001' will be 255.

The logic is a little more complicated, but shouldn't be hard to fit
into an FPGA.

If you want the data to fit neatly into seven bits, give your counter a
period of 127, and assign X'00' to overflow, with X'01' through X'7f'
denoting a bit hit at that count. That'll insure unique data, without
blowing out your bit-count budget.

That is what I was thinking, but probably didn't get quite right.

You'll complicate the decoding a bit, but that's an easy task for a
computer to carry out.

Well, easy for a computer, slightly harder to write as a state
machine in verilog, but not that much harder.

-- glen

 

[glen herrmannsfeldt]

Philip Pemberton <philpem@philpem.me.uk> wrote:

On Fri, 06 Jan 2012 12:18:50 -0600, Tim Wescott wrote:

If you want the data to fit neatly into seven bits, give your counter a
period of 127, and assign X'00' to overflow, with X'01' through X'7f'
denoting a bit hit at that count. That'll insure unique data, without
blowing out your bit-count budget.

You'll complicate the decoding a bit, but that's an easy task for a
computer to carry out.

That's more or less what I'm doing now.

The problem is, there's a race condition -- if there's an overflow at the
same time as an incoming data pulse, then the overflow is stored, but the
transition is ignored. In the example you gave, that's when the count is
*exactly* 127, or a modulo thereof.

What I need to do is eliminate this race condition...

No, it is worse than a race condition, you have one state that
has two meanings.

OK, the easy way is to do it modulo 64, and use the 64 value to mean 64,
and 0 to mean 0. (It is easier to write modulo 64 than 127.)

For 5.25in FM, the data rate is 125000b/s, the transitions are either
at 8us or 4us, so there will be a lot of 80s. Not so good.

For 5.25in MFM, the data rate is 250000b/s, there is no clock pulse
if either the cell before or after has a data pulse. I believe that
means either 4us or 6us, except for an address mark where it
would be 8us. (I haven't thought about this for a while.
Find a reference to the IBM standard.)

OK, do it modulo 127 with 127 as the overflow code which allows
for a zero code to mean zero. (Most likely only after a 127.)
It isn't that hard to test for 127, and you have plenty of
time to do it.

-- glen

 

[Steven Hirsch]

On 01/05/2012 10:18 PM, glen herrmannsfeldt wrote:

The bit rate for floppies is 125000b/s at the low end, and, as far as
I know, 1000000b/s at the high end. To decode FM or MFM data, you
don't need a hugh amount of resolution. There are only two different
times between transitions, at least ideally.

The GCR code used by older Apple formats has more different times, but
not all that many, and the bit rate is on the low side.

To write a floppy from data, you need to consider precompensation,
which needs a little finer timing. Note that writing a floppy from
your data stream, you will also need to consider precompensation.

Anyway, my suggestion is to use a slower clock. Then it should only
overflow at splice points, or other places where the data doesn't
matter.

What Philip failed to mention was that the product in question is also
intended to read data streams from MFM hard drives. A slower clock may not be
an option.

 

[glen herrmannsfeldt]

Steven Hirsch <snhirsch@gmail.com> wrote:

(snip, I wrote)

To write a floppy from data, you need to consider precompensation,
which needs a little finer timing. Note that writing a floppy from
your data stream, you will also need to consider precompensation.

Anyway, my suggestion is to use a slower clock. Then it should only
overflow at splice points, or other places where the data doesn't
matter.

What Philip failed to mention was that the product in question is also
intended to read data streams from MFM hard drives. A slower clock may not be
an option.

I was assuming that you knew which type of drive you were working
with and could select accordingly.

Interestingly, though, the IBM PC/AT disk controller uses a 24MHz
crystal for the floppy drive, and, if I remember right, 10MHz
for the MFM hard drive.

-- glen

 

[Tim Wescott]

On Fri, 06 Jan 2012 20:49:01 +0000, Philip Pemberton wrote:

On Fri, 06 Jan 2012 12:18:50 -0600, Tim Wescott wrote:

If you want the data to fit neatly into seven bits, give your counter a
period of 127, and assign X'00' to overflow, with X'01' through X'7f'
denoting a bit hit at that count. That'll insure unique data, without
blowing out your bit-count budget.

You'll complicate the decoding a bit, but that's an easy task for a
computer to carry out.

That's more or less what I'm doing now.

The problem is, there's a race condition -- if there's an overflow at
the same time as an incoming data pulse, then the overflow is stored,
but the transition is ignored. In the example you gave, that's when the
count is *exactly* 127, or a modulo thereof.

What I need to do is eliminate this race condition...

(lots of headache-inducing code snipped)

It shouldn't. A number between 1 and 127 means that you got an event at
that time. A zero means that your counter went from 127 to _1_ (because
0 is forbidden).

// Note that my Verilog is very rusty: hopefully you'll get the idea

// This should cause a count between 1 and 127 to be emitted when an
// event occurs, and a 'count' of 0 to be emitted _only_ when an
// overflow occurs.
if (event)
begin
out <= count;
emit <= 1;
end
else if (count == 127)
begin
out <= 0;
emit <= 1;
end

// The counter update is independent to the above:
if (count == 127)
count <= 1;
else
count <= count + 1;

Like Glen said, the above is somewhat logic-intensive because you're
counting modulo 127 -- but it can be done.

--
Tim Wescott
Control system and signal processing consulting
www.wescottdesign.com