Removing DC offset from ADC Buffer...

[Gold_Spark]

I\'m using a STM32 Cortex M0+ to read an AC signal from a CT. I\'m sampling at 6kHz and storing 400 samples. The signal has a DC bias equal to Vcc/2 = 1.65V. In the digital domain this is 2048. In hardware this DC value is very precise, but when sampling it, it varies from 2044 to 2052 inside the buffer. Now if I want to do RMS in that set of data, I need to find a way to deal with this DC bias variation.

I have been thinking the following:

1- Subtract a fixed value of 2048 from each ADC reading. This is no so good as I said above this value may vary slightly. Also, if I want to read zero cross it may cause errors to choose exactly 2048 as reference.
2- Sample the DC bias and average it.
3- Don\'t remove the DC bias. If I calculate RMS then I would get the DC value when there\'s no input signal.
4- Use a more sophisticated software high pass filter?

Example of DC bias readings:

ADC_buffer
[0] 2048
[1] 2046
[2] 2049
[3] 2051
[4] 2051
[5] 2052
[6] 2050
[7] 2050
[8] 2050
[9] 2047
[10] 2049
[11] 2050
[12] 2049

I appreciate any help or suggestion.

 

[server]

On Sat, 19 Sep 2020 20:38:03 -0700 (PDT), Gold_Spark
<bluelectronx@gmail.com> wrote:

I\'m using a STM32 Cortex M0+ to read an AC signal from a CT. I\'m sampling at 6kHz and storing 400 samples. The signal has a DC bias equal to Vcc/2 = 1.65V. In the digital domain this is 2048. In hardware this DC value is very precise, but when sampling it, it varies from 2044 to 2052 inside the buffer. Now if I want to do RMS in that set of data, I need to find a way to deal with this DC bias variation.

I have been thinking the following:

1- Subtract a fixed value of 2048 from each ADC reading. This is no so good as I said above this value may vary slightly. Also, if I want to read zero cross it may cause errors to choose exactly 2048 as reference.
2- Sample the DC bias and average it.
3- Don\'t remove the DC bias. If I calculate RMS then I would get the DC value when there\'s no input signal.
4- Use a more sophisticated software high pass filter?

Example of DC bias readings:

ADC_buffer
[0] 2048
[1] 2046
[2] 2049
[3] 2051
[4] 2051
[5] 2052
[6] 2050
[7] 2050
[8] 2050
[9] 2047
[10] 2049
[11] 2050
[12] 2049

I appreciate any help or suggestion.

Just average the samples and subtract that average from each new
sample. There are several ways to do that average:

Sum the last N samples and divide by N.

Exponential smoothing: Avg = Avg + (new-Avg) / N

A real higher-order lowpass filter, in software.


You could average each 400 points and use that as the \"dc bias\" to
subtract out before doing the RMS, but 400 isn\'t a lot of points, so
the zero value will be noisy. Unless you really have a use for 400
points, there\'s no need to store any of the samples to get a running
RMS.

I\'d go for the exponential smoothing. Then square each zero-corrected
sample, lowpass filter, square root when needed.



--

John Larkin Highland Technology, Inc

Science teaches us to doubt.

Claude Bernard

 

[whit3rd]

On Saturday, September 19, 2020 at 9:49:47 PM UTC-7, jla...@highlandsniptechnology.com wrote:

On Sat, 19 Sep 2020 20:38:03 -0700 (PDT), Gold_Spark
bluele...@gmail.com> wrote:

I\'m using a STM32 Cortex M0+ to read an AC signal from a CT. I\'m sampling at 6kHz and storing 400 samples. The signal has a DC bias equal to Vcc/2 = 1.65V. In the digital domain this is 2048. In hardware this DC value is very precise, but when sampling it, it varies from 2044 to 2052 inside the buffer. Now if I want to do RMS in that set of data, I need to find a way to deal with this DC bias variation.

I have been thinking the following:

1- Subtract a fixed value of 2048 from each ADC reading. This is no so good as I said above this value may vary slightly. Also, if I want to read zero cross it may cause errors to choose exactly 2048 as reference.
2- Sample the DC bias and average it.
3- Don\'t remove the DC bias. If I calculate RMS then I would get the DC value when there\'s no input signal.
4- Use a more sophisticated software high pass filter?

Example of DC bias readings:

ADC_buffer
[0] 2048
[1] 2046
[2] 2049
[3] 2051
[4] 2051
[5] 2052
[6] 2050
[7] 2050
[8] 2050
[9] 2047
[10] 2049
[11] 2050
[12] 2049

I appreciate any help or suggestion.
Just average the samples and subtract that average from each new
sample. There are several ways to do that average:

Sum the last N samples and divide by N.

that\'s a FIR filter (finite impulse response)
.... if you choose the sample size and know the likely interference sources
(like, 60 Hz ripple), it allows you to place a null appropriately

> Exponential smoothing: Avg = Avg + (new-Avg) / N

that\'s a IIR filter (infinite impulse response); usually not a great choice

> A real higher-order lowpass filter, in software.
Like, FIR with weights.

 

[server]

On Sun, 20 Sep 2020 00:26:47 -0700 (PDT), whit3rd <whit3rd@gmail.com>
wrote:

On Saturday, September 19, 2020 at 9:49:47 PM UTC-7, jla...@highlandsniptechnology.com wrote:
On Sat, 19 Sep 2020 20:38:03 -0700 (PDT), Gold_Spark
bluele...@gmail.com> wrote:

I\'m using a STM32 Cortex M0+ to read an AC signal from a CT. I\'m sampling at 6kHz and storing 400 samples. The signal has a DC bias equal to Vcc/2 = 1.65V. In the digital domain this is 2048. In hardware this DC value is very precise, but when sampling it, it varies from 2044 to 2052 inside the buffer. Now if I want to do RMS in that set of data, I need to find a way to deal with this DC bias variation.

I have been thinking the following:

1- Subtract a fixed value of 2048 from each ADC reading. This is no so good as I said above this value may vary slightly. Also, if I want to read zero cross it may cause errors to choose exactly 2048 as reference.
2- Sample the DC bias and average it.
3- Don\'t remove the DC bias. If I calculate RMS then I would get the DC value when there\'s no input signal.
4- Use a more sophisticated software high pass filter?

Example of DC bias readings:

ADC_buffer
[0] 2048
[1] 2046
[2] 2049
[3] 2051
[4] 2051
[5] 2052
[6] 2050
[7] 2050
[8] 2050
[9] 2047
[10] 2049
[11] 2050
[12] 2049

I appreciate any help or suggestion.
Just average the samples and subtract that average from each new
sample. There are several ways to do that average:

Sum the last N samples and divide by N.

that\'s a FIR filter (finite impulse response)
... if you choose the sample size and know the likely interference sources
(like, 60 Hz ripple), it allows you to place a null appropriately

Exponential smoothing: Avg = Avg + (new-Avg) / N

that\'s a IIR filter (infinite impulse response); usually not a great choice

Why not? I see a lot of irrational prejuduce against simple IIR
filters, in code and in FPGAs. Some people would rather write a
hundred lines of code instead of one.

If integer math is all that\'s available, the divide-by-N is just a
right shift.

A real higher-order lowpass filter, in software.
Like, FIR with weights.

Or a multipole IIR. It\'s fun to do those with just right-shifts too.
Again, just a few lines of code.







--

John Larkin Highland Technology, Inc

Science teaches us to doubt.

Claude Bernard

 

[Uwe Bonnes]

Gold_Spark <bluelectronx@gmail.com> wrote:

I\'m using a STM32 Cortex M0+ to read an AC signal from a CT. I\'m sampling at 6kHz and storing 400 samples. The signal has a DC bias equal to Vcc/2 = 1.65V. In the digital domain this is 2048. In hardware this DC value is very precise, but when sampling it, it varies from 2044 to 2052 inside the buffer. Now if I want to do RMS in that set of data, I need to find a way to deal with this DC bias variation.

I have been thinking the following:

1- Subtract a fixed value of 2048 from each ADC reading. This is no so good as I said above this value may vary slightly. Also, if I want to read zero cross it may cause errors to choose exactly 2048 as reference.
2- Sample the DC bias and average it.
3- Don\'t remove the DC bias. If I calculate RMS then I would get the DC value when there\'s no input signal.
4- Use a more sophisticated software high pass filter?

Example of DC bias readings:

ADC_buffer
[0] 2048
[1] 2046
[2] 2049
[3] 2051
[4] 2051
[5] 2052
[6] 2050
[7] 2050
[8] 2050
[9] 2047
[10] 2049
[11] 2050
[12] 2049

Run ADC with DMA continous at a high rate and oversample. Things will get much smoother that way,
if your sampling is done right.
--
Uwe Bonnes bon@elektron.ikp.physik.tu-darmstadt.de

Institut fuer Kernphysik Schlossgartenstrasse 9 64289 Darmstadt
--------- Tel. 06151 1623569 ------- Fax. 06151 1623305 ---------

 

[server]

On 20 Sep 2020 11:50:50 GMT, Uwe Bonnes
<bon@hertz.ikp.physik.tu-darmstadt.de> wrote:

Gold_Spark <bluelectronx@gmail.com> wrote:
I\'m using a STM32 Cortex M0+ to read an AC signal from a CT. I\'m sampling at 6kHz and storing 400 samples. The signal has a DC bias equal to Vcc/2 = 1.65V. In the digital domain this is 2048. In hardware this DC value is very precise, but when sampling it, it varies from 2044 to 2052 inside the buffer. Now if I want to do RMS in that set of data, I need to find a way to deal with this DC bias variation.

I have been thinking the following:

1- Subtract a fixed value of 2048 from each ADC reading. This is no so good as I said above this value may vary slightly. Also, if I want to read zero cross it may cause errors to choose exactly 2048 as reference.
2- Sample the DC bias and average it.
3- Don\'t remove the DC bias. If I calculate RMS then I would get the DC value when there\'s no input signal.
4- Use a more sophisticated software high pass filter?

Example of DC bias readings:

ADC_buffer
[0] 2048
[1] 2046
[2] 2049
[3] 2051
[4] 2051
[5] 2052
[6] 2050
[7] 2050
[8] 2050
[9] 2047
[10] 2049
[11] 2050
[12] 2049


Run ADC with DMA continous at a high rate and oversample. Things will get much smoother that way,
if your sampling is done right.

He still needs to take the DC offset out. Given that the signal comes
from a CT, and has no inherent DC component, it can be auto-zeroed by
just averaging lots of samples.



--

John Larkin Highland Technology, Inc

Science teaches us to doubt.

Claude Bernard

 

[whit3rd]

On Sunday, September 20, 2020 at 3:26:13 AM UTC-7, jla...@highlandsniptechnology.com wrote:

On Sun, 20 Sep 2020 00:26:47 -0700 (PDT), whit3rd <whi...@gmail.com
wrote:
On Saturday, September 19, 2020 at 9:49:47 PM UTC-7, jla...@highlandsniptechnology.com wrote:

Just average the samples and subtract that average from each new
sample. There are several ways to do that average:

Sum the last N samples and divide by N.

that\'s a FIR filter (finite impulse response)
... if you choose the sample size and know the likely interference sources
(like, 60 Hz ripple), it allows you to place a null appropriately

Exponential smoothing: Avg = Avg + (new-Avg) / N

that\'s a IIR filter (infinite impulse response); usually not a great choice

Why not? I see a lot of irrational prejuduce against simple IIR
filters, in code and in FPGAs. Some people would rather write a
hundred lines of code instead of one.

Oh, it\'s simple, all right, but it has a long startup transient.
That means it doesn\'t deal with lightning-strike artifacts well, either.
In cases (like, reading an analog tape) where you want to reject a
particular frequency (the recording bias generator), it doesn\'t have enough
flexibility. FIR was eventually the big winner in CD playback, for instance.

 

[server]

On Sun, 20 Sep 2020 13:33:11 -0700 (PDT), whit3rd <whit3rd@gmail.com>
wrote:

On Sunday, September 20, 2020 at 3:26:13 AM UTC-7, jla...@highlandsniptechnology.com wrote:
On Sun, 20 Sep 2020 00:26:47 -0700 (PDT), whit3rd <whi...@gmail.com
wrote:
On Saturday, September 19, 2020 at 9:49:47 PM UTC-7, jla...@highlandsniptechnology.com wrote:

Just average the samples and subtract that average from each new
sample. There are several ways to do that average:

Sum the last N samples and divide by N.

that\'s a FIR filter (finite impulse response)
... if you choose the sample size and know the likely interference sources
(like, 60 Hz ripple), it allows you to place a null appropriately

Exponential smoothing: Avg = Avg + (new-Avg) / N

that\'s a IIR filter (infinite impulse response); usually not a great choice

Why not? I see a lot of irrational prejuduce against simple IIR
filters, in code and in FPGAs. Some people would rather write a
hundred lines of code instead of one.

Oh, it\'s simple, all right, but it has a long startup transient.

Any lowpass filter or averager does. Just poke a starting value into
the integrator node if you\'re in a hurry, ADC midscale in this case.
Or go for 2nd order. An auto-zero system works fine with a droopy 1st
order filter.

You have to wait for an FIR filter to fill up, and the output is
nonsense until it does. You can poke a start value into it, but you
have to load all the nodes.

>That means it doesn\'t deal with lightning-strike artifacts well, either.

Presumably an ADC rails on a huge transient. Why would an IIR filter
be worse than a FIR for a spike? If you hate long tails, go 2nd order.

In cases (like, reading an analog tape) where you want to reject a
particular frequency (the recording bias generator), it doesn\'t have enough
flexibility. FIR was eventually the big winner in CD playback, for instance.

It\'s impressive how many convoluted arguments people make to avoid IIR
digital filters. Most of them reduce to \"It\'s too simple and I don\'t
like it.\"



--

John Larkin Highland Technology, Inc

Science teaches us to doubt.

Claude Bernard

 

[whit3rd]

On Sunday, September 20, 2020 at 2:01:05 PM UTC-7, jla...@highlandsniptechnology.com wrote:

On Sun, 20 Sep 2020 13:33:11 -0700 (PDT), whit3rd <whi...@gmail.com
wrote:

On Sunday, September 20, 2020 at 3:26:13 AM UTC-7, jla...@highlandsniptechnology.com wrote:
On Sun, 20 Sep 2020 00:26:47 -0700 (PDT), whit3rd <whi...@gmail.com
wrote:
On Saturday, September 19, 2020 at 9:49:47 PM UTC-7, jla...@highlandsniptechnology.com wrote:

Just average the samples and subtract that average from each new
sample. There are several ways to do that average:

Sum the last N samples and divide by N.

that\'s a FIR filter (finite impulse response)
... if you choose the sample size and know the likely interference sources
(like, 60 Hz ripple), it allows you to place a null appropriately

Exponential smoothing: Avg = Avg + (new-Avg) / N

that\'s a IIR filter (infinite impulse response); usually not a great choice

Why not? I see a lot of irrational prejuduce against simple IIR
filters, in code and in FPGAs. Some people would rather write a
hundred lines of code instead of one.

Oh, it\'s simple, all right, but it has a long startup transient.
Any lowpass filter or averager does. Just poke a starting value into
the integrator node if you\'re in a hurry, ADC midscale in this case.

A lowpass needn\'t be considered appropriate during startup (and
brute-force setting a starting value helps). FIR has a time-limit
on its history, which is often completely appropriate and useful.

That means it doesn\'t deal with lightning-strike artifacts well, either.
Presumably an ADC rails on a huge transient. Why would an IIR filter
be worse than a FIR for a spike?

Small signal in big digitizer range, of course. Your \'rails\' scenario is
a measurement failure, and there\'s multiple ways to treat such a thing,
which FIR does by... ignoring the spike a few samples afterward.
IIR doesn\'t do that, so saturating the digitizer is an alternate solution that
you don\'t seem to dislike.

> It\'s impressive...

I\'m pleased that my response impresses you.

how many convoluted arguments people make to avoid IIR
digital filters. Most of them reduce to \"It\'s too simple and I don\'t
like it.\"

But not any that I mentioned; what ARE those other \"convoluted arguments\"?
I\'d like to judge their merits for myself...

 

[server]

On Sun, 20 Sep 2020 16:29:30 -0700 (PDT), whit3rd <whit3rd@gmail.com>
wrote:

On Sunday, September 20, 2020 at 2:01:05 PM UTC-7, jla...@highlandsniptechnology.com wrote:
On Sun, 20 Sep 2020 13:33:11 -0700 (PDT), whit3rd <whi...@gmail.com
wrote:

On Sunday, September 20, 2020 at 3:26:13 AM UTC-7, jla...@highlandsniptechnology.com wrote:
On Sun, 20 Sep 2020 00:26:47 -0700 (PDT), whit3rd <whi...@gmail.com
wrote:
On Saturday, September 19, 2020 at 9:49:47 PM UTC-7, jla...@highlandsniptechnology.com wrote:

Just average the samples and subtract that average from each new
sample. There are several ways to do that average:

Sum the last N samples and divide by N.

that\'s a FIR filter (finite impulse response)
... if you choose the sample size and know the likely interference sources
(like, 60 Hz ripple), it allows you to place a null appropriately

Exponential smoothing: Avg = Avg + (new-Avg) / N

that\'s a IIR filter (infinite impulse response); usually not a great choice

Why not? I see a lot of irrational prejuduce against simple IIR
filters, in code and in FPGAs. Some people would rather write a
hundred lines of code instead of one.

Oh, it\'s simple, all right, but it has a long startup transient.
Any lowpass filter or averager does. Just poke a starting value into
the integrator node if you\'re in a hurry, ADC midscale in this case.

A lowpass needn\'t be considered appropriate during startup (and
brute-force setting a starting value helps). FIR has a time-limit
on its history, which is often completely appropriate and useful.


That means it doesn\'t deal with lightning-strike artifacts well, either.
Presumably an ADC rails on a huge transient. Why would an IIR filter
be worse than a FIR for a spike?

Small signal in big digitizer range, of course. Your \'rails\' scenario is
a measurement failure, and there\'s multiple ways to treat such a thing,
which FIR does by... ignoring the spike a few samples afterward.
IIR doesn\'t do that, so saturating the digitizer is an alternate solution that
you don\'t seem to dislike.

It\'s impressive...

I\'m pleased that my response impresses you.

how many convoluted arguments people make to avoid IIR
digital filters. Most of them reduce to \"It\'s too simple and I don\'t
like it.\"

But not any that I mentioned; what ARE those other \"convoluted arguments\"?
I\'d like to judge their merits for myself...

I was just thinking how crazy it woud be to use, say, a 5000-tap FIR
filter to compute a good autozero average value of the last 5000
samples.

Those 5000 multiply-by-1-and-add blocks will need a lot of logic.



--

John Larkin Highland Technology, Inc

Science teaches us to doubt.

Claude Bernard

 

[Ricketty C]

On Saturday, September 19, 2020 at 11:38:09 PM UTC-4, Gold_Spark wrote:

I\'m using a STM32 Cortex M0+ to read an AC signal from a CT. I\'m sampling at 6kHz and storing 400 samples. The signal has a DC bias equal to Vcc/2 = 1.65V. In the digital domain this is 2048. In hardware this DC value is very precise, but when sampling it, it varies from 2044 to 2052 inside the buffer. Now if I want to do RMS in that set of data, I need to find a way to deal with this DC bias variation.

I have been thinking the following:

1- Subtract a fixed value of 2048 from each ADC reading. This is no so good as I said above this value may vary slightly. Also, if I want to read zero cross it may cause errors to choose exactly 2048 as reference.

Only you can tell us if this fairly small variations are large enough to cause problems.

The part I am confused about is when you say the bias is very stable in the analog domain this bias voltage is very stable, but you show pretty small variation in the ADC readings (2044 to 2052, ±4) or about ±0.003V, ±0.2% max. I think this really is the DC bias you are measuring.

I guess I was thinking you were blaming it on the ADC rather than the input signal, but you didn\'t say that.


> 2- Sample the DC bias and average it.

What\'s wrong with that? Is this easy to do?


> 3- Don\'t remove the DC bias. If I calculate RMS then I would get the DC value when there\'s no input signal.

Again, you tell us if that\'s a problem?


> 4- Use a more sophisticated software high pass filter?

Funny that people propose you use a low pass filter and subtract. A high pass would just provide the AC without the DC. It would take some time to start up, but can work very effectively. If this sampling is continuous that would work fine.

Example of DC bias readings:

ADC_buffer
[0] 2048
[1] 2046
[2] 2049
[3] 2051
[4] 2051
[5] 2052
[6] 2050
[7] 2050
[8] 2050
[9] 2047
[10] 2049
[11] 2050
[12] 2049

I appreciate any help or suggestion.

You don\'t provide any info on relative time of the samples. We did some tests on an STM32 ADC using RC ramps and found the signal to be a very good fit to a log curve, so low linearity errors. However there was a small amount of noise in the ADC output which was withing the data sheet spec of 4 or 5 lsb counts depending on the mode. Which chip are you using?

I\'m thinking this amount of noise you are seeing is not enough to disturb most calculations, but I don\'t know what you are using the data for. Not may ADCs are good for every bit of resolution. There are often linearity and noise issues that trash the lsb or two. What happens if you round your ADC values to 10 bits? Does that make you happier or do you need all 12 bits of resolution?

--

Rick C.

- Get 1,000 miles of free Supercharging
- Tesla referral code - https://ts.la/richard11209

 

[Jasen Betts]

On 2020-09-21, jlarkin@highlandsniptechnology.com <jlarkin@highlandsniptechnology.com> wrote:

On Sun, 20 Sep 2020 16:29:30 -0700 (PDT), whit3rd <whit3rd@gmail.com
wrote:

On Sunday, September 20, 2020 at 2:01:05 PM UTC-7, jla...@highlandsniptechnology.com wrote:
On Sun, 20 Sep 2020 13:33:11 -0700 (PDT), whit3rd <whi...@gmail.com
wrote:

On Sunday, September 20, 2020 at 3:26:13 AM UTC-7, jla...@highlandsniptechnology.com wrote:
On Sun, 20 Sep 2020 00:26:47 -0700 (PDT), whit3rd <whi...@gmail.com
wrote:
On Saturday, September 19, 2020 at 9:49:47 PM UTC-7, jla...@highlandsniptechnology.com wrote:

Just average the samples and subtract that average from each new
sample. There are several ways to do that average:

Sum the last N samples and divide by N.

that\'s a FIR filter (finite impulse response)
... if you choose the sample size and know the likely interference sources
(like, 60 Hz ripple), it allows you to place a null appropriately

Exponential smoothing: Avg = Avg + (new-Avg) / N

that\'s a IIR filter (infinite impulse response); usually not a great choice

Why not? I see a lot of irrational prejuduce against simple IIR
filters, in code and in FPGAs. Some people would rather write a
hundred lines of code instead of one.

Oh, it\'s simple, all right, but it has a long startup transient.
Any lowpass filter or averager does. Just poke a starting value into
the integrator node if you\'re in a hurry, ADC midscale in this case.

A lowpass needn\'t be considered appropriate during startup (and
brute-force setting a starting value helps). FIR has a time-limit
on its history, which is often completely appropriate and useful.


That means it doesn\'t deal with lightning-strike artifacts well, either.
Presumably an ADC rails on a huge transient. Why would an IIR filter
be worse than a FIR for a spike?

Small signal in big digitizer range, of course. Your \'rails\' scenario is
a measurement failure, and there\'s multiple ways to treat such a thing,
which FIR does by... ignoring the spike a few samples afterward.
IIR doesn\'t do that, so saturating the digitizer is an alternate solution that
you don\'t seem to dislike.

It\'s impressive...

I\'m pleased that my response impresses you.

how many convoluted arguments people make to avoid IIR
digital filters. Most of them reduce to \"It\'s too simple and I don\'t
like it.\"

But not any that I mentioned; what ARE those other \"convoluted arguments\"?
I\'d like to judge their merits for myself...


I was just thinking how crazy it woud be to use, say, a 5000-tap FIR
filter to compute a good autozero average value of the last 5000
samples.

Those 5000 multiply-by-1-and-add blocks will need a lot of logic.

Clearly not the best way to make a boxcar filter...
LOL. I see what you did there.

--
Jasen.

 

[whit3rd]

On Sunday, September 20, 2020 at 6:37:42 PM UTC-7, jla...@highlandsniptechnology.com wrote:

I was just thinking how crazy it woud be to use, say, a 5000-tap FIR
filter to compute a good autozero average value of the last 5000
samples.

Those 5000 multiply-by-1-and-add blocks will need a lot of logic.

5000 words of circular buffer RAM and one adder, one subtractor, needs less.
A resistor and capacitor needs less still.
That\'s electronic design for you.

 

[Tauno Voipio]

On 20.9.20 6.38, Gold_Spark wrote:

I\'m using a STM32 Cortex M0+ to read an AC signal from a CT. I\'m sampling at 6kHz and storing 400 samples. The signal has a DC bias equal to Vcc/2 = 1.65V. In the digital domain this is 2048. In hardware this DC value is very precise, but when sampling it, it varies from 2044 to 2052 inside the buffer. Now if I want to do RMS in that set of data, I need to find a way to deal with this DC bias variation.

I have been thinking the following:

1- Subtract a fixed value of 2048 from each ADC reading. This is no so good as I said above this value may vary slightly. Also, if I want to read zero cross it may cause errors to choose exactly 2048 as reference.
2- Sample the DC bias and average it.
3- Don\'t remove the DC bias. If I calculate RMS then I would get the DC value when there\'s no input signal.
4- Use a more sophisticated software high pass filter?

Example of DC bias readings:

ADC_buffer
[0] 2048
[1] 2046
[2] 2049
[3] 2051
[4] 2051
[5] 2052
[6] 2050
[7] 2050
[8] 2050
[9] 2047
[10] 2049
[11] 2050
[12] 2049

I appreciate any help or suggestion.

The DSP book by Rick Lyons (Understanding digital signal processing,
ISBN 0-13-70241-9) has a good handling of DC removal in part 13.23,
pages 761-765 in my copy.

--

-TV

 

[Ricketty C]

On Monday, September 21, 2020 at 12:30:52 AM UTC-4, Jasen Betts wrote:

On 2020-09-21, jlarkin@highlandsniptechnology.com <jlarkin@highlandsniptechnology.com> wrote:
On Sun, 20 Sep 2020 16:29:30 -0700 (PDT), whit3rd <whit3rd@gmail.com
wrote:

On Sunday, September 20, 2020 at 2:01:05 PM UTC-7, jla...@highlandsniptechnology.com wrote:
On Sun, 20 Sep 2020 13:33:11 -0700 (PDT), whit3rd <whi...@gmail.com
wrote:

On Sunday, September 20, 2020 at 3:26:13 AM UTC-7, jla...@highlandsniptechnology.com wrote:
On Sun, 20 Sep 2020 00:26:47 -0700 (PDT), whit3rd <whi...@gmail.com
wrote:
On Saturday, September 19, 2020 at 9:49:47 PM UTC-7, jla...@highlandsniptechnology.com wrote:

Just average the samples and subtract that average from each new
sample. There are several ways to do that average:

Sum the last N samples and divide by N.

that\'s a FIR filter (finite impulse response)
... if you choose the sample size and know the likely interference sources
(like, 60 Hz ripple), it allows you to place a null appropriately

Exponential smoothing: Avg = Avg + (new-Avg) / N

that\'s a IIR filter (infinite impulse response); usually not a great choice

Why not? I see a lot of irrational prejuduce against simple IIR
filters, in code and in FPGAs. Some people would rather write a
hundred lines of code instead of one.

Oh, it\'s simple, all right, but it has a long startup transient.
Any lowpass filter or averager does. Just poke a starting value into
the integrator node if you\'re in a hurry, ADC midscale in this case.

A lowpass needn\'t be considered appropriate during startup (and
brute-force setting a starting value helps). FIR has a time-limit
on its history, which is often completely appropriate and useful.


That means it doesn\'t deal with lightning-strike artifacts well, either.
Presumably an ADC rails on a huge transient. Why would an IIR filter
be worse than a FIR for a spike?

Small signal in big digitizer range, of course. Your \'rails\' scenario is
a measurement failure, and there\'s multiple ways to treat such a thing,
which FIR does by... ignoring the spike a few samples afterward.
IIR doesn\'t do that, so saturating the digitizer is an alternate solution that
you don\'t seem to dislike.

It\'s impressive...

I\'m pleased that my response impresses you.

how many convoluted arguments people make to avoid IIR
digital filters. Most of them reduce to \"It\'s too simple and I don\'t
like it.\"

But not any that I mentioned; what ARE those other \"convoluted arguments\"?
I\'d like to judge their merits for myself...


I was just thinking how crazy it woud be to use, say, a 5000-tap FIR
filter to compute a good autozero average value of the last 5000
samples.

Those 5000 multiply-by-1-and-add blocks will need a lot of logic.

Clearly not the best way to make a boxcar filter...
LOL. I see what you did there.

--
Jasen.

You mean to construct a straw man design and then shoot it down? An IIR filter is nothing like a 5000 tap boxcar filter.

One point no one has talked about is the sensitivity of various filters to the artifacts caused by frequency content that is not a multiple of the sample rate. If you don\'t get an integer number of wave cycles in a filter time window, the result is artifacts that will mess up an average calculation..

With a FIR filter the coefficients can be tailored to mitigate this effect, but this isn\'t true of a simple IIR filter.

I\'m not at all clear any of this needs to be done. I expect the variations seen on individual measurements are just \"noise\" in the ADC readings with virtually no impact on the average.

--

Rick C.

+ Get 1,000 miles of free Supercharging
+ Tesla referral code - https://ts.la/richard11209

 

[Jasen Betts]

On 2020-09-21, Ricketty C <gnuarm.deletethisbit@gmail.com> wrote:

On Monday, September 21, 2020 at 12:30:52 AM UTC-4, Jasen Betts wrote:
On 2020-09-21, jlarkin@highlandsniptechnology.com <jlarkin@highlandsniptechnology.com> wrote:
On Sun, 20 Sep 2020 16:29:30 -0700 (PDT), whit3rd <whit3rd@gmail.com
wrote:

But not any that I mentioned; what ARE those other \"convoluted arguments\"?
I\'d like to judge their merits for myself...


I was just thinking how crazy it woud be to use, say, a 5000-tap FIR
filter to compute a good autozero average value of the last 5000
samples.

Those 5000 multiply-by-1-and-add blocks will need a lot of logic.

Clearly not the best way to make a boxcar filter...
LOL. I see what you did there.


You mean to construct a straw man design and then shoot it down?

No, I\'ll let you know tomorrow, I don\'t want to spoil it for others.


--
Jasen.

 

[Kevin Aylward]

\"Ricketty C\" wrote in message
news:97ab1d7f-07cf-4caa-9faa-1e66f2b86b4eo@googlegroups.com...

On Saturday, September 19, 2020 at 11:38:09 PM UTC-4, Gold_Spark wrote:
I\'m using a STM32 Cortex M0+ to read an AC signal from a CT. I\'m sampling
at 6kHz and storing 400 samples. The signal has a DC bias equal to Vcc/2 =
1.65V. In the digital domain this is 2048. In hardware this DC value is
very precise, but when sampling it, it varies from 2044 to 2052 inside the
buffer. Now >if I want to do RMS in that set of data, I need to find a way
to deal with this DC bias variation.

I have been thinking the following:

1- Subtract a fixed value of 2048 from each ADC reading. This is no so
good as I said above this value may vary slightly. Also, if I want to
read zero cross it >>may cause errors to choose exactly 2048 as
reference.

4- Use a more sophisticated software high pass filter?

Funny that people propose you use a low pass filter and subtract. A high
pass would just provide the AC without the DC. It would take some time to
start >up, but can work very effectively. If this sampling is continuous
that would work fine.

Yeah... seems like the twilight zone here. All this FIR, averaging and
$hit... seems to be... the plot is lost....

From the description the poster is measurement an AC signal, thus stick a
cap on the input to block the DC and you\'re done....



-- Kevin Aylward
http://www.anasoft.co.uk - SuperSpice
http://www.kevinaylward.co.uk/ee/index.html

 

[server]

On Monday, 21 September 2020 19:21:44 UTC+1, Kevin Aylward wrote:

\"Ricketty C\" wrote in message
news:97ab1d7f-07cf-4caa-9faa-1e66f2b86b4eo@googlegroups.com...

On Saturday, September 19, 2020 at 11:38:09 PM UTC-4, Gold_Spark wrote:
I\'m using a STM32 Cortex M0+ to read an AC signal from a CT. I\'m sampling
at 6kHz and storing 400 samples. The signal has a DC bias equal to Vcc/2 =
1.65V. In the digital domain this is 2048. In hardware this DC value is
very precise, but when sampling it, it varies from 2044 to 2052 inside the
buffer. Now >if I want to do RMS in that set of data, I need to find a way
to deal with this DC bias variation.

I have been thinking the following:

1- Subtract a fixed value of 2048 from each ADC reading. This is no so
good as I said above this value may vary slightly. Also, if I want to
read zero cross it >>may cause errors to choose exactly 2048 as
reference.

4- Use a more sophisticated software high pass filter?

Funny that people propose you use a low pass filter and subtract. A high
pass would just provide the AC without the DC. It would take some time to
start >up, but can work very effectively. If this sampling is continuous
that would work fine.

Yeah... seems like the twilight zone here. All this FIR, averaging and
$hit... seems to be... the plot is lost....

From the description the poster is measurement an AC signal, thus stick a
cap on the input to block the DC and you\'re done....

There is no dc to remove from the input. Its a current transformer.
The problem is that the ADC needs to be biased at the midpoint of
the positive-only supply so that it can digitise positive and negative
outputs from the CT. The converted values of that midpoint bias are
not completely stable and need to be removed before the rms calculation, otherwise small alternating signals are swamped by the offset error.

John

 

[John Larkin]

On Mon, 21 Sep 2020 15:36:26 -0700 (PDT), jrwalliker@gmail.com wrote:

On Monday, 21 September 2020 19:21:44 UTC+1, Kevin Aylward wrote:
\"Ricketty C\" wrote in message
news:97ab1d7f-07cf-4caa-9faa-1e66f2b86b4eo@googlegroups.com...

On Saturday, September 19, 2020 at 11:38:09 PM UTC-4, Gold_Spark wrote:
I\'m using a STM32 Cortex M0+ to read an AC signal from a CT. I\'m sampling
at 6kHz and storing 400 samples. The signal has a DC bias equal to Vcc/2 =
1.65V. In the digital domain this is 2048. In hardware this DC value is
very precise, but when sampling it, it varies from 2044 to 2052 inside the
buffer. Now >if I want to do RMS in that set of data, I need to find a way
to deal with this DC bias variation.

I have been thinking the following:

1- Subtract a fixed value of 2048 from each ADC reading. This is no so
good as I said above this value may vary slightly. Also, if I want to
read zero cross it >>may cause errors to choose exactly 2048 as
reference.

4- Use a more sophisticated software high pass filter?

Funny that people propose you use a low pass filter and subtract. A high
pass would just provide the AC without the DC. It would take some time to
start >up, but can work very effectively. If this sampling is continuous
that would work fine.

Yeah... seems like the twilight zone here. All this FIR, averaging and
$hit... seems to be... the plot is lost....

From the description the poster is measurement an AC signal, thus stick a
cap on the input to block the DC and you\'re done....

There is no dc to remove from the input. Its a current transformer.
The problem is that the ADC needs to be biased at the midpoint of
the positive-only supply so that it can digitise positive and negative
outputs from the CT. The converted values of that midpoint bias are
not completely stable and need to be removed before the rms calculation, otherwise small alternating signals are swamped by the offset error.

John

Exactly. You can auto-zero to a fraction of an ADC LSB. Then your RMS
measurement is limited by ADC quantization and linearity. ADC offset
usually drifts slowly, so the autozero can use tons of samples and
need not be especially fast. Averaging the last 65536 samples at your
6K rate would work fine. You\'d get a new az value every 10 seconds.

I did software az in my electric meters. And added a little noise
dither to the ADC front end. That added a little baseline offset to
the reported RMS currents, but vastly improved low-level power
measurement. It\'s actually hard to design electronics that\'s as good
as an old disk-type meter. The only spec I really beat them on was
tilt.

There\'s a trick to adding dither without increasing the apparent RMS
floor much. The nuclear guys do that in pulse-height spectroscopy.

 

[Phil Hobbs]

On 2020-09-20 17:00, jlarkin@highlandsniptechnology.com wrote:

On Sun, 20 Sep 2020 13:33:11 -0700 (PDT), whit3rd <whit3rd@gmail.com
wrote:

On Sunday, September 20, 2020 at 3:26:13 AM UTC-7, jla...@highlandsniptechnology.com wrote:
On Sun, 20 Sep 2020 00:26:47 -0700 (PDT), whit3rd <whi...@gmail.com
wrote:
On Saturday, September 19, 2020 at 9:49:47 PM UTC-7, jla...@highlandsniptechnology.com wrote:

Just average the samples and subtract that average from each new
sample. There are several ways to do that average:

Sum the last N samples and divide by N.

that\'s a FIR filter (finite impulse response)
... if you choose the sample size and know the likely interference sources
(like, 60 Hz ripple), it allows you to place a null appropriately

Exponential smoothing: Avg = Avg + (new-Avg) / N

that\'s a IIR filter (infinite impulse response); usually not a great choice

Why not? I see a lot of irrational prejuduce against simple IIR
filters, in code and in FPGAs. Some people would rather write a
hundred lines of code instead of one.

Oh, it\'s simple, all right, but it has a long startup transient.

Any lowpass filter or averager does. Just poke a starting value into
the integrator node if you\'re in a hurry, ADC midscale in this case.
Or go for 2nd order. An auto-zero system works fine with a droopy 1st
order filter.

You have to wait for an FIR filter to fill up, and the output is
nonsense until it does. You can poke a start value into it, but you
have to load all the nodes.

That means it doesn\'t deal with lightning-strike artifacts well, either.

Presumably an ADC rails on a huge transient. Why would an IIR filter
be worse than a FIR for a spike? If you hate long tails, go 2nd order.

In cases (like, reading an analog tape) where you want to reject a
particular frequency (the recording bias generator), it doesn\'t have enough
flexibility. FIR was eventually the big winner in CD playback, for instance.

It\'s impressive how many convoluted arguments people make to avoid IIR
digital filters. Most of them reduce to \"It\'s too simple and I don\'t
like it.\"

Fancy IIRs can have bad behaviour such as limit cycles. Slow 1-pole IIR
lowpasses can run out of precision if poorly designed, on account of the
LSBs of the new values getting lost in shifting.

Thought is required.

20 years or so back, I was building low-resolution, high sensitivity
pyroelectric infrared sensors using a PIC17 micro with an capacious 902
bytes of RAM including its registers. That worked out to 9.4 bytes per
pixel.

Its temporal response was intentionally made very slow for SNR
reasons--insulating the pixels with air slowed them down, but the
slowdown was due to bass boost rather than treble cut, so to speak, so
the SNR was better at all frequencies. One wrinkle was that the
accumulated charge got dumped on every measurement, so the raw data was
a first finite difference of a slow continuous-time function.

It would have needed a gigundo long FIR filter to fix it, but it turned
out that the desired pseudo-inverse function could be factored into a
3-sample FIR filter followed by a running sum.

(I still did the filtering on the PC side, but it could probably just
have been done on the micro.)

Cheers

Phil Hobbs

--
Dr Philip C D Hobbs
Principal Consultant
ElectroOptical Innovations LLC / Hobbs ElectroOptics
Optics, Electro-optics, Photonics, Analog Electronics
Briarcliff Manor NY 10510

http://electrooptical.net
http://hobbs-eo.com