Noise Is 3 Orders of Magnitude Greater Than A Wave Form

[Bret Cahill]

Everything is known about the transmitted wave, i.e., the shape &
phase angle, except the amplitude.

All that is necessary is to recover is the amplitude of the wave. Can
this be done when the noise is several orders of magnitude greater
than the signal?


Bret Cahill

 

[doug]

Bret Cahill wrote:

Everything is known about the transmitted wave, i.e., the shape &
phase angle, except the amplitude.

All that is necessary is to recover is the amplitude of the wave. Can
this be done when the noise is several orders of magnitude greater
than the signal?


Bret Cahill


Narrowing the final detection bandwidth is the only hope.

If the noise spectrum is white, narrow bandwidth through
averaging works. If you cannot average, there is trouble.
The old lockin amplifiers used a modulation or chopping signal
and a long time constant in the final filter. I had signals
at times that required an hour of integration to detect. It
was slow but it worked.

 

[Bret Cahill]

Everything is known about the transmitted wave, i.e., the shape &
phase angle, except the amplitude.

All that is necessary is to recover is the amplitude of the wave.  Can
this be done when the noise is several orders of magnitude greater
than the signal?

Bret Cahill

Narrowing the final detection bandwidth is the only hope.
If the noise spectrum is white, narrow bandwidth through
averaging works. If you cannot average, there is trouble.
The old lockin amplifiers used a modulation or chopping signal
and a long time constant in the final filter.  I had signals
at times that required an hour of integration to detect. It
was slow but it worked.

I forgot to mention that the signal wave form can be anything that can
be generated.

For example, if something similar to an AM radio signal, say, f(t) (sint)(sin10t)(sin10t) was possible and the frequency of the noise was
about the same as the sin(t) factor, then f(t) will plot the noise
every time f(t) = 0.

In this case that would be ten times as often as the sin(t) factor.

The noise can then be subtracted to recover the wave form.


Bret Cahill

 

[John Larkin]

On Thu, 2 Jul 2009 20:10:41 -0700 (PDT), Bret Cahill
<BretCahill@aol.com> wrote:

Everything is known about the transmitted wave, i.e., the shape &
phase angle, except the amplitude.

All that is necessary is to recover is the amplitude of the wave.  Can
this be done when the noise is several orders of magnitude greater
than the signal?

Bret Cahill

Narrowing the final detection bandwidth is the only hope.
If the noise spectrum is white, narrow bandwidth through
averaging works. If you cannot average, there is trouble.
The old lockin amplifiers used a modulation or chopping signal
and a long time constant in the final filter.  I had signals
at times that required an hour of integration to detect. It
was slow but it worked.


I forgot to mention that the signal wave form can be anything that can
be generated.

For example, if something similar to an AM radio signal, say, f(t) =
(sint)(sin10t)(sin10t) was possible and the frequency of the noise was
about the same as the sin(t) factor, then f(t) will plot the noise
every time f(t) = 0.

In this case that would be ten times as often as the sin(t) factor.

The noise can then be subtracted to recover the wave form.


Bret Cahill

I think what's happening is that you have

1. Defined the noise to be bandlimited to radian frequencies below
about 1

and

2. Up-converted (modulated) the signal to have usable (recoverable)
components around 10 times that frequency.

So (am I allowed to say 'duh'?) a simple highpass filtering operation
will remove the noise from the upconverted signal. You moved the
signal to where you knew there was no noise.

It's more interesting when the signal and the noise occupy the same
bandwidth.

John

 

[HardySpicer]

On Jul 2, 6:28 pm, Bret Cahill <BretCah...@aol.com> wrote:

Everything is known about the transmitted wave, i.e., the shape &
phase angle, except the amplitude.

All that is necessary is to recover is the amplitude of the wave.  Can
this be done when the noise is several orders of magnitude greater
than the signal?

Bret Cahill

ask comp.dsp

 

[Bob Masta]

On Thu, 2 Jul 2009 18:28:26 -0700 (PDT), Bret
Cahill <BretCahill@aol.com> wrote:

Everything is known about the transmitted wave, i.e., the shape &
phase angle, except the amplitude.

All that is necessary is to recover is the amplitude of the wave. Can
this be done when the noise is several orders of magnitude greater
than the signal?

In principle, this can be done with synchronous

averaging. I have a tutorial series starting
here:
<http://www.daqarta.com/tm01.htm>

The basic idea is that you must be able to sample
the signal synchronously with the sourve
(transmitted) wave. That means you need some sort
of trigger signal derived directly from the
source. On each trigger, you acquire some number
of samples, a long enough series for your needs.
Here you will want the series long enough to
encompass at least one waveform cycle, since you
are looking for amplitude of the overall wave.

Let's say that you acquire 1024 samples per
trigger. (It's generally OK to miss triggers, as
long as you always start acquisition on a
trigger.) Then you add those 1024 samples, one by
one, into a 1024-bin accumulator. The accumulator
will end up holding the average value of the
waveform, assuming that the waveform is constant
and the noise is not synchronous.

For every doubling of the number of samples you
add into the accumulator, the S/N improves by 3
dB. The overall improvement is thus determined by
how long you want to wait to accumulate enough
samples.

This is the technique used to monitor "evoked
potentials" in the brain. For example, the
subject is presented with a series of repeating
tone bursts of a given frequency, while scalp
electrodes monitor brain activity. The source of
the tone bursts is also the trigger for the
averager. The brain response to any given burst
is hopelessly buried in noise, since the scalp
electrodes see all the brain activity, not just
the auditory part. Only a tiny part of the total
is due to the auditory response, but that part is
in synchrony with the stimulus, while the rest of
the brain in general is not. So after several
thousand tone bursts, you can "see" the auditory
response. This is used to test hearing in lab
animals and infants who can't report what they
hear.

Best regards,


Bob Masta

DAQARTA v4.51
Data AcQuisition And Real-Time Analysis
www.daqarta.com
Scope, Spectrum, Spectrogram, Sound Level Meter
FREE Signal Generator
Science with your sound card!

 

[Bret Cahill]

Everything is known about the transmitted wave, i.e., the shape &
phase angle, except the amplitude.

All that is necessary is to recover is the amplitude of the wave.  Can
this be done when the noise is several orders of magnitude greater
than the signal?

Bret Cahill

Narrowing the final detection bandwidth is the only hope.
If the noise spectrum is white, narrow bandwidth through
averaging works. If you cannot average, there is trouble.
The old lockin amplifiers used a modulation or chopping signal
and a long time constant in the final filter.  I had signals
at times that required an hour of integration to detect. It
was slow but it worked.

I forgot to mention that the signal wave form can be anything that can
be generated.

For example, if something similar to an AM radio signal, say, f(t) > >(sint)(sin10t)(sin10t) was possible and the frequency of the noise was
about the same as the sin(t) factor, then f(t) will plot the noise
every time f(t) = 0.

In this case that would be ten times as often as the sin(t) factor.

The noise can then be subtracted to recover the wave form.

Bret Cahill

I think what's happening is that you have

1. Defined the noise to be bandlimited to radian frequencies below
about 1

The noise and the signal are both low and have about the same
frequency, ~ x/2pi. There is little that can be done to change this
situation. There is no time to wait more than several cycles for the
result either.

and

2. Up-converted (modulated) the signal to have usable (recoverable)
components around 10 times that frequency.

To plot the noise. Every time sin 10t = 0, f(t) = zero, and the only
thing left is the low frequency noise.

Then a high pass filter can smooth out the (sin10t)(sin10t) component
so something like the original signal can still be recovered after the
noise is subtracted out.

Alternatively traditional filters can be eliminated altogether. The
signal + noise as well as the noise alone can be traced out from the
high frequency signal.

The noise is then subtracted from the signal + noise to recover the
signal.

So (am I allowed to say 'duh'?) a simple highpass filtering operation
will remove the noise from the upconverted signal. You moved the
signal to where you knew there was no noise.

Plot f(t) and it's easy to see the original signal sill exists,
although in a somewhat discontinuous form.

It would be interesting if this has never been done before.


Bret Cahill

 

[Bret Cahill]

Everything is known about the transmitted wave, i.e., the shape &
phase angle, except the amplitude.

All that is necessary is to recover is the amplitude of the wave.  Can
this be done when the noise is several orders of magnitude greater
than the signal?

In principle, this can be done with synchronous
averaging.  I have a tutorial series starting
here:
http://www.daqarta.com/tm01.htm

The basic idea is that you must be able to sample
the signal synchronously with the sourve
(transmitted) wave.  That means you need some sort
of trigger signal derived directly from the
source.  

That was the plan. Complete control over and knowledge of the source
wave should provide some options to recover the signal, even if the
noise is orders of magnitude more than than the source wave.

Kind of like the military shooting down a tank of hydrazine with a
transponder on it and then claiming they took out a satellite.

On each trigger, you acquire some number
of samples, a long enough series for your needs.
Here you will want the series long enough to
encompass at least one waveform cycle, since you
are looking for amplitude of the overall wave.

The system will attenuate the low frequency wave more than a high
frequency wave that isn't part of the low frequency wave.

Determining the amount of attenuation of the low frequency wave was,
in fact, the goal.

The high frequency component, however, should attenuate much like the
low frequency wave if it is tracing out the low frequency wave.

Let's say that you acquire 1024 samples per
trigger.  (It's generally OK to miss triggers, as
long as you always start acquisition on a
trigger.)  Then you add those 1024 samples, one by
one, into a 1024-bin accumulator.  The accumulator
will end up holding the average value of the
waveform, assuming that the waveform is constant
and the noise is not synchronous.

For every doubling of the number of samples you
add into the accumulator, the S/N improves by 3
dB.  The overall improvement is thus determined by
how long you want to wait to accumulate enough
samples.

Maybe 5 - 7 cycles at most.

At first I thought a more sophisticated version of resonant frequency,
i.e., lots of different frequency waves added together, might have
been a solution but there wasn't enough time to get past the transient
effects.

This is the technique used to monitor "evoked
potentials" in the brain.  For example, the
subject is presented with a series of repeating
tone bursts of a given frequency, while scalp
electrodes monitor brain activity.  The source of
the tone bursts is also the trigger for the
averager.  The brain response to any given burst
is hopelessly buried in noise, since the scalp
electrodes see all the brain activity, not just
the auditory part.  Only a tiny part of the total
is due to the auditory response, but that part is
in synchrony with the stimulus, while the rest of
the brain in general is not. So after several
thousand tone bursts, you can "see" the auditory
response.  This is used to test hearing in lab
animals and infants who can't report what they
hear.

Best regards,

Bob Masta

              DAQARTA  v4.51
   Data AcQuisition And Real-Time Analysis
             www.daqarta.com
Scope, Spectrum, Spectrogram, Sound Level Meter
           FREE Signal Generator
        Science with your sound card!

 

[doug]

Bret Cahill wrote:

http://www.daqarta.com/tm01.htm


I'm not saying your approach won't work in my case but your Fig. 1
isn't exactly my situation.

It is the same situation just the spectrum of the noise is different.
In all of this, the game is the same because the physics is the
same. To get a signal to noise ratio or signal to interference ratio
larger than one, you need a bandwidth where the noise is smaller
than the signal. Averaging, which is applying a narrow bandpass
filter helps most for random noise.

My noise + signal, if you could see which was which, would look like a
large smooth curve with a much smaller amplitude sin curve with a
similar period superimposed just above it and/or just below it.

Just how similar a period? The closer the two are, the harder your
job is.

My solution was to multiply a high frequency _always positive_ wave
onto the original signal.

Life might be simpler if you just multiplied your signal by a
square wave whose value is 0 or 1.

The new signal is discontinuous but it still looks and acts a lot like
the original.

The difference is every time the high frequency wave was zero, the
entire function would be zero and only the noise would remain. The
noise is smooth so it could be accurately determined by numerical
regression even if the high frequency signal wasn't all that high.

Once the noise is known it can be subtracted from the entire output
from the receiver.

What you are describing is not noise but interference.

The noise needs to be known to at least 4 decimal place accuracy.

That is a tall order.

Bret Cahill

 

[Bret Cahill]

http://www.daqarta.com/tm01.htm

I'm not saying your approach won't work in my case but your Fig. 1
isn't exactly my situation.

My noise + signal, if you could see which was which, would look like a
large smooth curve with a much smaller amplitude sin curve with a
similar period superimposed just above it and/or just below it.

My solution was to multiply a high frequency _always positive_ wave
onto the original signal.

The new signal is discontinuous but it still looks and acts a lot like
the original.

The difference is every time the high frequency wave was zero, the
entire function would be zero and only the noise would remain. The
noise is smooth so it could be accurately determined by numerical
regression even if the high frequency signal wasn't all that high.

Once the noise is known it can be subtracted from the entire output
from the receiver.

The noise needs to be known to at least 4 decimal place accuracy.


Bret Cahill

 

[John Larkin]

On Fri, 3 Jul 2009 08:04:15 -0700 (PDT), Bret Cahill
<BretCahill@aol.com> wrote:

Everything is known about the transmitted wave, i.e., the shape &
phase angle, except the amplitude.

All that is necessary is to recover is the amplitude of the wave.  Can
this be done when the noise is several orders of magnitude greater
than the signal?

Bret Cahill

Narrowing the final detection bandwidth is the only hope.
If the noise spectrum is white, narrow bandwidth through
averaging works. If you cannot average, there is trouble.
The old lockin amplifiers used a modulation or chopping signal
and a long time constant in the final filter.  I had signals
at times that required an hour of integration to detect. It
was slow but it worked.

I forgot to mention that the signal wave form can be anything that can
be generated.

For example, if something similar to an AM radio signal, say, f(t) =
(sint)(sin10t)(sin10t) was possible and the frequency of the noise was
about the same as the sin(t) factor, then f(t) will plot the noise
every time f(t) = 0.

In this case that would be ten times as often as the sin(t) factor.

The noise can then be subtracted to recover the wave form.

Bret Cahill

I think what's happening is that you have

1. Defined the noise to be bandlimited to radian frequencies below
about 1

The noise and the signal are both low and have about the same
frequency, ~ x/2pi. There is little that can be done to change this
situation. There is no time to wait more than several cycles for the
result either.

and

2. Up-converted (modulated) the signal to have usable (recoverable)
components around 10 times that frequency.

To plot the noise. Every time sin 10t = 0, f(t) = zero, and the only
thing left is the low frequency noise.

You are assuming the ability to high-frequency modulate the signal
before the noise is added to it. So you already know what the
noiseless signal looks like.

John

 

[Bret Cahill]

http://www.daqarta.com/tm01.htm

I'm not saying your approach won't work in my case but your Fig. 1
isn't exactly my situation.

It is the same situation just the spectrum of the noise is different.

Which is everything when you want to reduce noise.

In all of this, the game is the same because the physics is the
same.

The math is entirely different. A smooth low frequency noise curve
can be sampled fewer times over longer intervals and the result can be
very accurate, especially with regression.

To get a signal to noise ratio or signal to interference ratio
larger than one, you need a bandwidth where the noise is smaller
than the signal.  

Convert the signal to a higher frequency wave form that still has
characteristics of the original signal.

Averaging, which is applying a narrow bandpass
filter helps most for random noise.

Time = money. If the average takes more than a few cycles, the result
won't be any good for other reasons.

My noise + signal, if you could see which was which, would look like a
large smooth curve with a much smaller amplitude sin curve with a
similar period superimposed just above it and/or just below it.

Just how similar a period?

Maybe 0.5 - 1.5.

Say the sensor is getting something like 100sin(1.3t) [the noise] +
sint [the signal].

I need to know the signal to 0.25% accuracy.

The closer the two are, the harder your
job is.

Total control the signal should be worth _something_.

My solution was to multiply a high frequency _always positive_ wave
onto the original signal.

Life might be simpler if you just multiplied your signal by a
square wave whose value is 0 or 1.

What's the advantage?

The new signal is discontinuous but it still looks and acts a lot like
the original.

The difference is every time the high frequency wave was zero, the
entire function would be zero and only the noise would remain.  The
noise is smooth so it could be accurately determined by numerical
regression even if the high frequency signal wasn't all that high.

Once the noise is known it can be subtracted from the entire output
from the receiver.

What you are describing is not noise but interference.

The noise needs to be known to at least 4 decimal place accuracy.

That is a tall order.

If it's not possible there's a completely different approach.


Bret Cahill

 

[doug]

Bret Cahill wrote:

http://www.daqarta.com/tm01.htm


I'm not saying your approach won't work in my case but your Fig. 1
isn't exactly my situation.


It is the same situation just the spectrum of the noise is different.


Which is everything when you want to reduce noise.

No, the same considerations apply. You need a bandwidth where
the ratio of signal to noise meets your requirement.

In all of this, the game is the same because the physics is the
same.


The math is entirely different. A smooth low frequency noise curve
can be sampled fewer times over longer intervals and the result can be
very accurate, especially with regression.

No, what you are claiming above is for noise that has a vastly
different spectral content than the signal.

To get a signal to noise ratio or signal to interference ratio
larger than one, you need a bandwidth where the noise is smaller
than the signal.


Convert the signal to a higher frequency wave form that still has
characteristics of the original signal.

That has no effect on the SNR since the noise gets converted too.

Averaging, which is applying a narrow bandpass
filter helps most for random noise.


Time = money. If the average takes more than a few cycles, the result
won't be any good for other reasons.

If the signal is really one thousandth of the noise, and the noise is
random, you need to average a million traces to get an SNR of one. This
is because the SNR increases as the square root of the number of traces
averaged.

My noise + signal, if you could see which was which, would look like a
large smooth curve with a much smaller amplitude sin curve with a
similar period superimposed just above it and/or just below it.


Just how similar a period?


Maybe 0.5 - 1.5.

Say the sensor is getting something like 100sin(1.3t) [the noise] +
sint [the signal].

I need to know the signal to 0.25% accuracy.

So you want a bandpass filter at frequency t which has skirts down
120db at 1.3t (assuming the noise signal is narrow band and that
it is 1000 times the signal amplitude originally).

The closer the two are, the harder your
job is.


Total control the signal should be worth _something_.

Knowing the frequency and phase is already factored in the discussion.

My solution was to multiply a high frequency _always positive_ wave
onto the original signal.


Life might be simpler if you just multiplied your signal by a
square wave whose value is 0 or 1.


What's the advantage?

You get to look at the signal 50% of the time and the noise 50% of

the time to get their relative amplitudes.

The new signal is discontinuous but it still looks and acts a lot like
the original.


The difference is every time the high frequency wave was zero, the
entire function would be zero and only the noise would remain. The
noise is smooth so it could be accurately determined by numerical
regression even if the high frequency signal wasn't all that high.


Once the noise is known it can be subtracted from the entire output
from the receiver.


What you are describing is not noise but interference.


The noise needs to be known to at least 4 decimal place accuracy.


That is a tall order.


If it's not possible there's a completely different approach.

I would go after that.

Bret Cahill

 

[doug]

Bret Cahill wrote:

The sensor receives the loud noise, say, 10sin(0.7x), plus the small
signal, say, sin(x ). The signal, however, can be transformed into sin
(x)sin^2(100x) so the sensor receives

(10sin(0.7x) + sin(x)sin^2(100x)) x from 2 to 3.4

What you are doing is changing the problem. You are now looking at a
sin(100x) modulated by sin(x). This says the noise and the signal
are independent of one another. If you can do this, make it 1000x
and make you life easier.

This can be viewed by pasting the entire line into www.wolframalpha.com

The blue area is the signal to be extracted from the noise. It's over
the noise to the left of x = pi and under the noise to the right of
pi.

The high frequency curve runs between the signal and the noise and
maps out both curves to any precision depending on frequency and
regression.


Bret Cahill




Everything is known about the transmitted wave, i.e., the shape &
phase angle, except the amplitude.

All that is necessary is to recover is the amplitude of the wave. Can
this be done when the noise is several orders of magnitude greater
than the signal?

Bret Cahill

Narrowing the final detection bandwidth is the only hope.
If the noise spectrum is white, narrow bandwidth through
averaging works. If you cannot average, there is trouble.
The old lockin amplifiers used a modulation or chopping signal
and a long time constant in the final filter. I had signals
at times that required an hour of integration to detect. It
was slow but it worked.

I forgot to mention that the signal wave form can be anything that can
be generated.

For example, if something similar to an AM radio signal, say, f(t) =
(sint)(sin10t)(sin10t) was possible and the frequency of the noise was
about the same as the sin(t) factor, then f(t) will plot the noise
every time f(t) = 0.

In this case that would be ten times as often as the sin(t) factor.

The noise can then be subtracted to recover the wave form.

Bret Cahill- Hide quoted text -

- Show quoted text -

 

[Bret Cahill]

The sensor receives the loud noise, say, 10sin(0.7x), plus the small
signal, say, sin(x ). The signal, however, can be transformed into sin
(x)sin^2(100x) so the sensor receives

(10sin(0.7x) + sin(x)sin^2(100x)) x from 2 to 3.4

This can be viewed by pasting the entire line into www.wolframalpha.com

The blue area is the signal to be extracted from the noise. It's over
the noise to the left of x = pi and under the noise to the right of
pi.

The high frequency curve runs between the signal and the noise and
maps out both curves to any precision depending on frequency and
regression.


Bret Cahill

Everything is known about the transmitted wave, i.e., the shape &
phase angle, except the amplitude.

All that is necessary is to recover is the amplitude of the wave.  Can
this be done when the noise is several orders of magnitude greater
than the signal?

Bret Cahill

Narrowing the final detection bandwidth is the only hope.
If the noise spectrum is white, narrow bandwidth through
averaging works. If you cannot average, there is trouble.
The old lockin amplifiers used a modulation or chopping signal
and a long time constant in the final filter.  I had signals
at times that required an hour of integration to detect. It
was slow but it worked.

I forgot to mention that the signal wave form can be anything that can
be generated.

For example, if something similar to an AM radio signal, say, f(t) > (sint)(sin10t)(sin10t) was possible and the frequency of the noise was
about the same as the sin(t) factor, then f(t) will plot the noise
every time f(t) = 0.

In this case that would be ten times as often as the sin(t) factor.

The noise can then be subtracted to recover the wave form.

Bret Cahill- Hide quoted text -

- Show quoted text -

 

[doug]

Bret Cahill wrote:

The sensor receives the loud noise, say, 10sin(0.7x), plus the small
signal, say, sin(x ). The signal, however, can be transformed into sin
(x)sin^2(100x) so the sensor receives

(10sin(0.7x) + sin(x)sin^2(100x)) x from 2 to 3.4

What you are doing is changing the problem.


As I pointed out above, I forgot to mention in the OP that the
designer could change the signal to suit the problem.


You are now looking at a
sin(100x) modulated by sin(x).


Actually it's a sin^2(100x) which is always positive.

Sin100x will not work as the sign of the original signal must be
preserved.

Sin(100x) will work just fine. To detect sin(x) just multiply
the signal by sin(100x) again or, to be careful multiply by
sin(100x) and also by cos(100x). This does a quadrature detection
and the magnitude of the two term is independent of the
phase relative to sin(100x)

For example, try pasting

(10sin(0.7x) + sin(x)sin(100x)) x from 2 to 3.4

into www.wolframalpha.com

How would you know the noise curve?

The point of using the high frequency is to remove your
signal from the noise value.

This says the noise and the signal
are independent of one another.


Bingo!


If you can do this, make it 1000x
and make you life easier.


In some situations it may be difficult to use very high frequencies.

That's why regression was mentioned.

The physics requirements are always the same. You need to have the
signal power greater than the noise power in the detection bandwidth.
Any fancy processing scheme is just trying to narrow the detection
bandwidth.

Bret Cahill

 

[Bret Cahill]

The sensor receives the loud noise, say, 10sin(0.7x),  plus the small
signal, say, sin(x ).  The signal, however, can be transformed into sin
(x)sin^2(100x) so the sensor receives

(10sin(0.7x) + sin(x)sin^2(100x))       x from 2 to 3.4

What you are doing is changing the problem.

As I pointed out above, I forgot to mention in the OP that the
designer could change the signal to suit the problem.

You are now looking at a
sin(100x) modulated by sin(x).

Actually it's a sin^2(100x) which is always positive.

Sin100x will not work as the sign of the original signal must be
preserved.

For example, try pasting

(10sin(0.7x) + sin(x)sin(100x)) x from 2 to 3.4

into www.wolframalpha.com

How would you know the noise curve?

This says the noise and the signal
are independent of one another.

Bingo!

If you can do this, make it 1000x
and make you life easier.

In some situations it may be difficult to use very high frequencies.

That's why regression was mentioned.


Bret Cahill

 

[Bret Cahill]

Everything is known about the transmitted wave, i.e., the shape &
phase angle, except the amplitude.

All that is necessary is to recover is the amplitude of the wave.  Can
this be done when the noise is several orders of magnitude greater
than the signal?

Bret Cahill

Narrowing the final detection bandwidth is the only hope.
If the noise spectrum is white, narrow bandwidth through
averaging works. If you cannot average, there is trouble.
The old lockin amplifiers used a modulation or chopping signal
and a long time constant in the final filter.  I had signals
at times that required an hour of integration to detect. It
was slow but it worked.

I forgot to mention that the signal wave form can be anything that can
be generated.

For example, if something similar to an AM radio signal, say, f(t) > >> >(sint)(sin10t)(sin10t) was possible and the frequency of the noise was
about the same as the sin(t) factor, then f(t) will plot the noise
every time f(t) = 0.

In this case that would be ten times as often as the sin(t) factor.

The noise can then be subtracted to recover the wave form.

Bret Cahill

I think what's happening is that you have

1. Defined the noise to be bandlimited to radian frequencies below
about 1

The noise and the signal are both low and have about the same
frequency, ~ x/2pi.  There is little that can be done to change this
situation.  There is no time to wait more than several cycles for the
result either.

and

2. Up-converted (modulated) the signal to have usable (recoverable)
components around 10 times that frequency.

To plot the noise.  Every time sin 10t = 0, f(t) = zero, and the only
thing left is the low frequency noise.

You are assuming the ability to high-frequency modulate the signal
before the noise is added to it. So you already know what the
noiseless signal looks like.

As I said in the OP, "Everything is known about the transmitted wave,
i.e., the shape & phase angle, except the amplitude."

This works for the same reason reading a newspaper in a foreign
language is easy. You already know what they are going to say.


Bret Cahill

 

[Bret Cahill]

The sensor receives the loud noise, say, 10sin(0.7x),  plus the small
signal, say, sin(x ).  The signal, however, can be transformed into sin
(x)sin^2(100x) so the sensor receives

(10sin(0.7x) + sin(x)sin^2(100x))       x from 2 to 3.4

What you are doing is changing the problem.

As I pointed out above, I forgot to mention in the OP that the
designer could change the signal to suit the problem.

You are now looking at a
sin(100x) modulated by sin(x).

Actually it's a sin^2(100x) which is always positive.

Sin100x will not work as the sign of the original signal must be
preserved.

Sin(100x) will work just fine.

The shape and the effect of the original signal, sin x, on the rest of
the system cannot be preserved if the sign of the high frequency
factor alternates each cycle.

To detect sin(x) just multiply
the signal by sin(100x) again or, to be careful multiply by
sin(100x) and also by cos(100x). This does a quadrature detection
and the magnitude of the two term is independent of the
phase relative to sin(100x)



For example, try pasting

(10sin(0.7x) + sin(x)sin(100x))       x from 2 to 3.4

intowww.wolframalpha.com

How would you know the noise curve?

The point of using the high frequency is to remove your
signal from the noise value.

And this is done by first identifying the noise curve.

The high frequency curve and, therefore, the entire f(t), will equal 0
on every cycle of the high frequency.

Since we know when the signal is zero we know the value of the noise
at that time.

This says the noise and the signal
are independent of one another.

Bingo!

If you can do this, make it 1000x
and make you life easier.

In some situations it may be difficult to use very high frequencies.

That's why regression was mentioned.

The physics requirements are always the same. You need to have the
signal power greater than the noise power in the detection bandwidth.

Not if you can identify the noise to 99.995% accuracy.

Any fancy processing scheme is just trying to narrow the detection
bandwidth.

I didn't think the approach above was either fancy or "narrowing the
bandwith."

Obviously, if the noise frequency is about the same as signal
frequency it will necessary to end run the use of conventional filters
altogether.

It would be interesting to learn if and when something similar was
tried.


Bret Cahill

 

[John Larkin]

On Fri, 3 Jul 2009 15:30:36 -0700 (PDT), Bret Cahill
<BretCahill@aol.com> wrote:

Everything is known about the transmitted wave, i.e., the shape &
phase angle, except the amplitude.

All that is necessary is to recover is the amplitude of the wave.  Can
this be done when the noise is several orders of magnitude greater
than the signal?

Bret Cahill

Narrowing the final detection bandwidth is the only hope.
If the noise spectrum is white, narrow bandwidth through
averaging works. If you cannot average, there is trouble.
The old lockin amplifiers used a modulation or chopping signal
and a long time constant in the final filter.  I had signals
at times that required an hour of integration to detect. It
was slow but it worked.

I forgot to mention that the signal wave form can be anything that can
be generated.

For example, if something similar to an AM radio signal, say, f(t) =
(sint)(sin10t)(sin10t) was possible and the frequency of the noise was
about the same as the sin(t) factor, then f(t) will plot the noise
every time f(t) = 0.

In this case that would be ten times as often as the sin(t) factor.

The noise can then be subtracted to recover the wave form.

Bret Cahill

I think what's happening is that you have

1. Defined the noise to be bandlimited to radian frequencies below
about 1

The noise and the signal are both low and have about the same
frequency, ~ x/2pi.  There is little that can be done to change this
situation.  There is no time to wait more than several cycles for the
result either.

and

2. Up-converted (modulated) the signal to have usable (recoverable)
components around 10 times that frequency.

To plot the noise.  Every time sin 10t = 0, f(t) = zero, and the only
thing left is the low frequency noise.

You are assuming the ability to high-frequency modulate the signal
before the noise is added to it. So you already know what the
noiseless signal looks like.

As I said in the OP, "Everything is known about the transmitted wave,
i.e., the shape & phase angle, except the amplitude."

This works for the same reason reading a newspaper in a foreign
language is easy. You already know what they are going to say.


Bret Cahill

Are you familiar with the way a lockin amplifier works? That sounds
maybe like what you are doing. If you know everything about the signal
but its amplitude, then you have or can construct a normalized (unity
amplitude) version of it. That will positively correlate with the
unknown-amplitude version of the signal but have zero correlation to
random noise.

Things like IR absorption spectrometers commonly chop (square wave
modulate) the light source and recover the signal with a synchronous
rectifier. That washes out any noise picked up in the optical path or
the detector. Things like this commonly dig signals out from 1000x the
noise... but slowly.

If the noise is known to be bandlimited, it's a lot easier... almost
cheating.

John