Sensing \"indicator\" failures...

[Ricketty C]

On Tuesday, October 13, 2020 at 11:31:05 AM UTC-4, Piotr Wyderski wrote:

Don Y wrote:

For LEDs, I figured I could monitor the drop across the diode
over time (i.e., don\'t even worry about what it\'s \"initial,
theoretical value\" should be... just \"take notes\" and watch
for changes).  A diode that opens or shorts would be identifiable
with such a technique.  I\'m not sure I can learn/deduce anything
from gradual changes, over time...

LEDs are very fast, so instead of monitoring some static values you can
turn the diodes on for a microsecond periodically and check the params
synchronously with that. Human eye will not notice.

For these, I figure I could drive them with a current source
and, again, monitor the voltage across the device.  An open
load would be obvious as should a short (or stalled rotor?).

Those probably are based on brushed motors, so maybe it would better to
monitor the AC noise/switching component and compare it to a reference?
That way you will be able to discover the rotational frequency and
decide if it is within the limits. A low-resolution DFT is cheap even on
an 8-bitter.

Any likely faults that I\'m missing?

I am not sure if Juvenile was an EE, but by asking \"Quis custodiet ipsos
custodes?\" he had a point...

Best regards, Piotr

I knew a guy who worked on military electronics back when it was all TTL if that. They had a scanner to monitor the operation of some huge system looking for errors. The scanner was large enough they needed a scanner to scan the scanner called appropriately, the scanner scanner. When someone asked who scanned the scanner scanner the response was, \"The scanner scanner scans itself\".

--

Rick C.

+ Get 1,000 miles of free Supercharging
+ Tesla referral code - https://ts.la/richard11209

 

[Michael Terrell]

On Tuesday, October 13, 2020 at 2:19:23 PM UTC-4, Don Y wrote:

Interesting. I\'d heard similar stories of maintaining tube (valve)
based systems (and, that PM actually DECREASED reliability!).

People too easily dismiss the costs of these \"little things\" because they
can\'t wrap their heads around how the cost scales. E.g., replacing
lights in a home is a relatively low frequency/cost activity. OTOH,
replacing them in a hotel becomes a full-time job!

I used to run a pair of 16mm projectors at a B&W TV station. The lamps had a rated 20 hour life. They were difficult to keep in stock, and each projector used a pair of lamps. On the weekends, we would run almost 20 hours of film, per day. Our line voltage was unstable, and often hit 130 Volts. Since I didn\'t have to worry about color temperature, I reduced the lamp voltage a little at a time, until I started getting noise in the video. I went back up until it looked OK. That was around 90 Volts. I then averaged 130 hours per lamp. BTW, the projectors had an automatic bulb changer. If the filament opened, a motor was energized. That slid a rail up or down, where the lamps were mounted. Once it detected a good filament, it coasted to a soft detent. You couldn\'t see a change in brightness, since one was diming and the other getting brighter These were RCA TP66 Film Chain projectors.

As far as that VOA facility, I would bet that they phased out the 327 lamps and switched to the LED replacements once they became available. Each switch used four lamps, so it would still indicate with three dead lamps. They had two or four Microswitches as a snap on module, to the indicator module. In use, they looked quite nice. Here is a photo of one: https://www.ebay.com/itm/292482055231

 

[Don Y]

On 10/13/2020 11:20 AM, bitrex wrote:

On 10/13/2020 1:57 PM, Don Y wrote:
On 10/13/2020 7:14 AM, Klaus Kragelund wrote:
On Monday, October 12, 2020 at 2:54:38 AM UTC+2, Don Y wrote:
I\'m looking for inexpensive schemes to detect *gross* failures of
indicators intended to remain in service for decades.

IME, making devices signal actual or impending problems leaves
customers a lot more satisfied with their purchase choices -- instead
of continuously being vigilant for unexpected/unnoticed failures.

It\'s probably possible to detect dead segments in a 7 segment LED display by
using common cathode displays and driving them constant voltage, with a
constant voltage source in the cathode, and monitoring current with a single
low-value sense R, or looking at the feedback loop voltage.

A wild idea I thought of for automated lamp-test of many LEDS, maybe not useful
but since we\'re brainstorming I thought I\'d mention it.

Couple each LED to phototransistor/optical switch, and stack the switches in a
binary tree. Turn all lights on and force a current down each of the two main
branches of the binary switch tree, using feedback to equalize them, and
monitor the feedback loop voltage. If there\'s a fault somewhere in the tree the
loop voltage will swing out of tolerance, and the direction it swings should
show which branch of the tree the fault is in, and how much should show what
level. And then if it\'s a single open failure which lamp has failed can maybe
be detected by active probing.

I think that scheme would be able to localize single open failures and at least
detect multiple. There is probably a dual arrangement by measuring tree current
that can be used to detect the shorted condition.

You can do a lot more on the bench or in a tester than you can in a released
product (recurring cost issues). I\'d thought of adding an opto to just
\"check for light\" but ruled that out as likely increasing the probability
of failure (e.g., ESD damage to the opto).

[I prefer to sense \"outputs\" directly -- or, at least consequentially.
E.g., monitor the flow rate of a liquid being gated by a valve to
assure myself that the valve has, in fact, been actuated; \"listen\"
to sounds emitted by a speaker; etc. But, only when I can get those
data relatively cheaply (i.e., if there\'s already a microphone and a
loudspeaker on a device, then using each to \"test\" the other is just
a matter of finding an appropriate time -- in normal usage -- to do so.]

You can\'t lose track of the likely probability of failure when assessing how
much \"extra kit\" to throw at diagnosing same. E.g., I will expect the
little \"vibration motors\" to fail much sooner than the LEDs (despite having
orders of magnitude lower duty cycles). So, make them easy to \"validate\"
as well as replace (without a round trip to the depot!)

 

[Don Y]

On 10/14/2020 5:04 AM, Michael Terrell wrote:

On Tuesday, October 13, 2020 at 2:19:23 PM UTC-4, Don Y wrote:

Interesting. I\'d heard similar stories of maintaining tube (valve) based
systems (and, that PM actually DECREASED reliability!).

People too easily dismiss the costs of these \"little things\" because they
can\'t wrap their heads around how the cost scales. E.g., replacing lights
in a home is a relatively low frequency/cost activity. OTOH, replacing
them in a hotel becomes a full-time job!

I used to run a pair of 16mm projectors at a B&W TV station. The lamps had a
rated 20 hour life. They were difficult to keep in stock, and each projector
used a pair of lamps. On the weekends, we would run almost 20 hours of film,
per day. Our line voltage was unstable, and often hit 130 Volts. Since I
didn\'t have to worry about color temperature, I reduced the lamp voltage a
little at a time, until I started getting noise in the video. I went back up
until it looked OK. That was around 90 Volts. I then averaged 130 hours per
lamp. BTW, the projectors had an automatic bulb changer. If the filament
opened, a motor was energized. That slid a rail up or down, where the lamps
were mounted. Once it detected a good filament, it coasted to a soft detent.
You couldn\'t see a change in brightness, since one was diming and the other
getting brighter These were RCA TP66 Film Chain projectors.

This approach (cold spare) is also used in DLPs and \"overhead projectors\",
though the operator is responsible for \"sliding a lever\" to affect the
lamp change. Worth knowing if you\'re ever looking for \"spares\" for
ANOTHER device (check the ones with \"bad bulbs\" to see if their spares
have been put in service)

As far as that VOA facility, I would bet that they phased out the 327 lamps
and switched to the LED replacements once they became available. Each switch
used four lamps, so it would still indicate with three dead lamps. They had
two or four Microswitches as a snap on module, to the indicator module. In
use, they looked quite nice. Here is a photo of one:
https://www.ebay.com/itm/292482055231

I use the 327s in several \"jewel indicators\", here. Sort of a retro appeal.

 

[Michael Terrell]

On Wednesday, October 14, 2020 at 3:52:28 PM UTC-4, Don Y wrote:

On 10/14/2020 5:04 AM, Michael Terrell wrote:
On Tuesday, October 13, 2020 at 2:19:23 PM UTC-4, Don Y wrote:

Interesting. I\'d heard similar stories of maintaining tube (valve) based
systems (and, that PM actually DECREASED reliability!).

People too easily dismiss the costs of these \"little things\" because they
can\'t wrap their heads around how the cost scales. E.g., replacing lights
in a home is a relatively low frequency/cost activity. OTOH, replacing
them in a hotel becomes a full-time job!

I used to run a pair of 16mm projectors at a B&W TV station. The lamps had a
rated 20 hour life. They were difficult to keep in stock, and each projector
used a pair of lamps. On the weekends, we would run almost 20 hours of film,
per day. Our line voltage was unstable, and often hit 130 Volts. Since I
didn\'t have to worry about color temperature, I reduced the lamp voltage a
little at a time, until I started getting noise in the video. I went back up
until it looked OK. That was around 90 Volts. I then averaged 130 hours per
lamp. BTW, the projectors had an automatic bulb changer. If the filament
opened, a motor was energized. That slid a rail up or down, where the lamps
were mounted. Once it detected a good filament, it coasted to a soft detent.
You couldn\'t see a change in brightness, since one was diming and the other
getting brighter These were RCA TP66 Film Chain projectors.
This approach (cold spare) is also used in DLPs and \"overhead projectors\",
though the operator is responsible for \"sliding a lever\" to affect the
lamp change. Worth knowing if you\'re ever looking for \"spares\" for
ANOTHER device (check the ones with \"bad bulbs\" to see if their spares
have been put in service)
As far as that VOA facility, I would bet that they phased out the 327 lamps
and switched to the LED replacements once they became available. Each switch
used four lamps, so it would still indicate with three dead lamps. They had
two or four Microswitches as a snap on module, to the indicator module. In
use, they looked quite nice. Here is a photo of one:
https://www.ebay.com/itm/292482055231
I use the 327s in several \"jewel indicators\", here. Sort of a retro appeal.

The 327 and 328 used to be a common, but expensive lamp. I think I still have a couple surplus switches, and a handful of 327 lamps, somewhere. The idea of having a spare in place went with RCA\'s industrial approach to design of Broadcast equipment. Those projectors were about six feet tall, and the Electronics was modular and rack mounted inside. The same for things like the jeweled film gate assembly. It could be replaced in minutes, rather than stripping the projector down and shipping it to a service center.

 

[bitrex]

On 10/14/2020 3:48 PM, Don Y wrote:

On 10/13/2020 11:20 AM, bitrex wrote:
On 10/13/2020 1:57 PM, Don Y wrote:
On 10/13/2020 7:14 AM, Klaus Kragelund wrote:
On Monday, October 12, 2020 at 2:54:38 AM UTC+2, Don Y wrote:
I\'m looking for inexpensive schemes to detect *gross* failures of
indicators intended to remain in service for decades.

IME, making devices signal actual or impending problems leaves
customers a lot more satisfied with their purchase choices -- instead
of continuously being vigilant for unexpected/unnoticed failures.

It\'s probably possible to detect dead segments in a 7 segment LED
display by using common cathode displays and driving them constant
voltage, with a constant voltage source in the cathode, and monitoring
current with a single low-value sense R, or looking at the feedback
loop voltage.

A wild idea I thought of for automated lamp-test of many LEDS, maybe
not useful but since we\'re brainstorming I thought I\'d mention it.

Couple each LED to phototransistor/optical switch, and stack the
switches in a binary tree. Turn all lights on and force a current down
each of the two main branches of the binary switch tree, using
feedback to equalize them, and monitor the feedback loop voltage. If
there\'s a fault somewhere in the tree the loop voltage will swing out
of tolerance, and the direction it swings should show which branch of
the tree the fault is in, and how much should show what level. And
then if it\'s a single open failure which lamp has failed can maybe be
detected by active probing.

I think that scheme would be able to localize single open failures and
at least detect multiple. There is probably a dual arrangement by
measuring tree current that can be used to detect the shorted condition.

You can do a lot more on the bench or in a tester than you can in a
released
product (recurring cost issues).  I\'d thought of adding an opto to just
\"check for light\" but ruled that out as likely increasing the probability
of failure (e.g., ESD damage to the opto).

[I prefer to sense \"outputs\" directly -- or, at least consequentially.
E.g., monitor the flow rate of a liquid being gated by a valve to
assure myself that the valve has, in fact, been actuated; \"listen\"
to sounds emitted by a speaker; etc.  But, only when I can get those
data relatively cheaply (i.e., if there\'s already a microphone and a
loudspeaker on a device, then using each to \"test\" the other is just
a matter of finding an appropriate time -- in normal usage -- to do so.]

You can\'t lose track of the likely probability of failure when assessing
how
much \"extra kit\" to throw at diagnosing same.  E.g., I will expect the
little \"vibration motors\" to fail much sooner than the LEDs (despite having
orders of magnitude lower duty cycles).  So, make them easy to \"validate\"
as well as replace (without a round trip to the depot!)

The most rugged indicator I can think of are neon lamps; run
conservatively they\'re good for 10s of thousands of hours and are
near-impervious to any Earthly environmental effects short of mechanical
destruction.

 

[Don Y]

On 10/14/2020 8:59 PM, bitrex wrote:

The most rugged indicator I can think of are neon lamps; run conservatively
they\'re good for 10s of thousands of hours and are near-impervious to any
Earthly environmental effects short of mechanical destruction.

Eventually, the \"electrodes\" will lean into each other. But, I\'ve
no idea how many PoHrs have passed when this happens. The presence
of the ballast makes this a \"safe\" failure.

The biggest problem with neons is that they are really only good at
indicating static conditions -- like \"power on\" (or off!). You\'re
not likely to wink an error code with one.

 

[Cydrome Leader]

Don Y <blockedofcourse@foo.invalid> wrote:

On 10/11/2020 10:03 PM, Cydrome Leader wrote:
The next question is, if it\'s too hard to look at the LEDs for a lamp test
mode, who cares if they fail?

My point was that there is no natural point in typical usage where an automatic
lamp test could be inserted (unlike an automobile where the driver routinely
STARTS the vehicle and can be coerced to noticing the set of indicators
being tested on the dash). I would have to insert a MANUAL \"test lamps now\"
command (pushbutton, etc.) and then discipline users to do this \"at some
frequency\" (just how often SHOULD you test the indicators in your dashboard?
I suspect far less than \"every time you start the vehicle\" -- if you\'re just
looking to probabilistically catch an \"indicator failure\" before it \"has
consequences\").

You likely have smoke/CO detectors in your home. With nice big TEST buttons
in easily accessible locations. How often do you invoke that test MANUALLY?
And, one would assume that a device intended to save one\'s life would be a
great incentive to disciplining yourself to take the time to perform that test!

Ditto GFCI/AFCIs.

And, you likely only have a handful of such devices in your home. Imagine
having hundreds scattered around an industrial establishment. Do you task
your \"safety\" officer with *routinely* making the rounds and pressing the
\"lamp test\" buttons? How often? What\'s the TCO consequence of this -- vs.
having the device check itself and REPORT when it (infrequently) needs to
be serviced?

Back to the fire alarm scenario. Many of those detectors will always flash
a LED, just to let you know they\'re being polled and part of a loop. No
blinking means something is wrong and should be reported to maintenance.
No special lamp test mode is needed for the remote detectors. The
annunciator panel is different. They don\'t flash every light constantly.

Good luck finding an emergency exit light that works at restaurant, but
you can still tell if they\'re even wired to anything by whether or not the
light for the test button/charging light is even lit. If nobody is payaing
any attention to your devices, there\'s not really much you can do.

 

[Don Y]

On 10/14/2020 4:04 PM, Michael Terrell wrote:

On Wednesday, October 14, 2020 at 3:52:28 PM UTC-4, Don Y wrote:
On 10/14/2020 5:04 AM, Michael Terrell wrote:
On Tuesday, October 13, 2020 at 2:19:23 PM UTC-4, Don Y wrote:

Interesting. I\'d heard similar stories of maintaining tube (valve)
based systems (and, that PM actually DECREASED reliability!).

People too easily dismiss the costs of these \"little things\" because
they can\'t wrap their heads around how the cost scales. E.g.,
replacing lights in a home is a relatively low frequency/cost
activity. OTOH, replacing them in a hotel becomes a full-time job!

I used to run a pair of 16mm projectors at a B&W TV station. The lamps
had a rated 20 hour life. They were difficult to keep in stock, and
each projector used a pair of lamps. On the weekends, we would run
almost 20 hours of film, per day. Our line voltage was unstable, and
often hit 130 Volts. Since I didn\'t have to worry about color
temperature, I reduced the lamp voltage a little at a time, until I
started getting noise in the video. I went back up until it looked OK.
That was around 90 Volts. I then averaged 130 hours per lamp. BTW, the
projectors had an automatic bulb changer. If the filament opened, a
motor was energized. That slid a rail up or down, where the lamps were
mounted. Once it detected a good filament, it coasted to a soft detent.
You couldn\'t see a change in brightness, since one was diming and the
other getting brighter These were RCA TP66 Film Chain projectors.
This approach (cold spare) is also used in DLPs and \"overhead projectors\",
though the operator is responsible for \"sliding a lever\" to affect the
lamp change. Worth knowing if you\'re ever looking for \"spares\" for ANOTHER
device (check the ones with \"bad bulbs\" to see if their spares have been
put in service)
As far as that VOA facility, I would bet that they phased out the 327
lamps and switched to the LED replacements once they became available.
Each switch used four lamps, so it would still indicate with three dead
lamps. They had two or four Microswitches as a snap on module, to the
indicator module. In use, they looked quite nice. Here is a photo of
one: https://www.ebay.com/itm/292482055231
I use the 327s in several \"jewel indicators\", here. Sort of a retro
appeal.

The 327 and 328 used to be a common, but expensive lamp. I think I still
have a couple surplus switches, and a handful of 327 lamps, somewhere.

Dunno. A lab \"cleaned house\" many (manymany!) years ago and I rescued
boxes and boxes of lamps, fixtures, indicators, switches, etc. (e.g.,
\"datalight\", etc.) I spent many hours sorting through them and
creating individual bins/drawers for each type. I figure it was
a fair bit more than a lifetime supply, for me! :>

The idea of having a spare in place went with RCA\'s industrial approach to
design of Broadcast equipment. Those projectors were about six feet tall,
and the Electronics was modular and rack mounted inside. The same for
things like the jeweled film gate assembly. It could be replaced in minutes,
rather than stripping the projector down and shipping it to a service
center.

Yes, designing for high availability is different than designing for high
reliability. (though increasing availability can often adversely impact
reliability -- more things/mechanisms that can \"go wrong\")

One of my UPSs can essentially be replaced (batteries, electronics) without
disconnecting the (powered!) loads.

When I design industrial controls, I strive for small size and the ability
to remove-and-replace the entire system without having to spend lots of
labor (time) uncabling things, calibrating replacements, etc. You don\'t
want to have The Line down for more than minutes/hours -- definitely not
shifts or days!

[In some markets, this was a hard sell -- \"That\'s not the way we DO things!\".
But, once in place, led to increased sales (now everyone wants to have a cold
spare on-hand \"just in case\") and greater confidence in the product lines
among customers (\"We bought a spare but we\'ve never had to use it!\")]

In software, availability is relatively easy to provide if you design with
it in mind from the start. E.g., in my current system, I can take an axe
to any node and not affect the operation of the system, as a whole. (demoing
this later this month -- though I\'ll just cut cables instead of whacking things
with an axe!! :> ) Having redundant hardware usually comes at some recurring
cost.

 

[Don Y]

On 10/14/2020 10:29 PM, Cydrome Leader wrote:

Don Y <blockedofcourse@foo.invalid> wrote:
On 10/11/2020 10:03 PM, Cydrome Leader wrote:
The next question is, if it\'s too hard to look at the LEDs for a lamp test
mode, who cares if they fail?

My point was that there is no natural point in typical usage where an automatic
lamp test could be inserted (unlike an automobile where the driver routinely
STARTS the vehicle and can be coerced to noticing the set of indicators
being tested on the dash). I would have to insert a MANUAL \"test lamps now\"
command (pushbutton, etc.) and then discipline users to do this \"at some
frequency\" (just how often SHOULD you test the indicators in your dashboard?
I suspect far less than \"every time you start the vehicle\" -- if you\'re just
looking to probabilistically catch an \"indicator failure\" before it \"has
consequences\").

You likely have smoke/CO detectors in your home. With nice big TEST buttons
in easily accessible locations. How often do you invoke that test MANUALLY?
And, one would assume that a device intended to save one\'s life would be a
great incentive to disciplining yourself to take the time to perform that test!

Ditto GFCI/AFCIs.

And, you likely only have a handful of such devices in your home. Imagine
having hundreds scattered around an industrial establishment. Do you task
your \"safety\" officer with *routinely* making the rounds and pressing the
\"lamp test\" buttons? How often? What\'s the TCO consequence of this -- vs.
having the device check itself and REPORT when it (infrequently) needs to
be serviced?

Back to the fire alarm scenario. Many of those detectors will always flash
a LED, just to let you know they\'re being polled and part of a loop. No
blinking means something is wrong and should be reported to maintenance.
No special lamp test mode is needed for the remote detectors. The
annunciator panel is different. They don\'t flash every light constantly.

But casual observes don\'t notice these things. Our (residential) smoke/Co
detectors have a bunch of lights on them, all designed to be easily visible.
One is constantly illuminated (mains power). The others flash (I think)
when certain portions of the device are exercised (e.g., \"Nope, I don\'t
smell any smoke...\")

Having ONCE read a description of what the various LEDs mean, I no longer
have any interest in them. I know they flash so briefly that I\'d have to
deliberately stop beside a detector to verify that it DID, in fact, wink
at me.

I will *possibly* notice if the power indicator is extinguished. But,
I may not.

And, my other half won\'t notice any of the above!

Good luck finding an emergency exit light that works at restaurant, but
you can still tell if they\'re even wired to anything by whether or not the
light for the test button/charging light is even lit. If nobody is payaing
any attention to your devices, there\'s not really much you can do.

Folks will vaguely acknowledge that their dashboard lights up with
a bunch of \"indicators\" as they start the car. But, they can\'t tell
you how many or what they mean (without analyzing the entire suite
of \"icons\" and making educated guesses). I suspect there are a dozen
that come lit on our vehicle (tire pressure, seat belt, ABS system
failure, etc.). But, I couldn\'t tell you if one -- or 10! -- failed to
illuminate at any given time.

OTOH, if *any* idiot light comes on, it will be noticed (\"Gee, I wonder what
THAT one means?\").

You don\'t want to have to rely on users to verify that \"all is well\".
Rather, you want to ENSURE that you can inform the user when something is
NOT well -- and in a way that the user won\'t likely miss/ignore.

[Note the overkill of smoke detectors chirping nonstop as the battery
fails. \"Too much\" of an annoyance means folks will disable the smoke
detector while waiting to purchase a replacement battery -- possibly
forgetting this, too!]