triggering things with ethernet...

[Jasen Betts]

On 2023-04-17, Ricky <gnuarm.deletethisbit@gmail.com> wrote:

On Monday, April 17, 2023 at 3:18:59 PM UTC-4, John Larkin wrote:
Suppose one were to send a broadcast packet from a PC to multiple
boxes over regular 100 Mbit ethernet, without any fancy time protocols
like PTP or anything. Each box would accept the packet as a trigger.
Assume a pretty small private network and a reasonable number of
switches to fan out to many boxes.

Any guess as to how much the effective time trigger to various boxes
would skew? I\'ve seen one esimate of 125 usec, for cameras, with
details unclear.

I would use those RJ45 to RS-232 socket adapters and drive the cable from an RS-232 port. Wire all the cables in parallel, or daisy chain them. Very low skew. Use a high baud rate and send a very small message to keep the delay time low as well. Likely a 1 character \"packet\" would suffice.

I would use RS485 for a better impedance match and better fan-out.




--
Jasen.
🇺🇦 Слава Україні

 

[Jasen Betts]

On 2023-04-17, John Larkin <jlarkin@highlandSNIPMEtechnology.com> wrote:

Suppose one were to send a broadcast packet from a PC to multiple
boxes over regular 100 Mbit ethernet, without any fancy time protocols
like PTP or anything. Each box would accept the packet as a trigger.
Assume a pretty small private network and a reasonable number of
switches to fan out to many boxes.

Any guess as to how much the effective time trigger to various boxes
would skew? I\'ve seen one esimate of 125 usec, for cameras, with
details unclear.

They sell these not to spec multi-port POE injector boxes, that parallel the two unused
pairs in the RJ45 use them instead of switches, (or in adition to
switches if you need ethernet for some other purpose)

Put your signal pulse in the \"power\" channel




--
Jasen.
🇺🇦 Слава Україні

 

[Dimiter_Popoff]

On 4/18/2023 10:44, Don Y wrote:

On 4/17/2023 12:44 PM, Dimiter_Popoff wrote:
Hmmm, interesting question. Since switches do \"buffer then transmit\"
this might go either way; if the same copy is retransmitted to
all rj-45s this can be done with practically no skew (apart from
that at the receiving side, clock phases, software latencies etc.).

Remember, the switch isn\'t a wire.  Instead, it can be seen as a buffer
upstream of each port.  So, if port 4 is busy delivering a packet to it\'s
client (from port 7, e.g.) but port 3 is idle (because no one loves him!),
then a message intended for port 3 will be delivered before a message
(possibly the same BROADCAST message) will be delivered to port 4.

Depending on the switch architecture, a backlog at switch 4 may cause the
\"next\" message intended for it to be dropped, *in* the switch -- while
port 3 manages to receive it.
 > But I doubt many - if any - switches do that, my bet would be that they
transmit one cable at a time.
On the old coaxial Ethernet the answer is obvious but not much
use.

10Base2/5 preferable to <any>BaseT (*wire* instead of hub/switch).

But, even there, you have differences in the PHYs/NICs, stack, etc.
that add to the variability.

Well not as much, the PHY I remember (an NSC part) does not buffer
data so it must add just nanoseconds of delay. I meant that all
skew is receiver generated, MAC, CPU etc.

If you don\'t control the hardware *and* software in each node, you\'re
just pissing in the wind and hoping not to get wet!

Of course so, but I assume John controls all receiver nodes, his
devices I have seen. Obviously he will have to rely no someone else\'s
network design but taming it to behave coherently may be doable.

So, you have to resort to standards\' compliance and figure out how to
get the performance you want/need *in* that framework (and, then,
insist that everything you talk to is compliant).

If you need some diversity of triggered and/or trigger devices you
put on the market yes, of course. In scientific equipment this is
rarely the case, they typically need a particular system for a
particular problem and if you design all of it you have more
options.

 

[Martin Brown]

On 17/04/2023 20:18, John Larkin wrote:

Suppose one were to send a broadcast packet from a PC to multiple
boxes over regular 100 Mbit ethernet, without any fancy time protocols
like PTP or anything. Each box would accept the packet as a trigger.
Assume a pretty small private network and a reasonable number of
switches to fan out to many boxes.

Any guess as to how much the effective time trigger to various boxes
would skew? I\'ve seen one esimate of 125 usec, for cameras, with
details unclear.

My suggestion would be to measure it experimentally on a modest sized
configuration with the depth of switches you intend to use and have the
triggered devices send back a step function to the master box on an
identical length of coax. That should give you a good idea of the delay
per additional switch added and how the jitter increases with depth.

It will obviously depend a lot on the software and stack at the receiver
- if you can control that and/or put it into a quick response state then
you might be able to do quite well. That or have a means to calibrate
out the systematic delays using the same length of coax as a reference.
Depends a lot on good behaviour from the switches so you might have to
be careful about which chipsets you specify.

--
Martin Brown

 

[Don Y]

On 4/18/2023 4:36 AM, Dimiter_Popoff wrote:

On 4/18/2023 10:44, Don Y wrote:
On 4/17/2023 12:44 PM, Dimiter_Popoff wrote:
Hmmm, interesting question. Since switches do \"buffer then transmit\"
this might go either way; if the same copy is retransmitted to
all rj-45s this can be done with practically no skew (apart from
that at the receiving side, clock phases, software latencies etc.).

Remember, the switch isn\'t a wire.  Instead, it can be seen as a buffer
upstream of each port.  So, if port 4 is busy delivering a packet to it\'s
client (from port 7, e.g.) but port 3 is idle (because no one loves him!),
then a message intended for port 3 will be delivered before a message
(possibly the same BROADCAST message) will be delivered to port 4.

Depending on the switch architecture, a backlog at switch 4 may cause the
\"next\" message intended for it to be dropped, *in* the switch -- while
port 3 manages to receive it.
  > But I doubt many - if any - switches do that, my bet would be that they
transmit one cable at a time.
On the old coaxial Ethernet the answer is obvious but not much
use.

10Base2/5 preferable to <any>BaseT (*wire* instead of hub/switch).

But, even there, you have differences in the PHYs/NICs, stack, etc.
that add to the variability.

Well not as much, the PHY I remember (an NSC part) does not buffer
data so it must add just nanoseconds of delay. I meant that all
skew is receiver generated, MAC, CPU etc.

Presumably, there is a \"line of code\" *somewhere* that starts the
process. This code has to get executed and drag all of the
subsequent mechanisms in before the \"correct\" packet gets put
on the wire.

So, any variability in the execution path (e.g., conditionals,
preemption, etc.) appears in series with that \"line of code\"
just as any variability in the receiving (and transport) end
would.

If you don\'t control the hardware *and* software in each node, you\'re
just pissing in the wind and hoping not to get wet!

Of course so, but I assume John controls all receiver nodes, his
devices I have seen. Obviously he will have to rely no someone else\'s
network design but taming it to behave coherently may be doable.

Do *you* control all of the devices on the LAN your device occupies?
I think it\'s pretty unlikely that a completely closed system would
bother with ethernet for triggers -- why not an optical fiber with
a pulsed light source wired to the device in question? (or, a piece of
copper, etc.)

Ethernet as a generic choice imposes limitations that (if triggering
were the sole use) can complicate the design. E.g., with more than
two devices on the wire, you need a switch/router. Will you supply that,
as well? Will you *qualify* third party switches (how much resources will
you devote to this? do you know the switches that your customer is
*likely* going to have on hand -- or be willing to purchase JUST for you?)

[OTOH, a length of wire -- or optical fiber -- is pretty straightforward]

Using something as ubiquitous as ethernet opens the door for connection
to devices \"made by others\". So, *you* don\'t have to make everything
that a customer *might* want to use (with your primary product).

[The age of standalone boxes is quickly coming to a close; everything WANTS
to talk to everything else!]

So, you have to resort to standards\' compliance and figure out how to
get the performance you want/need *in* that framework (and, then,
insist that everything you talk to is compliant).

If you need some diversity of triggered and/or trigger devices you
put on the market yes, of course. In scientific equipment this is
rarely the case, they typically need a particular system for a
particular problem and if you design all of it you have more
options.

See above.

[I think mail is hosed, again :< ]

 

[Fred Bloggs]

On Monday, April 17, 2023 at 3:18:59 PM UTC-4, John Larkin wrote:

Suppose one were to send a broadcast packet from a PC to multiple
boxes over regular 100 Mbit ethernet, without any fancy time protocols
like PTP or anything. Each box would accept the packet as a trigger.
Assume a pretty small private network and a reasonable number of
switches to fan out to many boxes.

Any guess as to how much the effective time trigger to various boxes
would skew? I\'ve seen one esimate of 125 usec, for cameras, with
details unclear.

Too bad you can\'t set them up to do whatever it is at a specified time of day.

 

[Dimiter_Popoff]

On 4/18/2023 16:08, Don Y wrote:

On 4/18/2023 4:36 AM, Dimiter_Popoff wrote:
On 4/18/2023 10:44, Don Y wrote:
On 4/17/2023 12:44 PM, Dimiter_Popoff wrote:
Hmmm, interesting question. Since switches do \"buffer then transmit\"
this might go either way; if the same copy is retransmitted to
all rj-45s this can be done with practically no skew (apart from
that at the receiving side, clock phases, software latencies etc.).

Remember, the switch isn\'t a wire.  Instead, it can be seen as a buffer
upstream of each port.  So, if port 4 is busy delivering a packet to
it\'s
client (from port 7, e.g.) but port 3 is idle (because no one loves
him!),
then a message intended for port 3 will be delivered before a message
(possibly the same BROADCAST message) will be delivered to port 4.

Depending on the switch architecture, a backlog at switch 4 may cause
the
\"next\" message intended for it to be dropped, *in* the switch -- while
port 3 manages to receive it.
  > But I doubt many - if any - switches do that, my bet would be
that they
transmit one cable at a time.
On the old coaxial Ethernet the answer is obvious but not much
use.

10Base2/5 preferable to <any>BaseT (*wire* instead of hub/switch).

But, even there, you have differences in the PHYs/NICs, stack, etc.
that add to the variability.

Well not as much, the PHY I remember (an NSC part) does not buffer
data so it must add just nanoseconds of delay. I meant that all
skew is receiver generated, MAC, CPU etc.

Presumably, there is a \"line of code\" *somewhere* that starts the
process.  This code has to get executed and drag all of the
subsequent mechanisms in before the \"correct\" packet gets put
on the wire.

So, any variability in the execution path (e.g., conditionals,
preemption, etc.) appears in series with that \"line of code\"
just as any variability in the receiving (and transport) end
would.

I just assumed the initiation is sort of a pushed button by a person,
so all the skew comes after the broadcast packet is out.

If you don\'t control the hardware *and* software in each node, you\'re
just pissing in the wind and hoping not to get wet!

Of course so, but I assume John controls all receiver nodes, his
devices I have seen. Obviously he will have to rely no someone else\'s
network design but taming it to behave coherently may be doable.

Do *you* control all of the devices on the LAN your device occupies?

Well, sometimes yes. If they are all dps devices, I can control them.
Which is not to say I don\'t get surprised, stunned etc .

I think it\'s pretty unlikely that a completely closed system would
bother with ethernet for triggers -- why not an optical fiber with
a pulsed light source wired to the device in question?  (or, a piece of
copper, etc.)

John mentioned some 125us, apparently this is the ballpark he is after
and wants some verification. This is a huge amount of time, especially
by his picosecond standards, if he were after something that fast/low
skew obviously he would not have thought of Ethernet.

I have considered syncing multiple MCA-s so in list mode they have
skew well below a nanosecond but the customers for that sort of thing
just don\'t call. Obviously I did not think Ethernet, either .
As it has been typical, they will call here once they have run out of
other usable options...

[I think mail is hosed, again  :< ]

I did email you earlier today (nothing worth a second thought if it
gets lost).

 

[Ricky]

On Tuesday, April 18, 2023 at 7:30:50 AM UTC-4, Jasen Betts wrote:

On 2023-04-17, Ricky <gnuarm.del...@gmail.com> wrote:
On Monday, April 17, 2023 at 3:18:59 PM UTC-4, John Larkin wrote:
Suppose one were to send a broadcast packet from a PC to multiple
boxes over regular 100 Mbit ethernet, without any fancy time protocols
like PTP or anything. Each box would accept the packet as a trigger.
Assume a pretty small private network and a reasonable number of
switches to fan out to many boxes.

Any guess as to how much the effective time trigger to various boxes
would skew? I\'ve seen one esimate of 125 usec, for cameras, with
details unclear.

I would use those RJ45 to RS-232 socket adapters and drive the cable from an RS-232 port. Wire all the cables in parallel, or daisy chain them. Very low skew. Use a high baud rate and send a very small message to keep the delay time low as well. Likely a 1 character \"packet\" would suffice.
I would use RS485 for a better impedance match and better fan-out.

Interesting. That\'s an engineer\'s reply, optimizing before knowing the requirements. We all do that.

Doesn\'t matter. Larkin isn\'t going to use either. He\'s just fishing for someone to do his work for him. He has since said this will involve customer equipment that is on a network, so Ethernet it is!

--

Rick C.

+ Get 1,000 miles of free Supercharging
+ Tesla referral code - https://ts.la/richard11209

 

[John Larkin]

On Tue, 18 Apr 2023 06:08:26 -0700, Don Y
<blockedofcourse@foo.invalid> wrote:

On 4/18/2023 4:36 AM, Dimiter_Popoff wrote:
On 4/18/2023 10:44, Don Y wrote:
On 4/17/2023 12:44 PM, Dimiter_Popoff wrote:
Hmmm, interesting question. Since switches do \"buffer then transmit\"
this might go either way; if the same copy is retransmitted to
all rj-45s this can be done with practically no skew (apart from
that at the receiving side, clock phases, software latencies etc.).

Remember, the switch isn\'t a wire.  Instead, it can be seen as a buffer
upstream of each port.  So, if port 4 is busy delivering a packet to it\'s
client (from port 7, e.g.) but port 3 is idle (because no one loves him!),
then a message intended for port 3 will be delivered before a message
(possibly the same BROADCAST message) will be delivered to port 4.

Depending on the switch architecture, a backlog at switch 4 may cause the
\"next\" message intended for it to be dropped, *in* the switch -- while
port 3 manages to receive it.
  > But I doubt many - if any - switches do that, my bet would be that they
transmit one cable at a time.
On the old coaxial Ethernet the answer is obvious but not much
use.

10Base2/5 preferable to <any>BaseT (*wire* instead of hub/switch).

But, even there, you have differences in the PHYs/NICs, stack, etc.
that add to the variability.

Well not as much, the PHY I remember (an NSC part) does not buffer
data so it must add just nanoseconds of delay. I meant that all
skew is receiver generated, MAC, CPU etc.

Presumably, there is a \"line of code\" *somewhere* that starts the
process. This code has to get executed and drag all of the
subsequent mechanisms in before the \"correct\" packet gets put
on the wire.

So, any variability in the execution path (e.g., conditionals,
preemption, etc.) appears in series with that \"line of code\"
just as any variability in the receiving (and transport) end
would.

If you don\'t control the hardware *and* software in each node, you\'re
just pissing in the wind and hoping not to get wet!

Of course so, but I assume John controls all receiver nodes, his
devices I have seen. Obviously he will have to rely no someone else\'s
network design but taming it to behave coherently may be doable.

Do *you* control all of the devices on the LAN your device occupies?
I think it\'s pretty unlikely that a completely closed system would
bother with ethernet for triggers -- why not an optical fiber with
a pulsed light source wired to the device in question? (or, a piece of
copper, etc.)

Of course we can get picosecond precision in a big time
distribution/trigger system using fiber; I\'ve done that.

But CAT5 cables are cheap and easy to get installed. My question is,
what would time response be like? If I send a UTP broadcast packet to
my system, the GO trigger to a dozen boxes, does a switch relay them
all in parallel?

It\'s hard to find numbers.

Ethernet as a generic choice imposes limitations that (if triggering
were the sole use) can complicate the design. E.g., with more than
two devices on the wire, you need a switch/router. Will you supply that,
as well? Will you *qualify* third party switches (how much resources will
you devote to this? do you know the switches that your customer is
*likely* going to have on hand -- or be willing to purchase JUST for you?)

[OTOH, a length of wire -- or optical fiber -- is pretty straightforward]

Using something as ubiquitous as ethernet opens the door for connection
to devices \"made by others\". So, *you* don\'t have to make everything
that a customer *might* want to use (with your primary product).

[The age of standalone boxes is quickly coming to a close; everything WANTS
to talk to everything else!]

So, you have to resort to standards\' compliance and figure out how to
get the performance you want/need *in* that framework (and, then,
insist that everything you talk to is compliant).

If you need some diversity of triggered and/or trigger devices you
put on the market yes, of course. In scientific equipment this is
rarely the case, they typically need a particular system for a
particular problem and if you design all of it you have more
options.

See above.

[I think mail is hosed, again :< ]

 

[John Larkin]

On Tue, 18 Apr 2023 11:10:30 -0000 (UTC), Jasen Betts
<usenet@revmaps.no-ip.org> wrote:

On 2023-04-17, Ricky <gnuarm.deletethisbit@gmail.com> wrote:
On Monday, April 17, 2023 at 3:18:59?PM UTC-4, John Larkin wrote:
Suppose one were to send a broadcast packet from a PC to multiple
boxes over regular 100 Mbit ethernet, without any fancy time protocols
like PTP or anything. Each box would accept the packet as a trigger.
Assume a pretty small private network and a reasonable number of
switches to fan out to many boxes.

Any guess as to how much the effective time trigger to various boxes
would skew? I\'ve seen one esimate of 125 usec, for cameras, with
details unclear.

I would use those RJ45 to RS-232 socket adapters and drive the cable from an RS-232 port. Wire all the cables in parallel, or daisy chain them. Very low skew. Use a high baud rate and send a very small message to keep the delay time low as well. Likely a 1 character \"packet\" would suffice.

I would use RS485 for a better impedance match and better fan-out.

Ethernet is universal. Every PC, every laptop, has an Ethernet port.
CAT5 cables and switches are cheap and everybody knows how to buy and
install them.

 

[John Larkin]

On Mon, 17 Apr 2023 22:15:44 -0700, dplatt@coop.radagast.org (Dave
Platt) wrote:

In article <u1l0ur$3cn42$1@dont-email.me>,
Don Y <blockedofcourse@foo.invalid> wrote:
On 4/17/2023 6:38 PM, Joe Gwinn wrote:
It\'s been done, many times. Guilty. But PTP over ethernet is
sweeping all that stuff away.

PTP, itself, does nothing to eliminate the instantaneous variabilities;
it is intended to be used to discipline a local clock (with filtering
above the packet level).

One *could* use 1588 *hardware* to try to quantify the instantaneous
arrival times of packets. But, you\'d still not be sure when YOUR
packet arrived relative to other (broadcast) packets arriving at your
peers.

I.e., you still need *a* protocol built atop that.

I agree. The simplest reliable approach I can think of, is to use PTP
or something of similar intent to distribute a well-synchronized clock
to all of the devices in the house.

Then, send out a broadcast \"Turn on, at time X\" packet to all nodes,
where \"X\" is far enough in the future that you can reasonably ensure that
all nodes will receive the packet before time \"X\" arrives.

Each device sets a timer when it receives the packet.

The skew in the actual \"turn on\" times will depend on the success of
your time-synchronization protocol, and the accuracy of each node\'s
internal timer. It should be possible to get it down to much less
than the skew in transmission times.

Sure, but something like PTP adds a lot of complexity on both ends. A
standard Dell PC doesn\'t do PTP. A Raspberry Pi Pico doesn\'t either.

 

[John Larkin]

On Tue, 18 Apr 2023 11:15:41 -0000 (UTC), Jasen Betts
<usenet@revmaps.no-ip.org> wrote:

On 2023-04-17, John Larkin <jlarkin@highlandSNIPMEtechnology.com> wrote:


Suppose one were to send a broadcast packet from a PC to multiple
boxes over regular 100 Mbit ethernet, without any fancy time protocols
like PTP or anything. Each box would accept the packet as a trigger.
Assume a pretty small private network and a reasonable number of
switches to fan out to many boxes.

Any guess as to how much the effective time trigger to various boxes
would skew? I\'ve seen one esimate of 125 usec, for cameras, with
details unclear.

They sell these not to spec multi-port POE injector boxes, that parallel the two unused
pairs in the RJ45 use them instead of switches, (or in adition to
switches if you need ethernet for some other purpose)

Put your signal pulse in the \"power\" channel

Yes, I\'m considering using the two spare pairs for a 10 MHz clock and
for a trigger, with PoE on the usual data pairs. But a stock Dell PC
can\'t do that. It can send a broadcast UDP packet.

If we\'re controlling remote power supplies and motors, a few hundred
usec of uncertainty is OK.

 

[John Larkin]

On Tue, 18 Apr 2023 13:45:07 +0100, Martin Brown
<\'\'\'newspam\'\'\'@nonad.co.uk> wrote:

On 17/04/2023 20:18, John Larkin wrote:


Suppose one were to send a broadcast packet from a PC to multiple
boxes over regular 100 Mbit ethernet, without any fancy time protocols
like PTP or anything. Each box would accept the packet as a trigger.
Assume a pretty small private network and a reasonable number of
switches to fan out to many boxes.

Any guess as to how much the effective time trigger to various boxes
would skew? I\'ve seen one esimate of 125 usec, for cameras, with
details unclear.

My suggestion would be to measure it experimentally on a modest sized
configuration with the depth of switches you intend to use and have the
triggered devices send back a step function to the master box on an
identical length of coax. That should give you a good idea of the delay
per additional switch added and how the jitter increases with depth.

Looks like I\'ll have to do that. Googling, and discussing here,
doesn\'t seem to provide numbers. There are some ethernet camera
vendors who hint at hundreds-of-microseconds skew but the setup isn\'t
clear. Maybe I can email them and pretend to need a lot of cameras.

I did once make a bunch of proto PCBs that snoop a pair of RJ45s, out
to coax connectors, so I can scope the ethernet at a few destination
devices. I don\'t know how intelligable that would be.

It will obviously depend a lot on the software and stack at the receiver
- if you can control that and/or put it into a quick response state then
you might be able to do quite well. That or have a means to calibrate
out the systematic delays using the same length of coax as a reference.
Depends a lot on good behaviour from the switches so you might have to
be careful about which chipsets you specify.

My receivers will have to be designed to recognize the GO trigger
packet fast, and suitable switches would be needed. I can image that
some are better than others.

 

[Lasse Langwadt Christensen]

tirsdag den 18. april 2023 kl. 15.08.39 UTC+2 skrev Don Y:

On 4/18/2023 4:36 AM, Dimiter_Popoff wrote:
On 4/18/2023 10:44, Don Y wrote:
On 4/17/2023 12:44 PM, Dimiter_Popoff wrote:
Hmmm, interesting question. Since switches do \"buffer then transmit\"
this might go either way; if the same copy is retransmitted to
all rj-45s this can be done with practically no skew (apart from
that at the receiving side, clock phases, software latencies etc.).

Remember, the switch isn\'t a wire. Instead, it can be seen as a buffer
upstream of each port. So, if port 4 is busy delivering a packet to it\'s
client (from port 7, e.g.) but port 3 is idle (because no one loves him!),
then a message intended for port 3 will be delivered before a message
(possibly the same BROADCAST message) will be delivered to port 4.

Depending on the switch architecture, a backlog at switch 4 may cause the
\"next\" message intended for it to be dropped, *in* the switch -- while
port 3 manages to receive it.
But I doubt many - if any - switches do that, my bet would be that they
transmit one cable at a time.
On the old coaxial Ethernet the answer is obvious but not much
use.

10Base2/5 preferable to <any>BaseT (*wire* instead of hub/switch).

But, even there, you have differences in the PHYs/NICs, stack, etc.
that add to the variability.

Well not as much, the PHY I remember (an NSC part) does not buffer
data so it must add just nanoseconds of delay. I meant that all
skew is receiver generated, MAC, CPU etc.
Presumably, there is a \"line of code\" *somewhere* that starts the
process. This code has to get executed and drag all of the
subsequent mechanisms in before the \"correct\" packet gets put
on the wire.

So, any variability in the execution path (e.g., conditionals,
preemption, etc.) appears in series with that \"line of code\"
just as any variability in the receiving (and transport) end
would.

it is not the code that is the main issue.

100Mbit requires a switch and it may not send broadcast packages to all the down stream ports at the same time
with 10Mbit you can use a hub so all ports hear everything thing all the time

 

[Jan Panteltje]

On a sunny day (Tue, 18 Apr 2023 09:08:45 -0700) it happened John Larkin
<jlarkin@highlandSNIPMEtechnology.com> wrote in
<3tft3itr1pbrt35hk3gu74d2subbtfupql@4ax.com>:

On Tue, 18 Apr 2023 11:10:30 -0000 (UTC), Jasen Betts
usenet@revmaps.no-ip.org> wrote:

On 2023-04-17, Ricky <gnuarm.deletethisbit@gmail.com> wrote:
On Monday, April 17, 2023 at 3:18:59?PM UTC-4, John Larkin wrote:
Suppose one were to send a broadcast packet from a PC to multiple
boxes over regular 100 Mbit ethernet, without any fancy time protocols
like PTP or anything. Each box would accept the packet as a trigger.
Assume a pretty small private network and a reasonable number of
switches to fan out to many boxes.

Any guess as to how much the effective time trigger to various boxes
would skew? I\'ve seen one esimate of 125 usec, for cameras, with
details unclear.

I would use those RJ45 to RS-232 socket adapters and drive the cable from an RS-232 port. Wire all the cables in parallel,
or daisy chain them. Very low skew. Use a high baud rate and send a very small message to keep the delay time low as well.
Likely a 1 character \"packet\" would suffice.

I would use RS485 for a better impedance match and better fan-out.

Ethernet is universal. Every PC, every laptop, has an Ethernet port.
CAT5 cables and switches are cheap and everybody knows how to buy and
install them.

If you want to know the time delays, test it with a ping
for example
raspberrypi: ~ # ping 192.168.178.20
PING 192.168.178.20 (192.168.178.20) 56(84) bytes of data.
64 bytes from 192.168.178.20: icmp_seq=1 ttl=64 time=1.26 ms
64 bytes from 192.168.178.20: icmp_seq=2 ttl=64 time=0.452 ms
^C
--- 192.168.178.20 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 0.452/0.854/1.256/0.402 ms

The first one is usually slower...

All depends on the processor load of both the sender and receiver, network traffic on that ethernet link etc

This is to myself, where I am posting from now, also the 4G gateway
raspberrypi: ~ # ping 127.0.0.1
PING 127.0.0.1 (127.0.0.1) 56(84) bytes of data.
64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.148 ms
64 bytes from 127.0.0.1: icmp_seq=2 ttl=64 time=0.122 ms
^C
--- 127.0.0.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1053ms
rtt min/avg/max/mdev = 0.122/0.135/0.148/0.013 ms

So not even below 122 ns

If your PCs run a precision clock then you can instruct those to do something at a specific time.
One here runs on a GPS clock (IF there are enough satellites in view, sometimes there are not).

You can use programs like \'snort\' to see what else happens on the network.
like ARP requests etc etc, example excluding 2 IP adresses looking at eth1 port:
snort -s --nolock-pidfile -i eth1 -d -v not host 192.168.178.95 and not host 102.168.178.99

 

[John Larkin]

On Tue, 18 Apr 2023 09:30:52 -0700 (PDT), Lasse Langwadt Christensen
<langwadt@fonz.dk> wrote:

tirsdag den 18. april 2023 kl. 15.08.39 UTC+2 skrev Don Y:
On 4/18/2023 4:36 AM, Dimiter_Popoff wrote:
On 4/18/2023 10:44, Don Y wrote:
On 4/17/2023 12:44 PM, Dimiter_Popoff wrote:
Hmmm, interesting question. Since switches do \"buffer then transmit\"
this might go either way; if the same copy is retransmitted to
all rj-45s this can be done with practically no skew (apart from
that at the receiving side, clock phases, software latencies etc.).

Remember, the switch isn\'t a wire. Instead, it can be seen as a buffer
upstream of each port. So, if port 4 is busy delivering a packet to it\'s
client (from port 7, e.g.) but port 3 is idle (because no one loves him!),
then a message intended for port 3 will be delivered before a message
(possibly the same BROADCAST message) will be delivered to port 4.

Depending on the switch architecture, a backlog at switch 4 may cause the
\"next\" message intended for it to be dropped, *in* the switch -- while
port 3 manages to receive it.
But I doubt many - if any - switches do that, my bet would be that they
transmit one cable at a time.
On the old coaxial Ethernet the answer is obvious but not much
use.

10Base2/5 preferable to <any>BaseT (*wire* instead of hub/switch).

But, even there, you have differences in the PHYs/NICs, stack, etc.
that add to the variability.

Well not as much, the PHY I remember (an NSC part) does not buffer
data so it must add just nanoseconds of delay. I meant that all
skew is receiver generated, MAC, CPU etc.
Presumably, there is a \"line of code\" *somewhere* that starts the
process. This code has to get executed and drag all of the
subsequent mechanisms in before the \"correct\" packet gets put
on the wire.

So, any variability in the execution path (e.g., conditionals,
preemption, etc.) appears in series with that \"line of code\"
just as any variability in the receiving (and transport) end
would.

it is not the code that is the main issue.

100Mbit requires a switch and it may not send broadcast packages to all the down stream ports at the same time
with 10Mbit you can use a hub so all ports hear everything thing all the time

People don\'t make hubs any more. It\'s hard to even get the chips.

 

[John Larkin]

On Tue, 18 Apr 2023 07:00:11 -0700 (PDT), Fred Bloggs
<bloggs.fredbloggs.fred@gmail.com> wrote:

On Monday, April 17, 2023 at 3:18:59?PM UTC-4, John Larkin wrote:
Suppose one were to send a broadcast packet from a PC to multiple
boxes over regular 100 Mbit ethernet, without any fancy time protocols
like PTP or anything. Each box would accept the packet as a trigger.
Assume a pretty small private network and a reasonable number of
switches to fan out to many boxes.

Any guess as to how much the effective time trigger to various boxes
would skew? I\'ve seen one esimate of 125 usec, for cameras, with
details unclear.

Too bad you can\'t set them up to do whatever it is at a specified time of day.

Sure, there are ways to do that, but they get complex on both ends.

 

[Don Y]

On 4/18/2023 9:30 AM, Lasse Langwadt Christensen wrote:

tirsdag den 18. april 2023 kl. 15.08.39 UTC+2 skrev Don Y:
On 4/18/2023 4:36 AM, Dimiter_Popoff wrote:
On 4/18/2023 10:44, Don Y wrote:
On 4/17/2023 12:44 PM, Dimiter_Popoff wrote:
Hmmm, interesting question. Since switches do \"buffer then transmit\"
this might go either way; if the same copy is retransmitted to
all rj-45s this can be done with practically no skew (apart from
that at the receiving side, clock phases, software latencies etc.).

Remember, the switch isn\'t a wire. Instead, it can be seen as a buffer
upstream of each port. So, if port 4 is busy delivering a packet to it\'s
client (from port 7, e.g.) but port 3 is idle (because no one loves him!),
then a message intended for port 3 will be delivered before a message
(possibly the same BROADCAST message) will be delivered to port 4.

Depending on the switch architecture, a backlog at switch 4 may cause the
\"next\" message intended for it to be dropped, *in* the switch -- while
port 3 manages to receive it.
But I doubt many - if any - switches do that, my bet would be that they
transmit one cable at a time.
On the old coaxial Ethernet the answer is obvious but not much
use.

10Base2/5 preferable to <any>BaseT (*wire* instead of hub/switch).

But, even there, you have differences in the PHYs/NICs, stack, etc.
that add to the variability.

Well not as much, the PHY I remember (an NSC part) does not buffer
data so it must add just nanoseconds of delay. I meant that all
skew is receiver generated, MAC, CPU etc.
Presumably, there is a \"line of code\" *somewhere* that starts the
process. This code has to get executed and drag all of the
subsequent mechanisms in before the \"correct\" packet gets put
on the wire.

So, any variability in the execution path (e.g., conditionals,
preemption, etc.) appears in series with that \"line of code\"
just as any variability in the receiving (and transport) end
would.

it is not the code that is the main issue.

100Mbit requires a switch and it may not send broadcast packages to all the down stream ports at the same time
with 10Mbit you can use a hub so all ports hear everything thing all the time

Ethernet is an UNRELIABLE medium. That\'s why you layer protocols atop it.

In most cases, protocols are implemented in software.

So, *some* host has to (optionally configure \"slaves\" and) \"broadcast\"
a \"trigger\" event. This will be a packet sent down the wire (if not,
then you could just use A WIRE with a pulse).

Let\'s PRETEND the switch doesn\'t queue packets. Does that *guarantee* that
all hosts will \"see\" the message at the same time? Are you sure EVERY port
is idle when you decide to send your message? (if not, the switch won\'t
*queue* your packet so it will be lost!)

Let\'s assume the switch queues your packet. Does *that* guarantee that
all hosts will see the message (regardless as to time)? What if the
switch\'s architecture allocates buffer space on a per-port basis and
port 3 is \"out\"?

Assume the switch is a *nominal* switch. But, port 3 has a duplex mismatch.
How much *larger* will the skew be than \"nominal\"?

As you can\'t be sure the UDP packets will be seen by all of the \"slaves\"
at the same *instant* in time, you have to preset the action that you want.
I.e., \"*at* time X, I will want you all to trigger\". You must send this
message (unicast or broadcast) prior to time X. And, early enough that
you can be *assured* it has been received and recognized.

Because ethernet is unreliable, you will have to implement a protocol
to ensure this message is received (uncorrupted) by all slaves. This
will require them to acknowledge the message AND IT\'S (local) HANDLING.

ALL OF THEM.

Time, meanwhile, keeps flowing forward. So, if you opted for X to be 10
seconds from now and then your disk started thrashing -- BEFORE you\'ve
had a chance to even broadcast the message -- then those 10 seconds are
now fewer. And, you still have to complete telling EACH node about
your plan. And, receiving their confirmations.

Each of them, in turn, can be \"distracted\" by something *in* them or
directed *at* them (e.g., network traffic).

Did I mention, time keeps flowing forward?

If your goal is to \"sort of\" make this happen, then it will \"sort of\"
work. If, OTOH, your goal is to be *sure* it will happen (like
a user would expect with a dedicated \"pulse\" sent down a trigger wire)
then you will find disappointment.

People think ethernet is plug-and-play. But, there are subtleties
that make it far too easy for folks to screw up when you\'re trying to
push the envelope in any way. (\"Oh, our old switch broke so I\'m using
this one, instead\" \"I didn\'t have a cable of that length so pieced
two shorter ones together with a coupler (or *made* one)\" etc.)

If you don\'t have control over *everything*, then you\'ll spend time
troubleshooting your customers\' networks. Which will manifest as faults in
\"your\" product.

 

[Don Y]

On 4/18/2023 8:02 AM, Dimiter_Popoff wrote:

On 4/18/2023 16:08, Don Y wrote:
On 4/18/2023 4:36 AM, Dimiter_Popoff wrote:
On 4/18/2023 10:44, Don Y wrote:
On 4/17/2023 12:44 PM, Dimiter_Popoff wrote:
Hmmm, interesting question. Since switches do \"buffer then transmit\"
this might go either way; if the same copy is retransmitted to
all rj-45s this can be done with practically no skew (apart from
that at the receiving side, clock phases, software latencies etc.).

Remember, the switch isn\'t a wire.  Instead, it can be seen as a buffer
upstream of each port.  So, if port 4 is busy delivering a packet to it\'s
client (from port 7, e.g.) but port 3 is idle (because no one loves him!),
then a message intended for port 3 will be delivered before a message
(possibly the same BROADCAST message) will be delivered to port 4.

Depending on the switch architecture, a backlog at switch 4 may cause the
\"next\" message intended for it to be dropped, *in* the switch -- while
port 3 manages to receive it.
  > But I doubt many - if any - switches do that, my bet would be that they
transmit one cable at a time.
On the old coaxial Ethernet the answer is obvious but not much
use.

10Base2/5 preferable to <any>BaseT (*wire* instead of hub/switch).

But, even there, you have differences in the PHYs/NICs, stack, etc.
that add to the variability.

Well not as much, the PHY I remember (an NSC part) does not buffer
data so it must add just nanoseconds of delay. I meant that all
skew is receiver generated, MAC, CPU etc.

Presumably, there is a \"line of code\" *somewhere* that starts the
process.  This code has to get executed and drag all of the
subsequent mechanisms in before the \"correct\" packet gets put
on the wire.

So, any variability in the execution path (e.g., conditionals,
preemption, etc.) appears in series with that \"line of code\"
just as any variability in the receiving (and transport) end
would.

I just assumed the initiation is sort of a pushed button by a person,
so all the skew comes after the broadcast packet is out.

But, code has to see that button and generate the packet.
I doubt there\'s a dedicated bit of hardware that crams a packet
into a NIC and pushes it down the wire based on \"pin 1\" going low...

If you don\'t control the hardware *and* software in each node, you\'re
just pissing in the wind and hoping not to get wet!

Of course so, but I assume John controls all receiver nodes, his
devices I have seen. Obviously he will have to rely no someone else\'s
network design but taming it to behave coherently may be doable.

Do *you* control all of the devices on the LAN your device occupies?

Well, sometimes yes. If they are all dps devices, I can control them.
Which is not to say I don\'t get surprised, stunned etc .

I assume customers have PCs sharing the same network (if not network
SEGMENT!) with your device(s). So, if you want to RELIABLY interact
with them, you have to use RELIABLE protocols.

I think it\'s pretty unlikely that a completely closed system would
bother with ethernet for triggers -- why not an optical fiber with
a pulsed light source wired to the device in question?  (or, a piece of
copper, etc.)

John mentioned some 125us, apparently this is the ballpark he is after
and wants some verification. This is a huge amount of time, especially
by his picosecond standards, if he were after something that fast/low
skew obviously he would not have thought of Ethernet.

I have considered syncing multiple MCA-s so in list mode they have
skew well below a nanosecond but the customers for that sort of thing
just don\'t call. Obviously I did not think Ethernet, either .
As it has been typical, they will call here once they have run out of
other usable options...

As you know, in my world, EVERYTHING happens between physically
separate nodes. So, a common sense of time is crucial; everything
must agree to a shared \"schedule\" else B happens before A, etc.

But, I\'ve taken steps to \"guarantee\" that.

And, when a device is discovered to have lost sync, then it is no
longer given \"time sensitive\" (which doesn\'t mean \"fast\" or \"soon\")
tasks to perform. \"Here, recode this video for me...\" \"How
about diarising this snippet for me...\"

[Again, operating with an UNRELIABLE medium means you have to design
protocols that minimize the uncertainty AND react to it when encountered!]

[I think mail is hosed, again  :< ]

I did email you earlier today (nothing worth a second thought if it
gets lost).

I\'ll check when I wake up.

 

[John Larkin]

On Tue, 18 Apr 2023 11:40:59 -0700, Don Y
<blockedofcourse@foo.invalid> wrote:

On 4/18/2023 9:30 AM, Lasse Langwadt Christensen wrote:
tirsdag den 18. april 2023 kl. 15.08.39 UTC+2 skrev Don Y:
On 4/18/2023 4:36 AM, Dimiter_Popoff wrote:
On 4/18/2023 10:44, Don Y wrote:
On 4/17/2023 12:44 PM, Dimiter_Popoff wrote:
Hmmm, interesting question. Since switches do \"buffer then transmit\"
this might go either way; if the same copy is retransmitted to
all rj-45s this can be done with practically no skew (apart from
that at the receiving side, clock phases, software latencies etc.).

Remember, the switch isn\'t a wire. Instead, it can be seen as a buffer
upstream of each port. So, if port 4 is busy delivering a packet to it\'s
client (from port 7, e.g.) but port 3 is idle (because no one loves him!),
then a message intended for port 3 will be delivered before a message
(possibly the same BROADCAST message) will be delivered to port 4.

Depending on the switch architecture, a backlog at switch 4 may cause the
\"next\" message intended for it to be dropped, *in* the switch -- while
port 3 manages to receive it.
But I doubt many - if any - switches do that, my bet would be that they
transmit one cable at a time.
On the old coaxial Ethernet the answer is obvious but not much
use.

10Base2/5 preferable to <any>BaseT (*wire* instead of hub/switch).

But, even there, you have differences in the PHYs/NICs, stack, etc.
that add to the variability.

Well not as much, the PHY I remember (an NSC part) does not buffer
data so it must add just nanoseconds of delay. I meant that all
skew is receiver generated, MAC, CPU etc.
Presumably, there is a \"line of code\" *somewhere* that starts the
process. This code has to get executed and drag all of the
subsequent mechanisms in before the \"correct\" packet gets put
on the wire.

So, any variability in the execution path (e.g., conditionals,
preemption, etc.) appears in series with that \"line of code\"
just as any variability in the receiving (and transport) end
would.

it is not the code that is the main issue.


100Mbit requires a switch and it may not send broadcast packages to all the down stream ports at the same time
with 10Mbit you can use a hub so all ports hear everything thing all the time

Ethernet is an UNRELIABLE medium. That\'s why you layer protocols atop it.

In most cases, protocols are implemented in software.

So, *some* host has to (optionally configure \"slaves\" and) \"broadcast\"
a \"trigger\" event. This will be a packet sent down the wire (if not,
then you could just use A WIRE with a pulse).

Let\'s PRETEND the switch doesn\'t queue packets. Does that *guarantee* that
all hosts will \"see\" the message at the same time? Are you sure EVERY port
is idle when you decide to send your message? (if not, the switch won\'t
*queue* your packet so it will be lost!)

Let\'s assume the switch queues your packet. Does *that* guarantee that
all hosts will see the message (regardless as to time)? What if the
switch\'s architecture allocates buffer space on a per-port basis and
port 3 is \"out\"?

Assume the switch is a *nominal* switch. But, port 3 has a duplex mismatch.
How much *larger* will the skew be than \"nominal\"?

As you can\'t be sure the UDP packets will be seen by all of the \"slaves\"
at the same *instant* in time, you have to preset the action that you want.
I.e., \"*at* time X, I will want you all to trigger\". You must send this
message (unicast or broadcast) prior to time X. And, early enough that
you can be *assured* it has been received and recognized.

Because ethernet is unreliable, you will have to implement a protocol
to ensure this message is received (uncorrupted) by all slaves. This
will require them to acknowledge the message AND IT\'S (local) HANDLING.

ALL OF THEM.

Time, meanwhile, keeps flowing forward. So, if you opted for X to be 10
seconds from now and then your disk started thrashing -- BEFORE you\'ve
had a chance to even broadcast the message -- then those 10 seconds are
now fewer. And, you still have to complete telling EACH node about
your plan. And, receiving their confirmations.

Each of them, in turn, can be \"distracted\" by something *in* them or
directed *at* them (e.g., network traffic).

Did I mention, time keeps flowing forward?

Let\'s PRETEND that you are *not* being \"deliberately\" obnoxious.

If your goal is to \"sort of\" make this happen, then it will \"sort of\"
work. If, OTOH, your goal is to be *sure* it will happen (like
a user would expect with a dedicated \"pulse\" sent down a trigger wire)
then you will find disappointment.

People think ethernet is plug-and-play. But, there are subtleties
that make it far too easy for folks to screw up when you\'re trying to
push the envelope in any way. (\"Oh, our old switch broke so I\'m using
this one, instead\" \"I didn\'t have a cable of that length so pieced
two shorter ones together with a coupler (or *made* one)\" etc.)

If you don\'t have control over *everything*, then you\'ll spend time
troubleshooting your customers\' networks. Which will manifest as faults in
\"your\" product.

Oh WELL, I was ASKING about \"sort of\" QUANTIFYING *all* of THAT for
\"my\" product.

It is harder to type in obnoxious mode.