Rebooting radio in Santa Cruz mountains once a week via GitH

H

harry newton

Guest
If we wanted to automatically reboot our radios once a week, should we do
it with a GitHub script on the radio or do you know of a better way?
<https://github.com/stevejenkins/unifi-linux-utils>

We're on a classic small WISP outfit (2-man operation, maybe 100 homes) in
the Santa Cruz mountains, where, whenever we call for technical support,
they *always* tell us to reboot the radio - where - shockingly - often -
the performance goes up (don't ask me why or how - it just does).

So let's not argue the fact that our WISP Internet speeds are better (for
whatever reason) after we reboot our rooftop radios (which most of you
would call a "modem" but we call them transceivers, or just radios, for
short).

Let's just ask the question.

If we wanted to automatically reboot our radios once a week, can we do it
with a GitHub script on the radio or do you know of a better way?

For example, how does this GitHub script look to you Linux coders like
Marek Novotny and William Unruh?
<https://github.com/stevejenkins/unifi-linux-utils>
 
I am of the simple-minded, low-tech approach school. Does "rebooting" consist of some sort of power-down-power-up?

If so, get a timer that shuts off power at some specific time each week for some specified period of time.

https://images.homedepot-static.com/productImages/5b276b93-95b8-47ad-8ea2-348cec54411c/svn/gray-intermatic-timers-et8015c-64_1000.jpg

This one will do nicely. 30A should be able to handle the load in any case. Set it for 3:00 am of a Tuesday night and see how it goes. Unless some of your customers are running SETI, they should never know.

Peter Wieck
Melrose Park, PA
 
In <oufj4p$1n7b$1@gioia.aioe.org> harry newton <harry@is.invalid> writes:

If we wanted to automatically reboot our radios once a week, should we do
it with a GitHub script on the radio or do you know of a better way?
https://github.com/stevejenkins/unifi-linux-utils

If you're just looking for a periodic reboot then the
simplest way would be a 7 day timer with a poewr-down
at (say) 03:45 Thursday morning. Alternaitvely, you
could shut thing down Mon, Wed, and Sat.

If you're looking for "on demand", then there are
numerous options available using cell phones.

--
_____________________________________________________
Knowledge may be power, but communications is the key
dannyb@panix.com
[to foil spammers, my address has been double rot-13 encoded]
 
On 2017-11-14, harry newton <harry@is.invalid> wrote:
If we wanted to automatically reboot our radios once a week, should we do
it with a GitHub script on the radio or do you know of a better way?
https://github.com/stevejenkins/unifi-linux-utils

looks like it could work, if you've changed the admin passwords on your
radios you'll need to update the script. if not you probably should.

one reason why rebooting may help is if you skipped the firmware
release early this year, your radios may be being subverted by IOT
malware. (there was a more recent patch for wifi insecurity - that
should be applied too.





We're on a classic small WISP outfit (2-man operation, maybe 100 homes) in
the Santa Cruz mountains, where, whenever we call for technical support,
they *always* tell us to reboot the radio - where - shockingly - often -
the performance goes up (don't ask me why or how - it just does).

So let's not argue the fact that our WISP Internet speeds are better (for
whatever reason) after we reboot our rooftop radios (which most of you
would call a "modem" but we call them transceivers, or just radios, for
short).

Let's just ask the question.

If we wanted to automatically reboot our radios once a week, can we do it
with a GitHub script on the radio or do you know of a better way?

For example, how does this GitHub script look to you Linux coders like
Marek Novotny and William Unruh?
https://github.com/stevejenkins/unifi-linux-utils

--
This email has not been checked by half-arsed antivirus software
 
On Tue, 14 Nov 2017 20:19:38 +0000 (UTC), harry newton
<harry@is.invalid> wrote:

If we wanted to automatically reboot our radios once a week, should we do
it with a GitHub script on the radio or do you know of a better way?

I use a AC lamp timer to power cycle everything:
<https://www.intermatic.com/en/timer-controls>
Pick your flavor, they all work. I've been using these:
<https://www.intermatic.com/en/timer-controls/plug-in-timers/dt620mx>
The main problem is that the tiny button batteries don't last long
enough.

The problem is that I don't trust software solutions to fix a problem
caused by an operating system or application that has gone insane. The
reset has to be done by something that won't be trashed, hung, or
become too busy, which means an external device or independent
"heartbeat" timer.

Before you complain about crashing your system by killing the power,
please note that if your stuff can't handle a power glitch or failure,
it doesn't deserve to be run unattended.



--
Jeff Liebermann jeffl@cruzio.com
150 Felker St #D http://www.LearnByDestroying.com
Santa Cruz CA 95060 http://802.11junk.com
Skype: JeffLiebermann AE6KS 831-336-2558
 
On 11/14/17 21:19, harry newton wrote:
If we wanted to automatically reboot our radios once a week, should we do
it with a GitHub script on the radio or do you know of a better way?
https://github.com/stevejenkins/unifi-linux-utils

Hope you really have another admin user/password than ubnt.

If we wanted to automatically reboot our radios once a week, can we do it
with a GitHub script on the radio or do you know of a better way?

If you want the reboot to occur periodically, then a crontab job would
be simplest.

If you want more control, then have periodical checks of a site, and if
the value changes on the requested page say from 0 to 1, then reboot
(then have a grace period when it will not check the page, or else it
will keep rebooting until you change the value back to 0).

If making the page with some kind of logic, then each client sends a
hash unique for them to you, if the value is in your database, then send
value 0, if not send 1 and each time you want all units to reboot you
empty the table.

Sure you could use nrpe to do the same thing as your ssh connection, but
with the difference that you wouldn't need to know the password of the
device, lock it with ip restriction. This will work quite nicely till
someone does an arp poisoning and make it look for the device that you
would be sending the nrpe request while it's someone else.
I have used nrpe where the client been allowing request from a dynamical
ip server, this has the small drawback that you have a minute while you
can't access the nrpe service, but I do rather use it than ssh which
requires you to configure username/password in nagios/icinga.

--

//Aho
 
On Tue, 14 Nov 2017 21:17:02 -0800, in
<news:q1jn0d5n211v1msf4c25b9mvbtdhlscaiu@4ax.com>, Jeff Liebermann wrote:

The problem is that I don't trust software solutions to fix a problem
caused by an operating system or application that has gone insane. The
reset has to be done by something that won't be trashed, hung, or
become too busy, which means an external device or independent
"heartbeat" timer.

Klugey approach?

Set a WatchDog Timer reboot.

Specify an IP that won't respond to pings, set up the WatchDog timer to
ping it every 24*60*60 seconds, with a fail count of 7. (or suitable
numbers that the GUI will accept).
 
On Wed, 15 Nov 2017 07:22:29 -0000 (UTC), Blake Snyder wrote
in response to Blake Snyder <blakdeblakesnyder@outlook.com>

Klugey approach?

Set a WatchDog Timer reboot.

Specify an IP that won't respond to pings, set up the WatchDog timer to
ping it every 24*60*60 seconds, with a fail count of 7. (or suitable
numbers that the GUI will accept).

You could even have it ping a machine on your network somewhere that you
could take off-line when you wanted to reboot everything.
 
On 2017-11-15 08:25, ATANARJUAT wrote:
On Wed, 15 Nov 2017 07:22:29 -0000 (UTC), Blake Snyder wrote
in response to Blake Snyder <blakdeblakesnyder@outlook.com

Klugey approach?

Set a WatchDog Timer reboot.

Specify an IP that won't respond to pings, set up the WatchDog timer to
ping it every 24*60*60 seconds, with a fail count of 7. (or suitable
numbers that the GUI will accept).

You could even have it ping a machine on your network somewhere that you
could take off-line when you wanted to reboot everything.

Do you know of a reasonably cheap hardware device that can monitor
something on the network, and powercycle it when needed? A watchdog that
acts on a hung device, say.

I know one or two, but they are expensive.
A timer reboot is too aggressive when a reboot is not needed.


--
Cheers, Carlos.
 
Carlos E.R. wrote:

On 2017-11-15 08:25, ATANARJUAT wrote:
On Wed, 15 Nov 2017 07:22:29 -0000 (UTC), Blake Snyder wrote
in response to Blake Snyder <blakdeblakesnyder@outlook.com

Klugey approach?

Set a WatchDog Timer reboot.

Specify an IP that won't respond to pings, set up the WatchDog timer to
ping it every 24*60*60 seconds, with a fail count of 7. (or suitable
numbers that the GUI will accept).

You could even have it ping a machine on your network somewhere that you
could take off-line when you wanted to reboot everything.

Do you know of a reasonably cheap hardware device that can monitor
something on the network, and powercycle it when needed? A watchdog that
acts on a hung device, say.

I know one or two, but they are expensive.
A timer reboot is too aggressive when a reboot is not needed.

Use a Raspberry
 
He who is Peter K+APY-hlmann said on Wed, 15 Nov 2017 14:26:20 +-0100:

You could even have it ping a machine on your network somewhere that you
could take off-line when you wanted to reboot everything.

Do you know of a reasonably cheap hardware device that can monitor
something on the network, and powercycle it when needed? A watchdog that
acts on a hung device, say.

I know one or two, but they are expensive.
A timer reboot is too aggressive when a reboot is not needed.


Use a Raspberry

The idea of using a Raspberry Pi would work!

It's actually probably perfect in that it's highly programmable, and it has
all the necessary WiFi, Ethernet, and USB connections!

I did try out the WatchDog feature of all Ubiquiti radios.
Here's my Rocket M5, for example, which has 28 dBm of transmit power into a
30 dBm dish which means I can pickup or throw my Internet for ten miles.
<http://wetakepic.com/images/2017/11/15/Clipboard01c7b07.jpg>

The Ping Watchdog facility was unused when I looked just now:
<http://wetakepic.com/images/2017/11/15/Clipboard02.jpg>

I turned the Ping Watchdog on, and set it to 86,400 seconds (1 day), and
then set the ping failure number to 7 days.
<http://wetakepic.com/images/2017/11/15/Clipboard03.jpg>

The question is what IP address to ping that doesn't exist on the net?
It wouldn't take 192.168.1.256
<http://wetakepic.com/images/2017/11/15/Clipboard04.jpg>

So I picked an IP address that doesn't exist on my net of 192.168.1.254
<http://wetakepic.com/images/2017/11/15/Clipboard05.jpg>

This only works if you have the login to the radio, which most people on
WISP do not have access to though. So the WISP admin has to set it up.
 
On Wed, 15 Nov 2017 07:22:29 -0000 (UTC), Blake Snyder
<blakdeblakesnyder@outlook.com> wrote:

On Tue, 14 Nov 2017 21:17:02 -0800, in
news:q1jn0d5n211v1msf4c25b9mvbtdhlscaiu@4ax.com>, Jeff Liebermann wrote:

The problem is that I don't trust software solutions to fix a problem
caused by an operating system or application that has gone insane. The
reset has to be done by something that won't be trashed, hung, or
become too busy, which means an external device or independent
"heartbeat" timer.

Klugey approach?

It's spelled kludge. In the not so distant past, I helped maintain a
series of mountain top weather stations. Service calls were expensive
and best avoided. As an added bonus, this was in an environment full
of RF pollution.

>Set a WatchDog Timer reboot.

Sure, if the watchdog timer is independent of what it's monitoring.
Long ago Kantronics KPC-2 TNC (packet radio modem) had a built in
watchdog timer. Too bad it was all software and located in the same
chip it was monitoring. When the KPC-2 hung, the watchdog timer also
hung. In later models, they simply removed the watchdog timer.

Roll forward a few years, and I'm maintaining servers in a big server
farm. Remote reboot via ethernet was problematic. It was quite
common to arrive at the ISP and find a message declaring that the OS
refuses to reboot until some obscure process agrees to die gracefully.
The customer got tired of paying me to reboot his servers, and I got
tried of driving 50 miles to flip a switch. So, I install a paging
receiver and decoder to initiate a reboot. That was quite a challenge
as server farms are full of RF interference.

However, even that didn't quite work. It seems that most servers have
a "feature" called WOL (Wake on LAN) that allows me to remotely power
on the server. In order to do that, it needs to have the power left
on to the LAN card(s) even when the server is turned off. (Note: WOL
is mostly used for desktops, but at the time was also appearing in
servers). Sometimes, the ethernet card would hang. If I reboot the
machine, the LAN card would remain hung. If I flipped the power
on/off switch on the server, it still would remain hung. Of course,
with no connectivity, I couldn't do a remote reboot in software.
Compaq later introduced a server management card that provided a
secondary management channel, but it was too expensive. The only good
solution was to pull the plug on the server.

For server farms, I eventually went to SNMP managed remote power
switches. I still have a bunch of APC AP9211 switches in service.
<https://www.google.com/search?q=apc+ap9211&tbm=isch>
Primary control is via ethernet, but some had a secondary control
channel via the serial port.

I've tried other schemes and solutions. Some worked, but all had a
surprise hidden somewhere.

Specify an IP that won't respond to pings, set up the WatchDog timer to
ping it every 24*60*60 seconds, with a fail count of 7. (or suitable
numbers that the GUI will accept).

Won't work and may I humbly suggest that you think about this a bit
more. The problem is that in any ISO layer cake device, it is
possible to have the lower layers working, while the upper layers are
hung or stuck in a spin look making the lower layers too busy to
respond. I've seen machines that respond quite nicely to ICMP pings
where the main function (email or web server) is totally hung. For
these things, you need to test the higher level services and not rely
on the lower levels.

In this situation, there's one big advantage that MIGHT make such a
simple ping work with a wireless link. All wireless connectivity is
done at layer 2 (MAC layer). The IP layer is only involved in
managing the device. If one pings by IP address, it's fairly good
assumption that the underlying layers are working. Some services such
as SNMP, SMTP for email fault notification, and the usual internal web
server might be hung, but with layer 2 still working, the wireless
bridge would likely still be doing its job.

Now, back to the original problem. Heartbeat timers and timed reboots
are a kludge. They're needed because the manufacturer of the wireless
bridge radios didn't do a decent job of keeping his hardware up and
running 24x7. The failure might be in software, susceptibility to
power glitches, susceptibility to DoS attacks, crappy components, or
environmental issues (overheating). If the wireless is bridge is so
unreliable that it has to be rebooted once per week, I suggest you
look into the cause of this unreliability, and not apply a band-aid.


--
Jeff Liebermann jeffl@cruzio.com
150 Felker St #D http://www.LearnByDestroying.com
Santa Cruz CA 95060 http://802.11junk.com
Skype: JeffLiebermann AE6KS 831-336-2558
 
On Wed, 15 Nov 2017 16:35:15 +0000 (UTC), harry newton
<harry@is.invalid> wrote:

><http://wetakepic.com/images/2017/11/15/Clipboard05.jpg>

One of these daze, I should read the Ubiquiti instructions. I didn't
know it had a built in watchdog reboot feature. Same warning as with
the Kantronics TNC. If the watchdog feature lives in the same
hardware that it's trying to monitor, then chances are good that if
one hangs, so will the other.

I've also had the reboot timer in DD-WRT screw up on me a few times,
but that was long ago.

Bah Humbug (T'is almost the season).

--
Jeff Liebermann jeffl@cruzio.com
150 Felker St #D http://www.LearnByDestroying.com
Santa Cruz CA 95060 http://802.11junk.com
Skype: JeffLiebermann AE6KS 831-336-2558
 
On 2017-11-15 14:26, Peter KĂśhlmann wrote:
Carlos E.R. wrote:

On 2017-11-15 08:25, ATANARJUAT wrote:
On Wed, 15 Nov 2017 07:22:29 -0000 (UTC), Blake Snyder wrote
in response to Blake Snyder <blakdeblakesnyder@outlook.com

Klugey approach?

Set a WatchDog Timer reboot.

Specify an IP that won't respond to pings, set up the WatchDog timer to
ping it every 24*60*60 seconds, with a fail count of 7. (or suitable
numbers that the GUI will accept).

You could even have it ping a machine on your network somewhere that you
could take off-line when you wanted to reboot everything.

Do you know of a reasonably cheap hardware device that can monitor
something on the network, and powercycle it when needed? A watchdog that
acts on a hung device, say.

I know one or two, but they are expensive.
A timer reboot is too aggressive when a reboot is not needed.


Use a Raspberry

Yes, that's one of the methods I'm considering for when I have the time.

--
Cheers, Carlos.
 
On 2017-11-15, Carlos E.R. <robin_listas@es.invalid> wrote:
On 2017-11-15 08:25, ATANARJUAT wrote:
On Wed, 15 Nov 2017 07:22:29 -0000 (UTC), Blake Snyder wrote
in response to Blake Snyder <blakdeblakesnyder@outlook.com

Klugey approach?

Set a WatchDog Timer reboot.

Specify an IP that won't respond to pings, set up the WatchDog timer to
ping it every 24*60*60 seconds, with a fail count of 7. (or suitable
numbers that the GUI will accept).

You could even have it ping a machine on your network somewhere that you
could take off-line when you wanted to reboot everything.

Do you know of a reasonably cheap hardware device that can monitor
something on the network, and powercycle it when needed? A watchdog that
acts on a hung device, say.

I know one or two, but they are expensive.
A timer reboot is too aggressive when a reboot is not needed.

I used transitor driven relay wired to the RTS line of a serial port.
open the serial device and the relay switches on, I used the normally-
closed contacts to interrupt the DC supply to the flakey device (said
DC supply also powered the relay)

A cron job would check the status of the internet and do "sleep 10 <
/dev/ttyS0" when a hard reboot was needed.

--
This email has not been checked by half-arsed antivirus software
 
He who is Jeff Liebermann said on Thu, 16 Nov 2017 16:03:27 -0800:

One of these daze, I should read the Ubiquiti instructions. I didn't
know it had a built in watchdog reboot feature. Same warning as with
the Kantronics TNC. If the watchdog feature lives in the same
hardware that it's trying to monitor, then chances are good that if
one hangs, so will the other.

I've also had the reboot timer in DD-WRT screw up on me a few times,
but that was long ago.

For what I need, the watchdog feature "should" work fine, since if it's
actually hung, I can always manually pull the POE for a minute (or put it
on a weekly or daily timer).

PS: The rains are coming.
 
On 2017-11-15, Jeff Liebermann <jeffl@cruzio.com> wrote:
On Wed, 15 Nov 2017 07:22:29 -0000 (UTC), Blake Snyder
blakdeblakesnyder@outlook.com> wrote:

For server farms, I eventually went to SNMP managed remote power
switches. I still have a bunch of APC AP9211 switches in service.
https://www.google.com/search?q=apc+ap9211&tbm=isch
Primary control is via ethernet, but some had a secondary control
channel via the serial port.

I've got a server on another continent that I need to do an
operating-system reinstall on. it's a 3 year old supermicro with the
IPMI having a java-webstart based remote console. at some level it
uses RDP, but seems to only work via the java web app.

the IPMI allows viewing the console, turing power on and off, hard reset,
turnging the identity LED on and off, mounting network files as boot
media. etc.

I'm going to try a PXE boot of Debian install medis from a server in the same
rack. (the virtual medis boot is poorly documented)

All this remote admin stuff is pretty insecure and, so runs on LAN
addresses, not puclic IP addresses.

The moral of this story, spend extra for server grade SSDs, don't use
laptop drives.

--
This email has not been checked by half-arsed antivirus software
 
On 2017-11-17 10:51, Jasen Betts wrote:
On 2017-11-15, Carlos E.R. <robin_listas@es.invalid> wrote:
On 2017-11-15 08:25, ATANARJUAT wrote:
On Wed, 15 Nov 2017 07:22:29 -0000 (UTC), Blake Snyder wrote
in response to Blake Snyder <blakdeblakesnyder@outlook.com

Klugey approach?

Set a WatchDog Timer reboot.

Specify an IP that won't respond to pings, set up the WatchDog timer to
ping it every 24*60*60 seconds, with a fail count of 7. (or suitable
numbers that the GUI will accept).

You could even have it ping a machine on your network somewhere that you
could take off-line when you wanted to reboot everything.

Do you know of a reasonably cheap hardware device that can monitor
something on the network, and powercycle it when needed? A watchdog that
acts on a hung device, say.

I know one or two, but they are expensive.
A timer reboot is too aggressive when a reboot is not needed.

I used transitor driven relay wired to the RTS line of a serial port.
open the serial device and the relay switches on, I used the normally-
closed contacts to interrupt the DC supply to the flakey device (said
DC supply also powered the relay)

A cron job would check the status of the internet and do "sleep 10
/dev/ttyS0" when a hard reboot was needed.

True, but I don't have any computer on that room.

Have a look at these:

http://www.hw-group.com/products/ip_watchdog/index_lite_es.html
https://www.hw-group.com/products/ip_watchdog/ip-watchdog2_Lite_en.html

They are the exact thing, but expensive for my needs.

There are domotic switches, but those I find are controlled remotely via
a phone app - however, as the device that hangs in my house is precisely
the router, there is no Internet and those switches would not work
remotely. I need something that acts on its own.


I could use an arduino, but I'd have to learn how - which might be
interesting, anyhow.

The best and cheaper seems to be switches than have the chip "ESP8266
sonoff" and can be reflashed. But again, I need to learn the procedure
and the programming.

--
Cheers, Carlos.
 
On Wed, 15 Nov 2017 16:25:57 +0900, ATANARJUAT
<atanarjuat@Inuksuk.net> wrote:

On Wed, 15 Nov 2017 07:22:29 -0000 (UTC), Blake Snyder wrote
in response to Blake Snyder <blakdeblakesnyder@outlook.com

Klugey approach?

Set a WatchDog Timer reboot.

Specify an IP that won't respond to pings, set up the WatchDog timer to
ping it every 24*60*60 seconds, with a fail count of 7. (or suitable
numbers that the GUI will accept).

You could even have it ping a machine on your network somewhere that you
could take off-line when you wanted to reboot everything.

Or query the SNMP "time since restart" and force a rest when the
monitored device has been restarted?

On a cisco router (and lots of other infrastructure tin) you can set
"reboot after xxx minutes"
- very useful when reconfiguring remotely something that may affect
the in band link the control traffic arrives on
- as long as you remember to cancel it when you are done......

however a timer with a real power cycle is going to catch "stuff" that
a reset may not - some hardware lock ups have been known to need a
physical power cycle to recover.....

--
Stephen
 
On Fri, 17 Nov 2017 09:58:09 +0000 (UTC), harry newton
<harry@is.invalid> wrote:

>PS: The rains are coming.

Yeah right. A few days ago, it was suppose to rain Sunday and Monday.
Yesterday, the prediction was Monday only. Today, it's no rain.
See the "NOAA Weather Prediction Center"
<http://www.wpc.ncep.noaa.gov>
Mouse over the "Day 1, Day 2, Day 3" just above the weather map.
Notice how the southern boundary of the predicted rainfall stops just
south of San Francisco Bay.

Must be a conspiracy. I want my rain.

--
Jeff Liebermann jeffl@cruzio.com
150 Felker St #D http://www.LearnByDestroying.com
Santa Cruz CA 95060 http://802.11junk.com
Skype: JeffLiebermann AE6KS 831-336-2558
 

Welcome to EDABoard.com

Sponsor

Back
Top