We will be replacing the aging and increasingly problematic service radio atop Downtown Tower late tomorrow morning (Wednesday 10/10) with a new and faster unit. Expect an outage of 10-15 minutes as the old unit is unbolted and the new unit secured. This outage will affect only users served directly by Downtown Wickenburg Tower — subscribers served by all other towers will be unaffected.
We’re experiencing an issue with some of our towers in which the speed to subscribers degrades as equipment uptime increases. Towers identified so far have included Skip-In Ranch Tower, Downtown Tower, and all three APs on Morristown Tower. In all cases, power-cycling the radio unit restores full performance.
Our equipment manufacturer, MikroTik, suspects a memory leak in their OS, and has asked us to provide them with diagnostic dumps from the affected equipment, which we are in the process of assembling for them today.
While we wait for a response from MikroTik, we’ll be setting up an automatic powercycle schedule on the affected towers. This will result in an approximately four-minute interruption of service between 2:00-4:00 AM several times each week at these towers. We hope to have this periodic refresh cycle operational by Monday.
We placed an order for new armored power cabling for Mockingbird Tower late last Tuesday (UPS three-day) with a new cable dealer who loudly advertises “same-day shipping.” They didn’t actually ship our order until Friday evening (without telling us)… so the cable, originally planned to arrive Monday (Tuesday at latest), still hasn’t shown up and we don’t know what time it will arrive.
This has forced us to postpone today’s tower upgrade to next Wednesday, the 25th. With luck, we’ll be able to dodge monsoon power outages for one extra week.
This Wednesday (7/18) around mid-day, we’re going to be upgrading the power supply and power cabling at Mockingbird Tower. We’re going to continue running on the existing cabling until we get everything laid out and ready to go, at which time we’ll cut over to the new power feed. If everything goes as planned, there will be no interruption in tower service at all; if it doesn’t, subscribers served directly by Mockingbird Tower might experience at most a two-minute outage as the tower resets. (No other subscribers will be affected by this maintenance.)
Part of the upgrade includes a battery backup system designed to be able to operate the tower for over an hour, to ride out any short-term monsoon-induced power outages that may affect Mockingbird Hill. In addition, relieving our power crunch on this tower should also help improve speeds to our Rio Vista tower subscribers.
Recently, we determined that our Downtown Wickenburg Tower was exhibiting impaired performance depending on how long it was continuously operational. While awaiting the manufacturer’s diagnosis, we put a script in place to reboot this unit in the wee hours every four days, to keep it operating at peak.
The manufacturer replied today with the suggestion that reinitializing the unit from scratch should return it to proper operation. We’ll be performing this procedure around noon on Tuesday (tomorrow). This will result in a service outage of 10-15 minutes, which will affect only users directly served by Downtown Tower. We hope this will result in improved performance for our Downtown subscribers.
We’ve determined from subscriber feedback and subsequent testing that the east sector of Morristown Tower is not functioning properly and is delivering insufficient bandwidth. We’ll be replacing that unit Wednesday morning sometime between 9:15 and 10:30. This will involve an outage of 10-15 minutes, that should affect only Morristown Tower subscribers east of the 267th Avenue alignment.
Since our last post, we’ve been hard at work on a new system to deliver more bandwidth to our four neighborhood towers in west Wickenburg, and to ramp our popular Mockingbird Tower up to the speeds necessary to support our increasing subscriber levels.
Subscribers of all our towers from Mockingbird Road westward may encounter some brief (5-20 minute) network interruptions this coming Wednesday at staggered times of the day, as we perform certain of the equipment upgrades described below.
Here are some details about what we’ve been doing, how far along we’ve gotten, and what we hope to accomplish Wednesday. Continue reading
There’s about a 70% chance that we will have an opportunity this afternoon (Wednesday 1/31) to be able to perform an equipment upgrade at Diamondback Tower. Subscribers served by that tower may encounter several brief (about two minute) outages, along with possibly somewhat slower performance from the temporary mast we’ll be setting up to provide service while the original tower is being re-engineered. This work will affect only subscribers served by Diamondback Tower.
Each week, we get one or two “my Internet doesn’t work” calls from subscribers. Unless we’ve had a recent weather event (such as a wind or lightning storm), the cause is almost always never our network or equipment. Even more regrettable is when a subscriber calls to tell us their Internet has been down for a day or more, when simple remedial action by the subscriber could have restored service immediately.
Here’s a brief refresher on what to do before calling us for service: Continue reading
This week’s networking crisis is that black-hats have found another way to violate your privacy. Called the KRACK exploit, it allows hackers to spoof WiFi protocol so as to break its encryption and read your traffic.
It’s worth pointing out that this is not just a zero-day bug in some manufacturer’s implementation—it’s a defect in the WiFi standard itself, and all WiFi (802.11) encryption is vulnerable to it. It does require a hacker to be close enough to have physical access to your radio signal.
We wanted to let you know what we were doing about it, and also what you should be doing about it yourself. Continue reading