Our router manufacturer issued a software upgrade on Monday to close an “exploitable hole” that put the security of our network and your data at risk. Unfortunately, they issued the fix first in the “new features” release chain, and were delayed issuing it in the “current bugfix” release chain. Unwilling to delay, we took a leap of faith and installed the “new features” release to get the security hole closed as promptly as possible.
Our faith was betrayed.
We are currently re-installing the (finally available) “current bugfix” release with the security patch on all the routers in our network. Since this is technically a downgrade, the installation is much less automatic and much more labor-intensive than the original upgrade, needing to be performed manually on upwards of 200 machines, certain of which are barely communicating well enough at the moment to load the software. We ask your patience while we back out the misbehaving wireless software suite installed earlier this week.
This week our tower bandwidth improvement project is for all practical purposes complete. Read on to learn about the improvements we’ve made to our performance in Wickenburg and Morristown… plus a little bit of nostalgia in celebration of our 11th anniversary. Continue reading →
We’ve determined from subscriber feedback and subsequent testing that the east sector of Morristown Tower is not functioning properly and is delivering insufficient bandwidth. We’ll be replacing that unit Wednesday morning sometime between 9:15 and 10:30. This will involve an outage of 10-15 minutes, that should affect only Morristown Tower subscribers east of the 267th Avenue alignment.
Since our last post, we’ve been hard at work on a new system to deliver more bandwidth to our four neighborhood towers in west Wickenburg, and to ramp our popular Mockingbird Tower up to the speeds necessary to support our increasing subscriber levels.
Subscribers of all our towers from Mockingbird Road westward may encounter some brief (5-20 minute) network interruptions this coming Wednesday at staggered times of the day, as we perform certain of the equipment upgrades described below.
Here are some details about what we’ve been doing, how far along we’ve gotten, and what we hope to accomplish Wednesday. Continue reading →
Subscribers served by Constellation Tower (this includes subscribers served by Diamondback, County Line, Black Mountain, and Easy Street Towers) will experience several short micro-outages around mid-day, as equipment and cables are swapped to upgrade Constellation Tower to gigabit bandwidths. The outages will affect almost all Wickenburg subscribers west of Burger King (except for those served directly by Mockingbird and Downtown Towers), and should last at most a minute or two; several micro-outages will occur before work is complete.
Watch for a posting later on this week in which we will discuss the infrastructure improvements we’ve been making over the past month to provide you faster service.
As a result of a long-term project accomplished earlier this week, subscribers are now seeing faster speeds and improved response times. Most subscribers are experiencing sustained speeds exceeding their contract limits (some are seeing double their contract limit), with an additional speed boost layered on top whenever possible for brief, non-sustained transactions such as web surfing. Continue reading →
I’m happy to report that, as far as we have been able to determine, all our speed-related problems at Mockingbird Tower are now resolved. Assuming this announcement doesn’t generate a spate of contrary responses, our next task will be to arrange for incremental UPS protection at our towers, beginning with the ones that are most highly subscribed; and working with our consultants to assure that we can deliver to our subscribers all the untapped bandwidth currently available at our Internet gateway.
For the technically curious, a brief description of the source of our speed problems appears below. (Warning: unpaved roads ahead!) Continue reading →
We found and corrected the source of the frequent four-second timeout pauses. Mockingbird subscribers will now at least be seeing consistent data delivery, just not yet at the speeds we would like to deliver. What the tower is delivering right now should be fine for browsing, mail, and even a Netflix movie, but heavy data demands like Windows/Mac updates will still take longer than usual.
We are continuing to work on the speed problem with our dedicated and knowledgeable consultants.
After last night’s testing, we suspect the most probable cause of the performance problem at Mockingbird is a bad customer roof unit radio hashing our tower. We’ve exhausted all the minimally invasive tests we can run. Starting tonight, we’re running a Big Sleep test on all our Mockingbird subscriber units. Continue reading →
Condensed version: We’ve narrowed down the Mockingbird Tower problem to RF noise that spikes periodically. Our tower equipment and configuration are clean and validated. We now need to perform after-midnight testing over the next few days that will result in brief (10-20 minute) interruptions of service for a number of the subscribers on this tower.