After last night’s testing, we suspect the most probable cause of the performance problem at Mockingbird is a bad customer roof unit radio hashing our tower. We’ve exhausted all the minimally invasive tests we can run. Starting tonight, we’re running a Big Sleep test on all our Mockingbird subscriber units. Continue reading
Condensed version: We’ve narrowed down the Mockingbird Tower problem to RF noise that spikes periodically. Our tower equipment and configuration are clean and validated. We now need to perform after-midnight testing over the next few days that will result in brief (10-20 minute) interruptions of service for a number of the subscribers on this tower.
Details and analysis follow for the technically curious. Continue reading
Mockingbird Tower subscribers (only) will experience a short service outage of 15-30 minutes this afternoon (Saturday), as the tower is being taken down for regrounding and recabling work to help address the RF issues causing poor performance from this tower.
After several abortive ordering attempts (in which “in stock” on a website listing apparently translated to “let us check our warehouse” once the order was placed), we located a dealer with a replacement antenna actually in stock. We have been verbally assured that our replacement unit shipped yesterday, and are awaiting written confirmation including a tracking number. In the absence of nasty surprises, we plan to be replacing the Mockingbird central antenna this weekend.
We are also going to be subjecting the Rio Vista antenna (same manufacturer and model) to testing to determine if it shows any early symptoms of the same failure mode, in case a prophylactic replacement is indicated.
Today’s status for Mockingbird Tower subscribers…
The battle continues on the Mockingbird speed issue.
In the past two days, our consultant pointed us towards two problems that allowed us to more than double the bandwidth capacity between our gateway and the Mockingbird Tower. Maddeningly, this increased bandwidth availability is still not making its way out to actual tower subscribers, and we are still at a loss as to why not. (Note that this is not happening on our other towers.) Our next step is to install some more sophisticated metering software at our gateway for our consultant to use to analyze traffic and routing issues within our network.
Today the new, faster, weatherized router model we’ve been awaiting (for months) to order, to replace the mid-2013 board currently operating Mockingbird Tower, finally hit US distributors’ shelves. We express-ordered one, which should arrive late on Friday. We plan to get it installed over the weekend, possibly solving our problem in whole or in part. (At the very worst, it will allow us to rule out a router hardware problem.) This installation will involve a brief outage, as cables have to be moved en masse from one unit to the other.
We wanted to let you know how the work is progressing, and that the Mockingbird speed issue is still our top priority… to the point where we have suspended all new subscriber installations to Mockingbird Tower until we have got it beat.
Thanks for your patience while we conquer this problem.
Subscribers homed to Mockingbird Tower (both Central and West) will experience a short interruption of service tomorrow (Tuesday) morning, as we rotate the mast down to replace the radio and pigtails for the Mockingbird Central antenna in order to attempt to alleviate our recent performance issues. We will also be correcting the tower’s cant at this time.
Should the problem turn out to be in the physical antenna itself (a much lower probability, but possible), time will be required to locate and ship a replacement for this unit, which appears to be in short supply due to a manufacturer merger. If this is the case, we’ll keep you informed via this blog.
The last two nights of record monsoon windstorms have caused us an unusual number of headaches.
The latest: the feed radio at the Easy Street neighborhood tower appears to have been blown a bit off aim by the recent high winds. Easy Street subscribers may see slower than usual network performance until we can dispatch a technician to reaim the unit.
As part of the process of analyzing a strange signal performance problem affecting a handful of Mockingbird tower subscribers, we have determined that the recent heat wave has actually warped the Mockingbird mast. The bend is causing eastbound signal to impact the terrain in Rio Vista Hills, creating destructive interference, increasing packet retries, and slowing down internet speeds to a few subscribers located on or near the 295th Avenue corridor in Rio Vista Hills and served by the Mockingbird tower. This problem does not affect other subscribers of Mockingbird Tower, nor does it affect subscribers in Rio Vista Hills served by the Rio Vista tower.
We are scheduling maintenance on the tower as heat forecasts and crew schedules allow. We hope to have the signal anomaly corrected by the middle to the end of next week.