In its report, Impact of the June 2012 Derecho on Communications Networks and Services, the FCC noted that the communications failures were due “in large part [to] avoidable planning and system failures, including the lack of functional backup power, notably in central offices.”
To address these issues, the FCC recommended maintaining sufficient backup power at central offices, and following best practices regarding maintenance and auditing of telecommunications systems.
Backup power. The most significant issue, according to the report, was the failure to have backup power at central 911 locations. For example, at the Arlington, Virginia, central office, the backup power requires two generators to run the office. However, one of the generators failed after air entered the fuel system. This caused the second generator to become overheated and shut down. Personnel attempted to bring in a mobile generator but could not obtain the unit until after power had been restored.
This power failure was especially crucial because the Arlington office was responsible for monitoring the status of equipment at 34 other sites. According to the report: “Thirty minutes after losing commercial and generator power in Arlington, and shortly after the rest of the Arlington office went on battery power, Verizon lost the ability to monitor its network in northern Virginia, crippling its repair efforts and ability to receive alarms that signal additional equipment failure.”
Because of this incident and others like it, the FCC recommended that communications providers “maintain robust, resilient backup power in central offices, supported by appropriate testing, maintenance, and records retention.”
The FCC report notes that many 911 system failures would have been avoided had telecommunications companies simply adhered to existing industry best practices as spelled out in guidance issued in 2011 by the agency’s Communications Security, Reliability and Interoperability Council (CSRIC). These best practices were designed to ensure “optimal security and reliability of communications systems, including telecommunications, media, and public safety.”
The best practices give guidance on routine maintenance, staffing, auditing, and power redundancy.
New technology. Though the report dealt primarily with existing technology, it does concede that new IP-based systems have advantages that make them more resilient, including greater redundancy and reliability. The FCC report noted that, had these technologies been in place, “they likely could have significantly lessened the derecho’s impact on emergency communications.”