Home > Articles > Networking

Hurricane Sandy and Widespread Telecom Outages: How Poor Disaster Recovery Measures Made the Aftermath Even Worse

In the wake of Hurricane Sandy, Leo Wrobel expalins how the reliability of wireless service in a disaster has become increasingly critical as more Americans ditch their landlines in favor of cell phones.
Like this article? We recommend

 

The lowly copper cable telephone landline has been the Rodney Dangerfield of technology lately; it doesn’t “get any respect.” This was true at least until Hurricane Sandy struck.

The U.S. population has converted en masse from landlines to wireless services over the past ten years. Every day more and more people dump their landlines in favor of modern, convenient wireless services. The percentage of cell phone-only households has risen from 18 percent in 2008 to 34 percent today, according to the National Center for Health Statistics. As communications have moved from the security of the wired environment to a wireless environment, the traditional standards designed to safeguard phone services have taken a back seat. Despite the urging of regulatory authorities to think ahead and plan for disasters, these watchdogs have been paper tigers, and largely powerless to influence an industry that considers itself “unregulated.” As a result, people and business suffer the consequences of the wireless industry’s inaction through widespread communications outages both during and after storms and other disasters. According to The Federal Communications Commission, at one point, 25% of cell towers in Hurricane Sandy’s path were affected by the storm.

Could something have been done about this issue in advance? The answer is a resounding yes, but it’s probably going to take activist regulatory action. After Hurricane Katrina knocked out communications along the Gulf Coast, federal regulators proposed that wireless companies have backup power at all cell towers. Some wireless industry providers sued to block the requirement, saying it would be a financial burden. The carriers also argued regulators did not have authority to impose such rules.¹ The Cellular Telephone Industries Association (CITA), was at the forefront of this effort.²

The widespread failures experienced by all providers—and in particular the wireless providers—has resurfaced questions of whether carriers should be required by Federal and State regulators to make their networks more fault tolerant. Harold Feld, senior vice president at Public Knowledge, a public interest group, commented that “The biggest issue is they have not wanted to invest the money in hardening their networks sufficiently against a catastrophic event.”

Things were not always this way. Less than a generation ago, the central offices that served traditional telephone service were engineered to allow for only an hour of down time every twenty years. That big black Clark Kent rotary dial phone on your kitchen wall rarely failed, regardless of weather. First, it was not powered inside your home. It received its power from the central office. The rigid engineering criteria for the central office meant that backup power was not only available but also plentiful. Three days of station battery in a central office was the minimum. Many serving central offices had a backup diesel generator, stored diesel fuel, and rotated their in-ground stock of fuel by periodic generator testing.

The Recent Landscape of Telecom Outages

So what happened in the wake of hurricane Sandy?

Four years ago almost to the day, my co-author and I wrote about a similar situation here in Texas when Hurricane Ike scored a direct hit on the city of Houston. The eye of the storm passed right over downtown. Ike was a strong Category 2 storm when it made landfall, leaving Galveston a ghost town. Like Sandy, mobility became a problem and getting food, water, and other essentials into the area was difficult. The 4.5 million residents of Houston fared slightly better, but roughly 90% lost power. Despite all of this, one forward-thinking telecommunications company, Cytel, stayed on the air throughout. Read their story here.

The difference is one of philosophy. Mission critical data centers must employ adequate physical and technology safeguards. These include not only backup equipment and technology but also written Operating and Security Standards on how and why to employ them. Can you imagine the data center for your bank not having backup power? Neither can anyone else. That’s why they have had these systems for decades.

When mission critical applications move out from the relative security of a large data center to an alternate arrangement—for example to distributed LAN platforms—the protective systems or their analog must still move out with them. Absent these precautions, “windows of vulnerability” inevitably develop when technology changes, applications migrate, and standards do not keep up. This is precisely what has happened when telecommunications went wireless. Over the last decade, communication has migrated from a regulated, well organized, and backed up system to one that in many cases does not even provide backup power. Many of those communications applications are mission critical.

Additional Complications

You may find that many Competitive Local Exchange Carriers (CLECs) and long distance companies are congregated in just a few “carrier hotels” or that they derive all of their connections to the Bell network in the same access tandem downtown. Indications are that this physical congestion of carriers contributed to the havoc following Sandy.

This concentration of facilities is not always the competitor’s fault. Oftentimes the monopoly incumbent makes it difficult for competitors by making switch diversity, “ROTN” (Route Other Than Normal) cable routing, fiber optic ring technology or “dark” fiber cost prohibitive or difficult to get.

For all carriers, downed power lines, trees, and other debris complicated mobility in the company’s response. At the time of this writing some services are still out and another nor’easter storm is further complicating restoration of services.

Even in one carrier’s headquarters in lower Manhattan (a critical node of its network infrastructure) flooding hampered efforts for it to repair crippled network equipment and restore service both to its own end users as well as other carriers that use their wholesale services. These carriers include one client of mine who just completed a mad scramble to find another service provider on November 5, many days after the storm. This client is a wholesale customer with millions of calls that were blocked due to what amounts to an error by the carrier. To view this issue another way, consider that any would-be Manhattan company looking to move calls off of this carrier and on to this alternate provider due to Sandy would have been unsuccessful. Both the carrier and this alternate provider were using same underlying facilities. Clearly telecommunications is still an incestuous business.

The issues were even more pronounced for the wireless carriers. AT&T, T-Mobile, Verizon and Sprint wireless customers were all impacted. Wireless providers said that the sustained power outages outlasted their back-up battery power at some cell towers. Remember the “old days” when a three-day supply of station battery in a central office was the minimum? Remember the backup generators that these offices employed? Obviously, this was not the case here. The outages in Sandy exposed other weaknesses in wireless networks during disasters. In addition, all wireless carriers employ “backhaul” facilities largely owned by Verizon to carry calls from cell towers to their switching equipment. Even a wireless carrier who is not a Verizon customer is therefore easily impacted by a flood or other failure that affects Verizon. I have often advocated microwave radio as a backup system, and it is perfect for wireless backhaul—after all, one end of the service already has a tower, right? The trouble is it is often difficult to clear new microwave frequencies in heavily congested areas like New York City.

How to Make it Better

So what can we do about it? For starters, take a look at your local facility.  Most local serving offices owned by major carriers like AT&T or Verizon are sound structures constructed of reinforced concrete. Because designs vary, you should drive by the serving Central Office (CO) for your area. Look for the obvious. Does the structure have a large portion of the surface area covered by windows? Does it appear to be in an area prone to flooding? Is there major construction activity planned or occurring in the immediate area around the CO? Ask some questions of your telephone company’s account representative. Is the CO manned 24 hours a day?  What type of fire prevention systems does it employ? How old is the structure? Is the CO a tandem or end office? End offices serve only end customers. Tandems switch traffic from one end office to another. Today, many COs perform both functions. This could have a bearing on which systems get recovered first in a disaster as well as how many people might be affected in a metropolitan area. For example, loss of a Tandem all the way across town could still affect you by contributing to network congestion and blocking your calls. What plans exist for restoration in the event of fire and flood? What services would be affected?

Finally, it’s time to hold wireless providers to the same standards that were second nature to landline central office providers. These standards have been published for years and are still available to anyone inclined to look for them.

The carriers currently have a veritable birds nest on the ground in wireless services. People want these services, and demand for them is insane. Wireless service is arguably not as capital intensive as digging up streets and laying cable or fiber. The present providers have a virtual monopoly due to limited frequency allocations. Now consider the fact that the wireless analog of a residential land line that cost $9.00 a month a generation ago (in a regulated environment) today brings Wireless carriers $50 a month or more. And there is virtually no regulation on profits! Yet when people need service it is not there. I think it’s time the wireless providers stopped simply pocketing all of the money and instead invest a small portion in network resiliency. Absent the ability for the providers to act of their own accord, federal and state regulators should step in and require it.

In summary, mission critical applications are migrating out to the wireless world. Basic human safety is also mission critical. The reliability of wireless service in a disaster has become increasingly critical as more Americans ditch their landlines in favor of convenient, modern wireless devices of all types. Until carriers bring the wireless infrastructure to the same level of protection as they employed in the wireline world, we can simply expect these kinds of issues again and again. This will not only cost money; it will cost lives when another Sandy, Katrina or Ike inevitably returns.


About the Author

Leo A. Wrobel has 25 years of experience with a host of firms engaged in banking, brokerage, heavy manufacturing, telecommunications services and government. A noted author and technical futurist, Leo is responsible for many technological firsts, including the first microwave "bypass" shot in Dallas (1985). He was also the first person in Dallas to run T1 telephone traffic over a cable television system in an agreement he pioneered in 1985. Leo also served ten years as an elected Mayor and City Councilman of a Dallas suburb (but says he is "better now").

A sought-after speaker, Wrobel has lectured throughout the United States and overseas in Israel, South America, and other locations, as well as appearing on several television news programs, including KLRU's "Austin at Issue." A knowledgeable and effective communicator, he has combined his political and technical savvy in repeated engagements in order to get things done, even at the highest policy levels.

Leo Wrobel served as President and CEO of Dallas-based Premiere Network Services Inc. from 1986 until 2005. Prior to this he was Director, Network Planning and Engineering for Lomas and Nettleton and held technical positions at AT&T in its former Long Lines subsidiary. Leo is a Vietnam Era Veteran (Sgt. U.S.A.F.) and holds degrees in Electronic Systems Technology and Telecommunications Systems Technology and from Los Angeles City College as well as in Business and Public Policy from the University of Texas at Dallas. Leo is a member of the I.E.E.E., Independent Telephone Pioneers of America, Southwestern Bell Pioneers, and other noteworthy organizations.

An active author and technical futurist, he has published ten books and over 300 trade articles on a wide variety of technical subjects, including: Understanding Emerging Network Services, Pricing, and Regulation (© Artech House Books), Managing Emerging Technologies for Competitive Advantage (© 1995 Computer Economics), the Definitive Guide to Business Resumption Planning, (© Artech House Books) and The MIS and LAN Managers Guide to Advanced Telecommunications (© I.E.E.E. Books)

Find out more about Leo’s books at http://www.4ci.us.

1. Last year, the Federal Communications Commission sought public comments on how wireless providers should strengthen their systems, citing "the inadequacy of backup power" as a key reason for wireless failures during emergencies. In response, CTIA, which represents the wireless industry, urged regulators not to adopt regulations requiring back-up power at cell towers, citing in part “the unnecessarily burden wireless carriers and potentially undermine the investments and network planning that have made their networks so successful."

2. CTIA-The Wireless Association® is an international nonprofit membership organization that has represented the wireless communications industry since 1984. Membership in the association includes wireless carriers and their suppliers, as well as providers and manufacturers of wireless data services and products. The association advocates on behalf of its members at all levels of government.

InformIT Promotional Mailings & Special Offers

I would like to receive exclusive offers and hear about products from InformIT and its family of brands. I can unsubscribe at any time.

Overview


Pearson Education, Inc., 221 River Street, Hoboken, New Jersey 07030, (Pearson) presents this site to provide information about products and services that can be purchased through this site.

This privacy notice provides an overview of our commitment to privacy and describes how we collect, protect, use and share personal information collected through this site. Please note that other Pearson websites and online products and services have their own separate privacy policies.

Collection and Use of Information


To conduct business and deliver products and services, Pearson collects and uses personal information in several ways in connection with this site, including:

Questions and Inquiries

For inquiries and questions, we collect the inquiry or question, together with name, contact details (email address, phone number and mailing address) and any other additional information voluntarily submitted to us through a Contact Us form or an email. We use this information to address the inquiry and respond to the question.

Online Store

For orders and purchases placed through our online store on this site, we collect order details, name, institution name and address (if applicable), email address, phone number, shipping and billing addresses, credit/debit card information, shipping options and any instructions. We use this information to complete transactions, fulfill orders, communicate with individuals placing orders or visiting the online store, and for related purposes.

Surveys

Pearson may offer opportunities to provide feedback or participate in surveys, including surveys evaluating Pearson products, services or sites. Participation is voluntary. Pearson collects information requested in the survey questions and uses the information to evaluate, support, maintain and improve products, services or sites, develop new products and services, conduct educational research and for other purposes specified in the survey.

Contests and Drawings

Occasionally, we may sponsor a contest or drawing. Participation is optional. Pearson collects name, contact information and other information specified on the entry form for the contest or drawing to conduct the contest or drawing. Pearson may collect additional personal information from the winners of a contest or drawing in order to award the prize and for tax reporting purposes, as required by law.

Newsletters

If you have elected to receive email newsletters or promotional mailings and special offers but want to unsubscribe, simply email information@informit.com.

Service Announcements

On rare occasions it is necessary to send out a strictly service related announcement. For instance, if our service is temporarily suspended for maintenance we might send users an email. Generally, users may not opt-out of these communications, though they can deactivate their account information. However, these communications are not promotional in nature.

Customer Service

We communicate with users on a regular basis to provide requested services and in regard to issues relating to their account we reply via email or phone in accordance with the users' wishes when a user submits their information through our Contact Us form.

Other Collection and Use of Information


Application and System Logs

Pearson automatically collects log data to help ensure the delivery, availability and security of this site. Log data may include technical information about how a user or visitor connected to this site, such as browser type, type of computer/device, operating system, internet service provider and IP address. We use this information for support purposes and to monitor the health of the site, identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents and appropriately scale computing resources.

Web Analytics

Pearson may use third party web trend analytical services, including Google Analytics, to collect visitor information, such as IP addresses, browser types, referring pages, pages visited and time spent on a particular site. While these analytical services collect and report information on an anonymous basis, they may use cookies to gather web trend information. The information gathered may enable Pearson (but not the third party web trend services) to link information with application and system log data. Pearson uses this information for system administration and to identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents, appropriately scale computing resources and otherwise support and deliver this site and its services.

Cookies and Related Technologies

This site uses cookies and similar technologies to personalize content, measure traffic patterns, control security, track use and access of information on this site, and provide interest-based messages and advertising. Users can manage and block the use of cookies through their browser. Disabling or blocking certain cookies may limit the functionality of this site.

Do Not Track

This site currently does not respond to Do Not Track signals.

Security


Pearson uses appropriate physical, administrative and technical security measures to protect personal information from unauthorized access, use and disclosure.

Children


This site is not directed to children under the age of 13.

Marketing


Pearson may send or direct marketing communications to users, provided that

  • Pearson will not use personal information collected or processed as a K-12 school service provider for the purpose of directed or targeted advertising.
  • Such marketing is consistent with applicable law and Pearson's legal obligations.
  • Pearson will not knowingly direct or send marketing communications to an individual who has expressed a preference not to receive marketing.
  • Where required by applicable law, express or implied consent to marketing exists and has not been withdrawn.

Pearson may provide personal information to a third party service provider on a restricted basis to provide marketing solely on behalf of Pearson or an affiliate or customer for whom Pearson is a service provider. Marketing preferences may be changed at any time.

Correcting/Updating Personal Information


If a user's personally identifiable information changes (such as your postal address or email address), we provide a way to correct or update that user's personal data provided to us. This can be done on the Account page. If a user no longer desires our service and desires to delete his or her account, please contact us at customer-service@informit.com and we will process the deletion of a user's account.

Choice/Opt-out


Users can always make an informed choice as to whether they should proceed with certain services offered by InformIT. If you choose to remove yourself from our mailing list(s) simply visit the following page and uncheck any communication you no longer want to receive: www.informit.com/u.aspx.

Sale of Personal Information


Pearson does not rent or sell personal information in exchange for any payment of money.

While Pearson does not sell personal information, as defined in Nevada law, Nevada residents may email a request for no sale of their personal information to NevadaDesignatedRequest@pearson.com.

Supplemental Privacy Statement for California Residents


California residents should read our Supplemental privacy statement for California residents in conjunction with this Privacy Notice. The Supplemental privacy statement for California residents explains Pearson's commitment to comply with California law and applies to personal information of California residents collected in connection with this site and the Services.

Sharing and Disclosure


Pearson may disclose personal information, as follows:

  • As required by law.
  • With the consent of the individual (or their parent, if the individual is a minor)
  • In response to a subpoena, court order or legal process, to the extent permitted or required by law
  • To protect the security and safety of individuals, data, assets and systems, consistent with applicable law
  • In connection the sale, joint venture or other transfer of some or all of its company or assets, subject to the provisions of this Privacy Notice
  • To investigate or address actual or suspected fraud or other illegal activities
  • To exercise its legal rights, including enforcement of the Terms of Use for this site or another contract
  • To affiliated Pearson companies and other companies and organizations who perform work for Pearson and are obligated to protect the privacy of personal information consistent with this Privacy Notice
  • To a school, organization, company or government agency, where Pearson collects or processes the personal information in a school setting or on behalf of such organization, company or government agency.

Links


This web site contains links to other sites. Please be aware that we are not responsible for the privacy practices of such other sites. We encourage our users to be aware when they leave our site and to read the privacy statements of each and every web site that collects Personal Information. This privacy statement applies solely to information collected by this web site.

Requests and Contact


Please contact us about this Privacy Notice or if you have any requests or questions relating to the privacy of your personal information.

Changes to this Privacy Notice


We may revise this Privacy Notice through an updated posting. We will identify the effective date of the revision in the posting. Often, updates are made to provide greater clarity or to comply with changes in regulatory requirements. If the updates involve material changes to the collection, protection, use or disclosure of Personal Information, Pearson will provide notice of the change through a conspicuous notice on this site or other appropriate way. Continued use of the site after the effective date of a posted revision evidences acceptance. Please contact us if you have questions or concerns about the Privacy Notice or any objection to any revisions.

Last Update: November 17, 2020