Protecting Against Cable Cuts and Other Accidental Disruptions
In all my years in the disaster recovery business, I have never met anyone who has never had a cable cut. Since we are planning for what will certainly happen, let’s briefly cover some of the ways we may protect against this threat.
Fiber Optic Cable Cuts (Long Haul)
When selecting a long-distance carrier, it’s important to evaluate the type of protection it provides against disruptions. Remember that long-haul services often traverse the same right of way. This might include routing fiber optic lines along railroads, highways, and other places where right of way exists.
Train derailments are a common cause of long-distance service outages, and the same accident can get multiple carriers. If the derailment also spills a load of hazardous chemicals, the outage could be even more prolonged.
Concerns about service outages go beyond right of way concerns as well. Consider the fact that there are only a limited number of mountain passes to easily get over the Rocky Mountains to the U.S. or Canadian west coast. In the springtime, rapid thaws have been known to wash out sometimes a mile or more of roadway—and cable—at a time.
The network has evolved to the point where this rarely represents a total failure any more (because of other routes in the network), but even with alternate paths, but the network can become saturated. (Callers will probably receive an "all circuits busy" recording when calling.)
Note that all the comments above pertain to switched services. Private lines or custom services are still more problematic because of being custom-engineered and generally nailed down in place. They were never designed to move and they still take longer to restore in a disaster.
Local Cable Cuts
Local telephone cables, commonly referred to as the last mile, are also a constant source of disruption. Since these cables run through metropolitan areas where construction is frequent, it becomes very easy for construction crews to either dig up or core through these cables. (People do not dig up streets, they core under them, and often through telephone cables.)
Moreover, rights of way in major cities have been in continuous use for decades, and recordkeeping is not always what it should be. When a contractor or road crew begins digging, they are never really sure what they will hit on the way down. Water lines, electric lines, and even sewer lines all use the same right of way.
Software and Service Management Disruptions
Many times, disruptions to telecommunication systems are due not to cable cuts or hardware failure but to software or service management failure. A few years ago, the city of Dallas, Texas had a four-hour-long disruption during which it was difficult to place any type of telephone call in the Dallas metropolitan exchange.
The cause of the disruption was not due to a cable cut, tornado, or hurricane—it was because Garth Brooks concert tickets were on sale. Radio announcers’ broadcast this coveted sale, inundating the network with calls, thereby making other types of telephone calls difficult or impossible.
Similarly, software failures might also affect the network. Most modern central office switches have an elaborate series of software codes, blocks, and classes of service, which can be difficult to troubleshoot in the event of software failure.
VoIP networks represent a whole different area of concern. Remember that even in the most capable local operating companies, many personnel are still more comfortable with telephone test sets than with data scopes. Services such as VoIP, which save an organization money today, could be problematic in the event of a catastrophic failure that must be recovered quickly. Make sure that your vendor is trained on them and has the organizational depth to react quickly in an emergency.
By the same token, IP is incredibly resilient in a lot of respects. It was originally used by the military at a time when most AT&T primary toll offices could suddenly be in the upper atmosphere after a nuclear strike. If IP circuits were expected to work after that, if properly engineered they could stand up to just about any problem your organization could experience.
We’ll discuss VoIP further in coming articles. The point here is to understand the dynamics of whatever technology you are using and be clear in advance on how long it would take to recover in a disaster.