Home > Articles > Networking

Packet over SONET

This chapter is from the book

PoS Network Designs

Resiliency is an important concern in SP networks. Outages result in lost revenue and might cause customers to cancel their service. SPs enter into Service Level Agreements (SLAs) with their customer. These SLAs guarantee certain levels of service. SLAs differ in many respects depending on the amount of risk the customer is willing to take. The more risk the customer is willing to take, the looser the SLA is and the cheaper the cost to the customer. The downside is that the customer is not guaranteed the same level of service as the customer who was not willing to take as much risk and paid more money for a stringent SLA.

PoS provides support of the optical 1+1 automatic protection switching (APS) mechanism. A customer desiring this level of protection orders two circuits from the SP: one for working traffic and one for protect traffic. SPs offer discounts for circuits that are used for protect traffic. The CPE router in this design could be a single point of failure, depending on how the circuit terminates at the CPE. Both circuits terminating on one line card of one router would result in a single point of failure from both the line card and router perspective. A method of slightly higher resiliency is to still use one router, but use separate line cards for the working and protect circuits. This scenario provides fault tolerance in the case of a line card failure, but not a router failure. A higher level of fault tolerance might be achieved if each circuit terminates on a separate router.

All of these survivability network designs are connected to one ADM at the service provider. APS 1+1 protection schemes are normally implemented per add/drop multiplexer. Ring failure is handled by the SP's robust SONET ring protection mechanisms. ADMs are carrier-class devices and must maintain a level of Five-Nines reliability. Five-Nines reliability refers to the amount of uptime a customer should expect from that network. Five-Nines reliability represents an uptime of 99.999 percent.

One Router

Figure 9-9 shows a design where there is one router at the customer premises with two optical interfaces used for APS 1+1 protection. Although a one-router CPE design does not provide the highest level of resiliency, this design does offer some advantages, including the following:

  • No routing convergence upon failure of the working circuit or optical interfaces.

  • 1+1 APS optical protection. Convergence time can be achieved in sub-60-ms time.

  • Low-complexity network configuration.

Figure 9Figure 9-9 One-Router CPE Design

Two Routers

In a two-router design, each router has one optical connection to the SP's add/drop multiplexer. Fault tolerance has been increased with this design because the CPE router is no longer a single point of failure and the routers can be located in different areas of the building to facilitate fault tolerance associated with issues that could arise in isolated areas.

Figure 9-10 illustrates the two-router design philosophy. Although each router has one optical interface to the ADM, one link is working (active) while the other link is protecting (standby) the working link.

Figure 10Figure 9-10 Two-Router CPE Design

Resources are wasted if only one router is actively forwarding traffic. To fully use both routers, you could use a design including four circuits. The design requires twice the number of interfaces and circuits, but this might still be cost advantageous depending on the amount of bandwidth required and the router hardware employed. Figure 9-11 displays an environment that includes two routers and four circuits in which both routers are in a working state for one circuit. The optical protection scheme is logically divided into APS protection groups that the routers monitor. A large router such as the Cisco 12000 can accommodate hundreds of APS groups.

Figure 10Figure 9-11 Two Routers with Four Circuits

Advantages of having two PoS-connected routers include the following:

  • Router redundancy in addition to circuit redundancy

  • Load balancing of traffic

Disadvantages of having two routers connected to the single ADM in the SONET/SDH network include the following:

  • Convergence time—The Layer 3 routing protocol must converge to optical circuit failure.

  • Complex network configuration—APS groups.

  • Cost—The costs associated with setting up and maintaining the design.

Failure recovery using two routers cannot achieve the sub-60-ms time that the one-router alternative offers. The Layer 3 routing protocol implemented in the infrastructure needs to reconverge around the failure. This is not an issue with one-router designs because both of the PoS interfaces on one router can have the same IP address with the PoS APS 1+1 configuration commands. This feature is allowable because only one of the interfaces is active at any one time.

PoS Protection Schemes

Packet over SONET protection uses the SONET APS 1+1 protection scheme. APS 1+1 looks at the K1 and K2 bytes of the SONET line overhead to determine whether issues exist with the SONET ring. A failure in the SONET network that affects the customer's working path causes a failover at the client site. This failover time occurs under 50 ms in the SONET Layer 1 network. The router interface uses a keepalive to determine whether the other side of the connection is alive. Keepalives are sent every 10 seconds by default, and the loss of 3 subsequent keepalives results in the interface going to an up/down state. After Layer 2 is lost, the Layer 3 routing protocol must converge around the link failure. Waiting for Layer 2 and Layer 3 to go through this procedure can take a long time (more than 30 seconds). Configuring the keepalives to 1 second lowers the convergence time to 3 seconds. Because PoS interfaces are Layer 3 implementations, the interfaces need to rely on a hierarchical error-recovery method such as that shown in Figure 9-12.

Figure 12Figure 9-12 PoS Hierarchical Error Recovery

Layer 3 provides rerouting decisions during network failures to provide intelligent resiliency to the network. Layer 3 routing might be needed during a link failure if the Layer 3 IP address is changed. The Layer 3 IP address in the one-router design would be identical and there would never need to be Layer 3 routing protocol reconvergence. PoS interfaces in different routers require different IP addresses and always result in routing protocol reconvergence.

APS 1+1 Protection

SONET APS 1+1 is used for any PoS design that has more than one optical interface. APS provides optical protection during times of optical failure in the SP network. This information is carried over the K1 and K2 bytes of the SONET overhead. The CPE listens to the K1 and K2 bytes generated by the SONET network and generates K1 and K2 bytes when a failure occurs on the customer side. If the working interface of an APS 1+1 group fails, the protect interface can quickly assume its traffic load. The Layer 1 APS 1+1 recovery mechanism operates in 60 ms.

NOTE

SONET rings have a 50-ms switchover time rather than the 60-ms switchover used in the Bellcore APS 1+1 specification. The APS 1+1 specification provides 10 ms for failure detection and 50 ms for switch initiation, which collectively equal 60 ms.

SONET APS works at Layer 1 providing switchover times significantly faster than any protocols operating at Layer 2 or 3.

The SONET protection mechanism used for PoS on Cisco products uses APS 1+1 with either unidirectional or bidirectional switching. (You can read more about 1+1 uni- and bidirectional switching in Chapter 3, "SONET Overview.")

The SONET APS 1+1 architecture designates that there will be two circuits and each will carry the same traffic. One circuit is considered the working circuit; the other is the protect line. This differs from 1:1 or 1:n electrical-protection schemes because the backup equipment in electrical-protection schemes only carries traffic upon failure. The working and protect lines of APS 1+1 are both always transporting traffic. The receiving device(s) only process the traffic being received on the working circuit.

Protection mechanisms are more complex when circuits are terminated on different routers. The protection router must somehow be identified of the failure situation. An additional protocol is needed to provide for this signaling. This protocol is a Cisco proprietary mechanism called the Protection Group Protocol (PGP).

If a signal fail (SF) or a signal degrade (SD) condition is detected, the hardware switches from the working circuit to the protect circuit. APS 1+1 has reversionary capabilities allowing the hardware to switch back to the working circuit automatically when the original signal is restored for the configured time interval. The configurable reversion time is used to prevent the system from switching back to the working circuit if it is flapping (repeatedly going up and down). Flapping is sometimes referred to as switch oscillation and should be avoided at all costs so that the SP equipment can meet the SLAs. If the revertive option is not used, after a switch has moved to the protect circuit, the hardware does not automatically revert back to the working circuit. A system administrator must manually perform this function. Bidirectional switching is the default operation in Cisco routers. A circuit that automatically switches back to the original facility is called a reversionary circuit.

The K1/K2 bytes from the line overhead of the SONET frame indicate the current status of the APS connection and convey any requests for action. In standard APS, the two ends of the connection use this signaling channel to maintain synchronization.

With Cisco PoS, the working and protect channels are synchronized through an independent communications channel that is not part of the standard SONET APS system. This independent channel works whether the interfaces are on the same or different routers. This low-bandwidth connection is the Cisco PGP.

Cisco Protect Group Protocol (PGP)

PGP is the Cisco proprietary APS communication channel that is used between routers to complement APS 1+1 protection signaling. APS 1+1 is normally only done on the same router, but PGP enables this functionality to span multiple routers for added resiliency.

Performing APS 1+1 operation between routers creates some Layer 3 convergence issues. The standard Layer 2 mechanism used to determine whether an interface is down is the keepalive function. To accommodate fast reconvergence times, the keepalive update timer should be changed to 1 second and the hold timer changed to 3 seconds. PGP is the signaling channel used to inform the router with the protect facility about the failure. PGP operation closely resembles that of Cisco Hot Standby Router Protocol (HSRP) performing a heartbeat operation over a low-speed interface that tracks the status of certain ports. You can configure different protection groups to monitor multiple ports. The PGP protocol is a connectionless protocol that uses User Datagram Protocol (UDP) port 172 for message transfer. Figure 9-13 displays two routers that are configured in the same APS group. Notice that PGP updates are propagated bidirectionally between the working and protect routers to exchange information regarding the status of the PoS interface.

Figure 13Figure 9-13 Protection Group Protocol Operation

Figure 9-14 displays a network in which an outage occurs between POP B and POP C on the working facility. The routers at POP B and POP C will have knowledge of this outage through a loss of signal (LOS) condition, and PGP will notify the other router that it will now become the working interface. The other routers in the network will learn of this occurrence through the K1/K2 byte signaling occurring throughout the network.

Figure 14Figure 9-14 PGP Link Selection


PoS Convergence

Convergence time is the amount of time required for all routers in a network to learn of changes in the network topology. Routers must propagate new route information from one end of the network to the other. Routing protocols are implemented to exchange this information. The routing protocol implemented should provide an ample amount of scalability to meet the future needs of the networks used in the environment. The faster the routing protocol can converge, the less downtime that will occur.

Scalable IP network routing protocols, such as Open Shortest Path First (OSPF), Integrated IS-IS, and Border Gateway Protocol (BGP), are responsible for recovering from error conditions in the network. Although the SONET APS 1+1 protection switching mechanisms guarantee a restoration time of 60 ms, the PoS interfaces are Layer 3 implementations and require some deal of routing protocol convergence. Typical convergence times for scalable routing protocols are several seconds or more depending on the environment and routing protocol design.

Figure 9-15 displays a design in which one router is used for the PoS interfaces. With this design, both of the PoS interfaces in the router can be configured with the same IP address. If a failure occurs, the router can perform switchover in the APS 1+1 switchover time of 60 ms. The Layer 3 routing protocol has not changed in any way on the LAN or WAN side of the router. The Layer 2 keepalive mechanism might not be aware of this switchover because it occurred in less than the lowest keepalive timer of 1 second. Regardless, three keepalives must be missed before an interface is determined as down.

Figure 15Figure 9-15 1-Router APS 1+1 Convergence

Figure 9-16 displays an environment that requires a higher degree of fault tolerance. This design uses two routers to implement the APS 1+1 group to protect the design from a router failure. The added resiliency creates some Layer 3 convergence issues because the interfaces used cannot have the same IP address if they reside on different physical routers. When the failure occurs, PGP is used to determine that the working interface has gone down, and the protect interface takes over. After this switchover has occurred, the Layer 3 routing protocol must communicate this information on both the LAN and WAN side so that the end to end network learns of the failure and solution. It is best to use HSRP on the LAN side if the PoS routers represent the default gateways out of the network. HSRP update and dead timers should be configured to match those of PGP.

Figure 16Figure 9-16 2-Router APS 1+1 Convergence

Flapping

Flapping is the operation of a transmission line regularly transitioning from an up/up to an up/down state in a short period of time. Intermittent failures can result in the APS protection mechanism switching between the working and protection traffic repeatedly, causing many fluctuations in the network. If a two-router PoS model is implemented, the Layer 3 routing protocols will flap, too. You can see this issue in Figure 9-17.

APS switches traffic upon failures, but the routing protocol must send out routing updates. If another failure happens (Failure 2), the failure results in another APS switchover and more routing updates. Subsequent failures (Failures 3 and 4) repeat the process. The result of this flapping is that the network could end up spending all the time sending routing updates and reconverging around repeating failures instead of sending data across the network.

Figure 17Figure 9-17 Flapping in a 2-Router PoS Design

The issue is manageable by tweaking the reversion timer to a time greater than that necessary for the Layer 3 routing protocol to converge. The interfaces would not bring down the network because they must be stable in that amount of time before any switchover will take place.

PoS Reflector Mode

PoS Reflector mode is a process that is used to inform the remote router of a change in the network topology due to a line failure. Figure 9-18 displays an environment with two routers where a failure has occurred on the working line. As soon as the protect router receives information of the down interface through PGP, the protect router initiates a packet to the other side of the connection to speed up convergence. The packet contains the router ID information needed by the routing protocol to create the new Layer 3 adjacency. The remote router can now change the IP adjacency information immediately and reduce the convergence time dramatically.

Figure 18Figure 9-18 PoS Reflector Mode

Load Balancing

Load balancing refers to the capability to have traffic traverse two separate paths simultaneously to maximize the resources at the site. Load balancing is possible in a PoS APS 1+1 environment where four circuits are present. APS groups are configured on each router. One router is the working router for Group 1, and the other router is the working router for Group 2. Each of these routers protects each other using the PGP mechanism to alert the other side of failures. Figure 9-19 shows this design. You can use Multigroup HSRP (MHSRP) on the LAN side to actively forward traffic to both of these devices while providing the resiliency necessary. Layer 3 convergence is an end-to-end solution.

Figure 19Figure 9-19 PoS Load Balancing

Alarms and PoS

Customers want to be notified of problems and errors that occur on their lines. PoS uses the same alarming of that used for SONET alarm reporting. The information that is carried in the overhead bytes of the Section, Line, and Path overhead layers are used by PoS to determine and report errors. This includes such items as the following:

  • Loss of signal (LOS)—Signal failure due to a loss of light on the receive interface. A loss of light can also be thought of as receiving an all-0s pattern before descrambling. A downstream AIS should be sent when an LOS is detected.

  • Loss of frame (LOF)—Issue created by receiving A1 and A2 bytes that do not indicate the 2-byte code of F628 in hexadecimal. An LOF condition is registered after no valid framing information has been received in 3 ms. The receipt of two subsequent valid A1/A2 frames clears this condition. A line alarm indication signal (AIS) must be sent downstream when this condition occurs.

  • Bit interleaved parity (BIP) errors—BIP-3 errors occur at the path layer. The PoS interface is a path terminating equipment (PTE) device. The B3 byte carries the path parity errors in this byte.

  • Loss of pointer (LOP)—When a pointer processor cannot obtain a valid pointer condition, an LOP state is declared, and a downstream AIS must be sent. Recall that the H1 through H3 bytes of the LOH are used for the pointer functionality.

Threshold registers record all the normal SONET counters for errors that occurred over the past 15 minutes and past 24 hours. You can view these by using IOS show commands. When the threshold register exceeds the threshold register settings, a threshold crossing alarm (TCA) indication occurs, meaning the device needs to notify the management station of the alarm.

InformIT Promotional Mailings & Special Offers

I would like to receive exclusive offers and hear about products from InformIT and its family of brands. I can unsubscribe at any time.

Overview


Pearson Education, Inc., 221 River Street, Hoboken, New Jersey 07030, (Pearson) presents this site to provide information about products and services that can be purchased through this site.

This privacy notice provides an overview of our commitment to privacy and describes how we collect, protect, use and share personal information collected through this site. Please note that other Pearson websites and online products and services have their own separate privacy policies.

Collection and Use of Information


To conduct business and deliver products and services, Pearson collects and uses personal information in several ways in connection with this site, including:

Questions and Inquiries

For inquiries and questions, we collect the inquiry or question, together with name, contact details (email address, phone number and mailing address) and any other additional information voluntarily submitted to us through a Contact Us form or an email. We use this information to address the inquiry and respond to the question.

Online Store

For orders and purchases placed through our online store on this site, we collect order details, name, institution name and address (if applicable), email address, phone number, shipping and billing addresses, credit/debit card information, shipping options and any instructions. We use this information to complete transactions, fulfill orders, communicate with individuals placing orders or visiting the online store, and for related purposes.

Surveys

Pearson may offer opportunities to provide feedback or participate in surveys, including surveys evaluating Pearson products, services or sites. Participation is voluntary. Pearson collects information requested in the survey questions and uses the information to evaluate, support, maintain and improve products, services or sites, develop new products and services, conduct educational research and for other purposes specified in the survey.

Contests and Drawings

Occasionally, we may sponsor a contest or drawing. Participation is optional. Pearson collects name, contact information and other information specified on the entry form for the contest or drawing to conduct the contest or drawing. Pearson may collect additional personal information from the winners of a contest or drawing in order to award the prize and for tax reporting purposes, as required by law.

Newsletters

If you have elected to receive email newsletters or promotional mailings and special offers but want to unsubscribe, simply email information@informit.com.

Service Announcements

On rare occasions it is necessary to send out a strictly service related announcement. For instance, if our service is temporarily suspended for maintenance we might send users an email. Generally, users may not opt-out of these communications, though they can deactivate their account information. However, these communications are not promotional in nature.

Customer Service

We communicate with users on a regular basis to provide requested services and in regard to issues relating to their account we reply via email or phone in accordance with the users' wishes when a user submits their information through our Contact Us form.

Other Collection and Use of Information


Application and System Logs

Pearson automatically collects log data to help ensure the delivery, availability and security of this site. Log data may include technical information about how a user or visitor connected to this site, such as browser type, type of computer/device, operating system, internet service provider and IP address. We use this information for support purposes and to monitor the health of the site, identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents and appropriately scale computing resources.

Web Analytics

Pearson may use third party web trend analytical services, including Google Analytics, to collect visitor information, such as IP addresses, browser types, referring pages, pages visited and time spent on a particular site. While these analytical services collect and report information on an anonymous basis, they may use cookies to gather web trend information. The information gathered may enable Pearson (but not the third party web trend services) to link information with application and system log data. Pearson uses this information for system administration and to identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents, appropriately scale computing resources and otherwise support and deliver this site and its services.

Cookies and Related Technologies

This site uses cookies and similar technologies to personalize content, measure traffic patterns, control security, track use and access of information on this site, and provide interest-based messages and advertising. Users can manage and block the use of cookies through their browser. Disabling or blocking certain cookies may limit the functionality of this site.

Do Not Track

This site currently does not respond to Do Not Track signals.

Security


Pearson uses appropriate physical, administrative and technical security measures to protect personal information from unauthorized access, use and disclosure.

Children


This site is not directed to children under the age of 13.

Marketing


Pearson may send or direct marketing communications to users, provided that

  • Pearson will not use personal information collected or processed as a K-12 school service provider for the purpose of directed or targeted advertising.
  • Such marketing is consistent with applicable law and Pearson's legal obligations.
  • Pearson will not knowingly direct or send marketing communications to an individual who has expressed a preference not to receive marketing.
  • Where required by applicable law, express or implied consent to marketing exists and has not been withdrawn.

Pearson may provide personal information to a third party service provider on a restricted basis to provide marketing solely on behalf of Pearson or an affiliate or customer for whom Pearson is a service provider. Marketing preferences may be changed at any time.

Correcting/Updating Personal Information


If a user's personally identifiable information changes (such as your postal address or email address), we provide a way to correct or update that user's personal data provided to us. This can be done on the Account page. If a user no longer desires our service and desires to delete his or her account, please contact us at customer-service@informit.com and we will process the deletion of a user's account.

Choice/Opt-out


Users can always make an informed choice as to whether they should proceed with certain services offered by InformIT. If you choose to remove yourself from our mailing list(s) simply visit the following page and uncheck any communication you no longer want to receive: www.informit.com/u.aspx.

Sale of Personal Information


Pearson does not rent or sell personal information in exchange for any payment of money.

While Pearson does not sell personal information, as defined in Nevada law, Nevada residents may email a request for no sale of their personal information to NevadaDesignatedRequest@pearson.com.

Supplemental Privacy Statement for California Residents


California residents should read our Supplemental privacy statement for California residents in conjunction with this Privacy Notice. The Supplemental privacy statement for California residents explains Pearson's commitment to comply with California law and applies to personal information of California residents collected in connection with this site and the Services.

Sharing and Disclosure


Pearson may disclose personal information, as follows:

  • As required by law.
  • With the consent of the individual (or their parent, if the individual is a minor)
  • In response to a subpoena, court order or legal process, to the extent permitted or required by law
  • To protect the security and safety of individuals, data, assets and systems, consistent with applicable law
  • In connection the sale, joint venture or other transfer of some or all of its company or assets, subject to the provisions of this Privacy Notice
  • To investigate or address actual or suspected fraud or other illegal activities
  • To exercise its legal rights, including enforcement of the Terms of Use for this site or another contract
  • To affiliated Pearson companies and other companies and organizations who perform work for Pearson and are obligated to protect the privacy of personal information consistent with this Privacy Notice
  • To a school, organization, company or government agency, where Pearson collects or processes the personal information in a school setting or on behalf of such organization, company or government agency.

Links


This web site contains links to other sites. Please be aware that we are not responsible for the privacy practices of such other sites. We encourage our users to be aware when they leave our site and to read the privacy statements of each and every web site that collects Personal Information. This privacy statement applies solely to information collected by this web site.

Requests and Contact


Please contact us about this Privacy Notice or if you have any requests or questions relating to the privacy of your personal information.

Changes to this Privacy Notice


We may revise this Privacy Notice through an updated posting. We will identify the effective date of the revision in the posting. Often, updates are made to provide greater clarity or to comply with changes in regulatory requirements. If the updates involve material changes to the collection, protection, use or disclosure of Personal Information, Pearson will provide notice of the change through a conspicuous notice on this site or other appropriate way. Continued use of the site after the effective date of a posted revision evidences acceptance. Please contact us if you have questions or concerns about the Privacy Notice or any objection to any revisions.

Last Update: November 17, 2020