Home > Articles > Networking > Storage

Securing Overlay Transport Virtualization (OTV) with Cisco TrustSec (CTS)

  • Print
  • + Share This
Using Overlay Transport Virtualization for your Data Center Interconnect is a hot trend in the cloud-enabled world we live in today. A common question customers ask is about layering security into the solution, and this article discusses just how to do that with MACSEC and AES-128 bit encryption. Ron Fuller will show you just how easy it is to take advantage of this capability on your Cisco Nexus 7000 series switches.

Read NX-OS and Cisco Nexus Switching: Next-Generation Data Center Architectures, 2nd Edition or more than 24,000 other books and videos on Safari Books Online. Start a free trial today.

From the author of

Data Center Interconnection (DCI) has become quite common in the past few years for a multitude of reasons, including workload mobility, data center migration, and enhanced disaster recovery posture, to name a few. Of the many DCI technologies that are available, Cisco's Overlay Transport Virtualization (OTV) has been widely deployed by customers in every vertical across the world.

OTV has many unique benefits in the DCI arena, including a scalable control plane protocol, ease of site additional and removal, optimized traffic localization capabilities, and the ability to use existing network infrastructure to provide connectivity. OTV is currently available on the flagship data center switching platform, the Cisco Nexus 7000, as well as the Cisco ASR 1000 series of routers that together provide an OTV solution to meet almost any customer's requirements.

For many customers, regulatory requirements or internal policies dictate that network traffic must be encrypted when passing between data centers.

Traditionally, there have been a few ways to facilitate this, ranging from dedicated and expensive hardware-based encryption devices that sit inline or the use of various VPN technologies such as IPSec to provide encryption services. See Figure 1.

One alternative to these technologies is using 802.1AE MACSEC on the Nexus 7000 to provide hardware-based, line rate encryption on the Data Center Interconnect links. 802.1AE MACSEC is an IEEE standard for link level encryption. The implementation of MACSEC on the Nexus 7000 is 128-bit Advanced Encryption Standard (AES) that is hardware-driven, which means no additional Supervisor CPU is used to encrypt data at any speeds.

As of the writing of this article, the M-Series modules on the Nexus 7000 support 802.1AE MACSEC on all ports, including the new M2-series modules. The F2e modules will have this feature enabled in the future.

It is important to note that because 802.1AE MACSEC is a link-level encryption, the two MACSEC-enabled endpoints, Nexus 7000 devices in our case, must be directly L2 adjacent. This means we direct fiber connection or one facilitated with optical gear is required. MACSEC has integrity checks for the frames and intermediate devices, like another switch, even at L2, will cause the integrity checks to fail. In most cases, this means metro-Ethernet services or carrier-provided label switched services will not work for a MACSEC connection. We will focus on a direct connection scenario in this article.

Figure 2 shows two data centers with OTV configured to provide data center interconnect services. The DCI link in this example is interface Ethernet4/1 on both of the Nexus switches. Prior to enabling CTS, the interface configuration looked like this:

version 6.1(2)

interface Ethernet4/1
  no ip redirects
  ip address
  ip ospf network point-to-point
  ip router ospf 100 area
  ip pim sparse-mode
  no shutdown

To enable CTS, we simply need to add the following commands that will do the following:

  • Enable cts and dot1x as a feature.
  • Configure cts manual mode.
  • Define a preshared key.
  • Increase the interface MTU to accommodate the extra header 802.1AE MACSEC adds.

The last step is important because CTS won't take effect until the interface is flapped with a shut/no shut sequence.

N7K-1-Core# config
Enter configuration commands, one per line.  End with CNTL/Z.
N7K-1-Core(config)# feature dot1x
N7K-1-Core(config)# feature cts
N7K-1-Core(config)# interface ethernet4/1
N7K-1-Core(config-if)# cts manual
N7K-1-Core(config-if-cts-manual)# sap pmk d00df001
N7K-1-Core(config-if-cts-manual)# exit
N7K-1-Core(config-if)# mtu 9216
N7K-1-Core(config-if)# shut
N7K-1-Core(config-if)# no shut
N7K-1-Core(config-if)# end

Once configured, you can verify that CTS is in use by looking at the interface:

N7K-1-Core# show cts interface e4/1
CTS Information for Interface Ethernet4/1:
    CTS is enabled, mode:   CTS_MODE_MANUAL
    IFC state:              CTS_IFC_ST_CTS_OPEN_STATE
    Authentication Status:  CTS_AUTHC_SKIPPED_CONFIG
      Peer Identity:
      Peer is:              Unknown in manual mode
      802.1X role:          CTS_ROLE_UNKNOWN
      Last Re-Authentication:
    Authorization Status:   CTS_AUTHZ_SKIPPED_CONFIG
      PEER SGT:             0
      Peer SGT assignment:  Not Trusted
    SAP Status:             CTS_SAP_SUCCESS
      Configured pairwise ciphers: GCM_ENCRYPT
      Replay protection: Enabled
      Replay protection mode: Strict
      Selected cipher: GCM_ENCRYPT
      Current receive SPI: sci:f866f2ac5aa80000 an:3
      Current transmit SPI: sci:30e4db1c019c0000 an:3
    Propagate SGT: Enabled


The bold text shows that the encryption is active and negotiated.

Be sure to enable CTS on both ends of the link because if only one end is configured and the interface is flapped, the link will not come up as shown below:

N7K-1-Core# show int e4/1
Ethernet4/1 is down (Authorization pending)
admin state is up, Dedicated Interface

That's it! How easy is that?! A few more thoughts on using CTS MACSEC for Data Center Interconnect:

  • No need to change routing protocols or other L2 protocols to understand MACSEC; because it is link-level encryption, it is transparent to the upper-layer technologies.
  • Optical taps don't work because the data is encrypted! SPAN/ERSPAN will still allow visibility into the frame as the CTS header is removed before the traffic is replicated.
  • Remember that 40 bytes are added for CTS—16 for the 802.1AE header, 8 bytes for the Cisco MetaData, and 16 for the Integrity Check Value (ICV). Plan your MTU accordingly, and if you are using OTV, remember that it adds 42 more bytes.
  • CTS was moved into the NX-OS base license as of NX-OS 6.1 on the Nexus 7000.

Cisco TrustSec and 802.1AE MACSEC provide a very attractive alternative to expensive external devices that take rack space, power, and cooling resources in my data center. Being transparent to the upper-layer protocols enables more ways to deploy line-rate encryption.

  • + Share This
  • 🔖 Save To Your Account