This chapter is from the book
This chapter is from the book
This chapter is from the book
How IPSec Works
The goal of IPSec is to protect the desired data with the needed security services. IPSec's operation can be broken into five primary steps:
|
Step 1 |
Define interesting traffic—Traffic is deemed interesting when the VPN device recognizes that the traffic you want to send needs to be protected. |
|
Step 2 |
IKE Phase 1—Between peers, a basic set of security services is negotiated and agreed on. This basic set of security services protects all subsequent communications between the peers. IKE Phase 1 sets up a secure communications channel between peers. |
|
Step 3 |
IKE Phase 2—IKE negotiates IPSec security association (SA) para-meters and sets up matching IPSec SAs in the peers. These security parameters are used to protect data and messages exchanged between endpoints. |
|
Step 4 |
Data transfer—Data is transferred between IPSec peers based on the IPSec parameters and keys stored in the SA database. |
|
Step 5 |
IPSec tunnel termination—IPSec SAs terminate through deletion or by timing out. |
Step 1: Define Interesting Traffic
Determining what traffic needs to be protected is done as part of formulating a security policy for use of a VPN. The policy is used to determine what traffic needs to be protected and what traffic can be sent in the clear. For every inbound and outbound packet, you have three choices:
Apply IPSec
Bypass IPSec
Discard the packet
For every packet protected by IPSec, the system administrator must specify the security services applied to the packet. The security policy database specifies the IPSec protocols, modes, and algorithms applied to the traffic. The services are then applied to traffic destined for each particular IPSec peer. With the VPN Client, you use menu windows to select connections you want secured by IPSec. When interesting traffic transits the IPSec client, the client initiates the next step in the process: negotiating an IKE Phase 1 exchange. Figure 1-21 shows two routers with Host A and Host B at either end. You have to decide whether to encrypt, not encrypt, or drop the packets.
)
Figure
1-21 Step 1: Define Interesting Traffic
Step 2: IKE Phase 1
The basic purpose of IKE Phase 1, shown in Figure 1-22, is to negotiate IKE policy sets, authenticate the peers, and set up a secure channel between the peers. IKE Phase 1 occurs in two modes: main mode and aggressive mode.
)
Figure
1-22 Step 2: IKE Phase 1
Main mode has three two-way exchanges between the initiator and receiver:
First exchange—The algorithms and hashes used to secure the IKE communications are negotiated and agreed on between peers.
Second exchange—Uses a DH exchange to generate shared secret keys and to pass nonces, which are random numbers sent to the other party, signed, and returned to prove their identity. The shared secret key is used to generate all the other encryption and authentication keys.
Third exchange—Verifies the other side's identity. It is used to authenticate the remote peer. The main outcome of main mode is a secure communication path for subsequent exchanges between the peers. Without proper authentication, it is possible to establish a secure communication channel with a hacker who is now stealing all your sensitive material.
In aggressive mode, fewer exchanges are done and with fewer packets. On the first exchange, almost everything is squeezed in: the IKE policy set negotiation; the DH public key generation; a nonce, which the other party signs; and an identity packet, which can be used to verify the identity via a third party. The receiver sends everything back that is needed to complete the exchange. The only thing left is for the initiator to confirm the exchange.
When trying to make a secure connection between Host A and B through the Internet, IKE security proposals are exchanged between Routers A and B. The proposals identify the IPSec protocol being negotiated (for example, ESP). Under each proposal, the originator must delineate which algorithms are employed in the proposal (for example, DES with MD5). Rather than negotiate each algorithm individually, the algorithms are grouped into IKE transform sets. A transform set delineates which encryption algorithm, authentication algorithm, mode, and key length are proposed. These IKE proposals and transform sets are exchanged during the IKE main mode first exchange phase. If a transform set match is found between peers, the main mode continues. If no match is found, the tunnel is torn down.
In Figure 1-23, Router A sends IKE transform sets 10 and 20 to Router B. Router B compares its set, transform set 15, with those received from Router A. In this instance, a match occurs: Router A's transform set 10 matches Router B's transform set 15.
)
Figure
1-23 Step 2: IKE Transform Sets
In a point-to-point application, each end might need only a single IKE policy set defined. However, in a hub-and-spoke environment, the central site might require multiple IKE policy sets to satisfy all the remote peers.
Step 3: IKE Phase 2
The purpose of IKE Phase 2 is to negotiate the IPSec security parameters that are applied to the interesting traffic traversing the tunnel negotiated during Phase 1. IKE Phase 2 performs the following functions:
Negotiates IPSec security parameters and IPSec transform sets
Establishes IPSec SAs
Periodically renegotiates IPSec SAs to ensure security
Optionally performs an additional DH exchange
IKE Phase 2 has one mode—quick mode. Quick mode occurs after IKE has established the secure tunnel in Phase 1. It negotiates a shared IPSec transform, derives shared secret keying material used for the IPSec security algorithms, and establishes IPSec SAs. Quick mode exchanges nonces that are used to generate new shared secret key material and to prevent replay attacks from generating bogus SAs.
Quick mode is used to renegotiate a new IPSec SA when the IPSec SA lifetime expires. It's also used to refresh the keying material used to create the shared secret key based on the keying material derived from the DH exchange in Phase 1. Figure 1-24 shows the negotiation of IPSec parameters between Router A and Router B.
)
Figure
1-24 Step 3: IKE Phase 2
The ultimate goal of IKE Phase 2 is to establish a secure IPSec session between endpoints. Before that can happen, each pair of endpoints negotiates the level of security required (for example, encryption and authentication algorithms for the session). Rather than negotiate each protocol individually, the protocols are grouped into IPSec transform sets. IPSec transform sets are exchanged between peers during quick mode. If a match is found between sets, IPSec session establishment continues. If no match is found, the session is torn down.
In Figure 1-25, Router A sends IPSec transform sets 30 and 40 to Router B. Router B compares its set, transform set 55, with those received from Router A. In this instance, a match occurs. Router A's transform set 30 matches Router B's transform set 55. These encryption and authentication algorithms form an SA.
When the peers agree on the security services, each VPN peer device enters the information in a security policy database (SPD). The information includes the encryption and authentication algorithm, destination IP address, transport mode, key lifetime, and so on. This information is the SA—a one-way logical connection that provides security to all traffic traversing the connection. Because most traffic is bidirectional, two SAs are required: one for inbound traffic, and one for outbound traffic. The VPN device indexes the SA with a number, a Security Parameter Index (SPI). Rather than send the SA's individual parameters across the tunnel, the source gateway, or host, inserts the SPI into the ESP header. When the IPSec peer receives the packet, it looks up the destination IP address, IPSec protocol, and SPI in its SA database (SAD) and then processes the packet according to the algorithms listed under the SPD.
)
Figure
1-25 Step 3: IPSec Transform Sets
The IPSec SA is a compilation of the SAD and SPD. The SAD identifies the SA destination IP address, IPSec protocol, and SPI number. The SPD defines the security services applied to the SA, encryption and authentication algorithms, and mode and key lifetime. For example, in the corporate-to-bank connection shown in Figure 1-26, the security policy provides a very secure tunnel using 3DES, SHA, tunnel mode, and a key lifetime of 28,800. The SAD value is 192.168.2.1, ESP, and SPI-12. For the remote user accessing e-mails, a less secure policy is negotiated using DES, MD5, tunnel mode, and a key lifetime of 28,800. The SAD values are a destination IP address of 192.169.12.1, ESP, and SPI-39.
With a password on your company PC, the longer you keep it, the more vulnerable it becomes. The same thing is true of keys and SAs. For good security, the SA and keys should be changed periodically. There are two parameters: lifetime type and duration. How is the lifetime measured? Is it measured by the number of bytes transmitted or the amount of time transpired? The second parameter is the unit of measure: kilobytes of data or seconds of time. An example is a lifetime based on 10,000 KB of data transmitted or 28,800 seconds of time expired. The keys and SAs remain active until their lifetime expires or until an external event—such as the client dropping the tunnel—causes them to be deleted.
)
Figure
1-26 Step 3: SA
Step 4: Data Transfer
After IKE Phase 2 is complete and quick mode has established IPSec SAs, traffic is exchanged between Hosts A and B via a secure tunnel, as shown in Figure 1-27. Interesting traffic is encrypted and decrypted according to the security services specified in the IPSec SA.
)
Figure
1-27 Step 4: IPSec Data Transfer
Step 5: IPSec Tunnel Termination
IPSec SAs terminate through deletion or by timing out. An SA can time out when a specified number of seconds has elapsed or when a specified number of bytes has passed through the tunnel. When the SAs terminate, the keys are also discarded. When subsequent IPSec SAs are needed for a flow, IKE performs a new Phase 2 (and, if necessary, a new Phase 1) negotiation. A successful negotiation results in new SAs and new keys. New SAs are usually established before the existing SAs expire so that a given flow can continue uninterrupted. This final step is shown in Figure 1-28.
)
Figure
1-28 Step 5: IPSec Tunnel Termination