OpenBSD manual page server

IPsec Protocols

IPsec provides these services using two new protocols: Authentication Header (AH), and Encapsulating Security Payload (ESP).

ESP can provide the properties authentication, integrity, replay protection, and confidentiality of the data (it secures everything in the packet that follows the IP header). Replay protection requires authentication and integrity (these two always go together). Confidentiality (encryption) can be used with or without authentication/integrity. Similarly, one could use authentication/integrity with or without confidentiality.

AH provides authentication, integrity, and replay protection (but not confidentiality). The main difference between the authentication features of AH and ESP is that AH also authenticates portions of the IP header of the packet (such as the source/destination addresses). ESP authenticates only the packet payload.

Authentication Header (AH)

AH works by computing a value that depends on all of the payload data, some of the IP header data, and a certain secret value (the authentication key). This value is then sent with the rest of each packet. The receiver performs the same computation, and if the value matches, he knows no one tampered with the data (integrity), the address information (authenticity) or a sequence number (replay protection). He knows this because the secret authentication key makes sure no active attacker (man-in-the-middle) can recompute the correct value after altering the packet. The algorithms used to compute these values are called hash algorithms and are parameters in the SA, just like the authentication key.

Encapsulating Security Payload (ESP)

ESP optionally does almost everything that AH does except that it does not protect the outer IP header but furthermore it encrypts the payload data with an encryption algorithm using a secret encryption key. Only the ones knowing this key can decrypt the data, thus providing confidentiality. Both the algorithm and the encryption key are parameters of the SA.

Security Associations (SAs)

These protocols require certain parameters for each connection, describing exactly how the desired protection will be achieved. These parameters are collected in an entity called a security association, or SA for short. Typical SA parameters include encryption algorithm, hash algorithm, encryption key, and authentication key, to name a few. When two peers have established matching SAs (one at each end), packets protected with one end's SA may be verified and/or decrypted using the information in the other end's SA. The only issue remaining is to ensure that both ends have matching SAs. This may be done manually, or automatically using a key management daemon.

Further information on manual SA establishment is described in ipsec.conf(5). Information on automated key management for IKEv1 can be found in isakmpd(8) and for IKEv2 in iked.conf(5).

Security Parameter Indexes (SPIs)

In order to identify an SA we need to have a unique name for it. This name is a triplet, consisting of the destination address, security parameter index (aka SPI) and the security protocol (ESP or AH). Since the destination address is part of the name, an SA is necessarily a unidirectional construct. For a bidirectional communication channel, two SAs are required, one outgoing and one incoming, where the destination address is our local IP address. The SPI is just a number that helps us make the name unique; it can be arbitrarily chosen in the range 0x100 - 0xffffffff. The security protocol number should be 50 for ESP and 51 for AH, as these are the protocol numbers assigned by IANA.

Modes of Operation

IPsec can operate in two modes, either tunnel or transport mode. In transport mode the ordinary IP header is used to deliver the packets to their endpoint; in tunnel mode the ordinary IP header just tells us the address of a security gateway which knows how to verify/decrypt the payload and forward the packet to a destination given by another IP header contained in the protected payload. Tunnel mode can be used for establishing virtual private networks (VPNs), where parts of the networks can be spread out over an unsafe public network, but security gateways at each subnet are responsible for encrypting and decrypting the data passing over the public net. An SA will contain information specifying whether it is a tunnel or transport mode SA, and for tunnels it will contain values to fill in into the outer IP header.

Lifetimes

The SA also holds a couple of other parameters, especially useful for automatic keying, called lifetimes, which puts a limit on how much we can use an SA for protecting our data. These limits can be in wall-clock time or in volume of our data.

IPsec Examples

To better illustrate how IPsec works, consider a typical TCP packet:

If we apply ESP in transport mode to the above packet, we will get:

Everything after the ESP header is protected by whatever services of ESP we are using (authentication/integrity, replay protection, confidentiality). This means the IP header itself is not protected.

If we apply ESP in tunnel mode to the original packet, we would get:

Again, everything after the ESP header is cryptographically protected. Notice the insertion of an IP header between the ESP and TCP header. This mode of operation allows us to hide who the true source and destination addresses of a packet are (since the protected and the unprotected IP headers don't have to be exactly the same). A typical application of this is in Virtual Private Networks (or VPNs), where two firewalls use IPsec to secure the traffic of all the hosts behind them. For example:

Net A <----> Firewall 1 <--- Internet ---> Firewall 2 <----> Net B

Firewall 1 and Firewall 2 can protect all communications between Net A and Net B by using IPsec in tunnel mode, as illustrated above.

This implementation makes use of a virtual interface, enc0, which can be used in packet filters to specify those packets that have been or will be processed by IPsec.

NAT can also be applied to enc# interfaces, but special care should be taken because of the interactions between NAT and the IPsec flow matching, especially on the packet output path. Inside the TCP/IP stack, packets go through the following stages:

UL/R -> [X] -> PF/NAT(enc0) -> IPsec -> PF/NAT(IF) -> IF
UL/R <-------- PF/NAT(enc0) <- IPsec <- PF/NAT(IF) <- IF

With IF being the real interface and UL/R the Upper Layer or Routing code. The [X] stage on the output path represents the point where the packet is matched against the IPsec flow database (SPD) to determine if and how the packet has to be IPsec-processed. If, at this point, it is determined that the packet should be IPsec-processed, it is processed by the PF/NAT code. Unless PF drops the packet, it will then be IPsec-processed, even if the packet has been modified by NAT.

Security Associations can be set up manually with ipsecctl(8) or automatically with the isakmpd(8) or iked(8) key management daemons.

Additional Variables

A number of sysctl(8) variables are relevant to ipsec. These are generally net.inet.ah.*, net.inet.esp.*, net.inet.ip.forwarding, net.inet6.ip6.forwarding, and net.inet.ip.ipsec-*. Full explanations can be found in sysctl(2), and variables can be set using the sysctl(8) interface.

A number of kernel options are also relevant to ipsec. See options(4) for further information.

API Details

The following IP-level setsockopt(2) and getsockopt(2) options are specific to ipsec. A socket can specify security levels for three different categories:

For each of the categories there are five possible levels which specify the security policy to use in that category:

When a new socket is created, it is assigned the default system security level in each category. These levels can be queried with getsockopt(2). Only a privileged process can lower the security level with a setsockopt(2) call.

For example, a server process might want to accept only authenticated connections to prevent session hijacking. It would issue the following setsockopt(2) call:

int level = IPSEC_LEVEL_REQUIRE;
error = setsockopt(s, IPPROTO_IP, IP_AUTH_LEVEL, &level, sizeof(int));

The system does guarantee that it will succeed at establishing the required security associations. In any case a properly configured key management daemon is required which listens to messages from the kernel.

A list of all security associations in the kernel tables can be obtained using the ipsecctl(8) command.