Digging Secure Tunnels with IPsec

Introduction

The Internet was born using plain text and no encryption. For a long time the TCP/IP protocol suite had no mechanism for cryptographically protecting transported data. Encryption was added at the application layer — Netscape's Secure Socket Layer (SSL) being a famous example. The design process of IPv6 incorporated encryption into the protocol itself, and the IPsec (IP security) framework came into existence. IPsec provides encryption and authentication at the packet level. While IPsec is obligatory for IPv6, you can optionally use it with IPv4. The Linux 2.6.x kernel series added full IPsec functionality to the main source tree. In this article we will explore how we can use IPsec to build encrypted data paths between networked machines.

The Parts of IPsec

IPsec consists of a number of protocols. Encryption was not the only design criteria. Protection against replay attacks, detection of unauthorised packet modification and correctly authenticating the communication partners are also included in the design requirements. IPSec also offers methods to manage keys used for encrypted communication. The protocols by name have the following task.

Authentication Header (AH)
This protocol protects against replay attacks and makes sure that the communication partners are "correct" (it verifies authentication).
Encapsulating Security Payload (ESP)
This protocol provides encryption. It also protects against replay attacks and in addition ensures authentication.
Internet Key Exchange (IKE)
This is a hybrid protocol that establishes the security policies between two IPsec endpoints and provides the authentication keys for services such as IPsec. If you provide the keys manually, you don't need to use IKE.

AH and ESP packets transport data. Both are new protocols whose protocol numbers can be found in /etc/protocols. AH and ESP are managed by the kernel, while IKE is managed by a userspace program.

Preparing Kernel and System

In order to use IPsec your kernel needs to have some code options enabled. Most modern GNU/Linux distributions have IPsec-capable kernels. If you want to compile your own, make sure that you have the following options set:

IPsec user configuration interface
PF_KEY sockets
IP: AH transformation
IP: ESP transformation
IP: IPComp transformation
IP: tunnel transformation

These options can be found in the section Networking --> Networking options. You also have to enable some or all cryptographic ciphers in the Cryptographic options submenu. You will need at least MD5, SHA1, HMAC, DES, Triple DES EDE and AES. Remember, the encryption is handled by the kernel. If the kernel doesn't know about a cipher, it cannot use it. This is important in case you want to speak to other IPsec devices or hosts (such as MS Windows or Cisco machines). If you want to use IPsec with IPv6, then you have to make sure the IP transformations are set for IPv6 as well. I have two screenshots for you where you can see the make menuconfig menu: screenshot 1 and screenshot 2. If your machine acts as a router, you might wish to consider activating the IP: advanced router option, too. The kernel treats packet queues differently if you use the advanced routing option.

Now our kernel can handle IPsec. We now need some tools to make it work. We will need at least the ipsec-tools package. This is the name of the project and the Debian package. If we want to deal with key management and IKE, we need another program. The Linux IPsec stack can work together with pluto from the Openswan project, OpenBSD's isakmpd or racoon from the KAME project. The use of IKE is optional though. We will use racoon in our examples.

Manual Keying, Policies, Tunnel and Transport Mode

IPsec can be used to link networks via tunnels by using the so-called tunnel mode. In its simplest form it can also be used to encrypt network traffic between two or more hosts by using the transport mode. The only things you need for that are the keys and a way to tell the kernel which packets need to be sent via IPsec. The 2.6.x kernels have no special device for handling IPsec packets. Everything is sent over the already existing network interfaces. The Security Policy Database (SPD) decides which packets are to be handled by IPsec. In order to manipulate this database you need the setkey command from the ipsec-tools package. Usually you prepare a file with all the settings and activate it by using setkey -f /etc/setkey.conf. As an example, let's say we want to enable IPsec between the machines 10.0.0.23 and 10.0.0.42. The policy for telling the kernel looks like this:

#!/usr/sbin/setkey -f
#
# SPD for 10.0.0.23
#
spdadd 10.0.0.23 10.0.0.42 any -P out ipsec
       esp/transport//require
       ah/transport//require;

spdadd 10.0.0.42 10.0.0.23 any -P in ipsec
       esp/transport//require
       ah/transport//require;

The first policy states that any packet coming from 10.0.0.23 and leaving for 10.0.0.42 has to be encapsulated in IPsec packets. Transport mode has to be used. The policy is valid for ESP and AH packets alike (that's why we have to use spdadd twice). IPsec is mandatory as indicated by the keyword "require". If one of the hosts lacks the right key or hasn't initialised its SPD, there won't be any traffic because it can't be encrypted. The second policy is the first one reversed. The IP addresses are swapped and the direction is changed from out to in.

So the kernel knows when to use IPsec. There are still no keys. Apart from that I told you about authentication checks that the IPsec protocols can do for us. Our setkey.conf needs to be extended to include this information as well. setkey also defines the Security Association Database (SAD). The SAD tells the kernel who our neigbours are and how we can make sure that we are not talking to an impostor. Extending our setkey.conf by the following lines enables authentication and encryption. In addition to that we supply the keys for every host.

# AH SAD entries with 160 bit keys
add 10.0.0.23 10.0.0.42 ah 0x200 -A hmac-sha1 0x46915c30ed7e2465b42861b6ab19f2772813020c;
add 10.0.0.42 10.0.0.23 ah 0x300 -A hmac-sha1 0xc4dac594f8228e0b94a54758f7fbf2fdf4e37f3e;

# ESP SAD entries with 192 bit keys
add 10.0.0.23 10.0.0.42 esp 0x201 -E rijndael-cbc 0xa3993b3dfc41ef0a1aa8d168a8bf2c27e48249ac17b61e09;
add 10.0.0.42 10.0.0.23 esp 0x301 -E rijndael-cbc 0x8f6498928ba354bd45cfad147f54c67b3b742896b3bafc02;

Again we have a line for every direction. The IP addresses are reversed, but we use a different key for every IP and protocol. The bit length of the key corresponds to the authentication or encryption algorithm used. The switches -A and -E indicate the algorithm to use for AH and ESP respectively. hmac-sha1 requires a key length of 160 bits or 20 bytes. rijndael-cbc can be used with 128, 192 or 256 bits. The example uses 192 bits or 24 bytes. The man page of setkey has a table with all possible values for every supported algorithm. Bear in mind that the kernel must also have a module for the algorithm in its cryptographic options or else you cannot use this particular algorithm. The hexadecimal value behind the protocol name is called Security Parameters Index (SPI). The SPI identifies a set of parameters used for the IPsec connection in combination with the IP addresses involved. When doing manual keying, make sure that the SPIs are unique. Speaking of unique, make sure that your keys are unique and random. Never use any keys that have been published! I used Ralf's method from the IPsec HOWTO to extract the sample keys from the Linux random device.

# dd if=/dev/random count=24 bs=1 | xxd -ps
24+0 records in
24+0 records out
8f6498928ba354bd45cfad147f54c67b3b742896b3bafc02
24 bytes transferred in 0.000180 seconds (133298 bytes/sec)

Set count to the desired byte amount. The command xxd used to convert the binary output from the device is part of the vim package.

Testing Transport Mode

We are now ready to test our configuration. In order to do this you will need the full setkey.conf. I only added two lines that clear the SAD and SPD before loading new rules, just to be sure.

flush;
spdflush;

Copy it to your hosts. Be careful, we've only created the setkey.conf for 10.0.0.23. If you use this file on 10.0.0.42 you have to swap the policy for the direction of the packet flow (the in and out keywords for the SPD). Now use a root shell on 10.0.0.23 and enter the command:

setkey -f /path/to/setkey.conf

Check if you can ping 10.0.0.42. This should not be possible, because we told 10.0.0.23 to communicate with 10.0.0.42 over IPsec only. If you run the setkey command on 10.0.0.42 as well, you should be able to ping 10.0.0.42 from 10.0.0.23. Take a look with your favourite sniffer in order to make sure the kernel isn't playing unencrypted tricks on you. If you use an ICMP ping, the sniffer should only show you encrypted AH or ESP packets. The same goes for TCP and UDP transmissions.

Next time we will configure an IPsec tunnel to connect two different networks and we will take a look at automatic keying with X.509 certificates.