IPsec (Internet Protocol Security) is essentially a method to encrypt connections between devices. When a device is using a public network, IPsec can help oversee the traffic and prevent the data that the device sent via the network from leaking out to the public.
One use case of this set of secure protocols is how users can do private transactions using a public network without revealing the bank credentials to other people using the same public network. This use case underscores the importance of applying IPsec when accessing the internet on public WiFi. Sometimes, a user needs to urgently access the internet to log into a database containing sensitive information, but the only available internet connection is public WiFi. Encryption, in this case, works as a secret code to conceal everything the user is doing on the device. The application of IPsec becomes important in certain circumstances, such as when other people using the same WiFi can access what the user is doing on the respective device. IPSec can help users hide internet activities by masking and encrypting the information.
Table of Contents
What is IPSec?
IPSec is shorthand for Internet Protocol Security, a framework for securing online connections. The first half of the term, “Internet Protocol (IP),” is the main routing protocol used to access certain websites, serving an essential role in what makes the internet work. The “Security” part of the term is how certain IPSec platforms can encrypt or conceal information from other users who might be using the same public network as the device. When referring to IPSec as a whole, on the other hand, the term refers to a whole suite of protocols, such as “Authentication Header,” “Encapsulating Security Protocol” and “Internet Key Exchange,” which is used to set up the “Security Association.” These are unique features that each function to make a connection secure.
How does IPSec Work?
When someone uses a computer to access the internet when IPSec is on, the technology allows for mutual authentication, meaning that two or more parties authenticate each other simultaneously. When this happens, IPSec will have to use cryptographic keys, represented by a series of random characters (numbers, signs and letters) to encrypt the device’s messages over to the network.
Access to the internet requires the device to go through a series of layers called the “OSI Networking Model.” This model has seven layers, and although IPSec is a commonly used method applied by VPN services, there is also another method, called SSL/TLS, that operates on another layer of the OSI model. At the top of the OSI model is the application layer, where the user’s browser is located. Meanwhile, the physical layer of the OSI model is at the bottom, such as the electrical pulses that travel through the wires used by internet routers.
IPSec essentially operates in the very middle of the OSI model, between layers 3 and 4. Layers 3 and 4 are known to be the network and transport layers, respectively. The main function of the transport layer is to coordinate and manage the data transfer process between two parties or hosts. Some examples of this are determining how much data to transfer, at what rate and where the data is going. Once the transfer layer has set all this, the data is then handed over to the network layer, where the operations are mostly controlled by the code running on the routers and other components that make up the network.
The next thing to understand is the first step of the IPSec process. IPSec is first executed by establishing a Security Association (SA) between two communicating hosts. This is where exchanging cryptographic keys happens to improve the network’s security. As mentioned before, the two hosts—or two computers—authenticate each other to determine the exact type of encryption so that the information matches when sending and receiving data.
What is an IPSec VPN?
Virtual private networks (VPNs) help encrypt connections between two or more computers. When accessing the internet through a public network with a VPN on, the user can be assured that the encryption provided by the VPN service prevents the data sent by the user over to the network from leaking to the public. The VPN definition is only an umbrella term because of the different types of VPNs, depending on the protocols used and how these protocols are configured to establish a secure connection.
The real connection between IPSec and VPNs is that IPSec is just another method for a VPN service to function. IPSec is not the only technology that enables VPN services because there is another method for securing and encrypting the user’s data, namely SSL/TLS. But that method operates in a different layer in the OSI Networking Model compared to IPSec. Each technology differs in function, such as the fact that IPSec is generally used in corporate settings because of the availability of directly connecting the host to a private network, while SSL/TLS is generally used for remote access.
What is the Procedure for Connecting to an IPSec VPN?
All VPN services require users to install the program for the service to work, so this is a necessary step to ensure that the device matches the program’s system requirements. When an IPSec VPN is successfully installed, the first thing a user should do is open up the VPN program and log in with the registered credentials.
Note:
Like most internet-based services, IPSec VPNs are password-protected, and users will have to register a user account and pay for the services to access the features. Some VPN services have a two-factor authentication feature to ensure that users’ data remains protected from outside sources. Simply stealing the user’s password does not guarantee that the hacker can steal the data right away.What do IPSec Connections Include?
The following features of an IPSec connection are needed to successfully execute the encryption, including instructions for the receiving host to decrypt the data packets.
- Packet headers and trailers
- Key exchange
- Encryption
- Authentication
- Transmission
- Decryption
1. Packet Headers and Trailers
When data is sent over to the network and back, the file is broken down into fragments containing the data information. These are called packets. Here, the actual data sent is called payload, and another thing inside a packet is the header. Headers act as introductory information that comes with the entirety of the packet so that the receiving computer knows the next steps when the operating system receives the payload. What IPSec does, in this case, is essentially add headers that instruct the receiver to encrypt the payload. Trailers are also added, with the same function as a header, but come after the payload is received, rather than as a piece of introductory information.
2. Key Exchange
Keys are necessary for encryption. These keys are represented by a random series of characters that can be used to encrypt and decrypt the data being exchanged over the network. These keys act like automatic passwords for the computer to access the received information.
3. Encryption
The use of keys is mainly to encrypt the payload of the packet. This encryption keeps the information inside the payload from being exposed to the public network.
4. Authentication
When using IPSec, each encrypted packet is sent out with an authentication. This works like a seal or proof that the packet is legitimate and informs the receiving host that the packet comes from a trusted source.
5. Transmission
IPSec uses User Datagram Protocol (UDP) to pass along the data packets instead of Transmission Control Protocol (TCP). This allows IPSec to deliver the packets while bypassing the firewall.
6. Decryption
When the data packet arrives at the destination, the receiving host can decrypt the packet to access the payload. At this point, applications that use the internet, such as a browser, can use this data.
What Protocols are Used in IPSec?
To encrypt and pass along the data, IPSec uses different protocols. While IPSec is used as a single term, the service consists of different protocols. Essentially, IPSec should be viewed as a suite of protocols instead of just one encryption and transmission process. Three protocols used in IPSec are given below.- Authentication Header (AH)
- Encapsulating Security Protocol (ESP)
- Security Association (SA)
1. Authentication Header (AH)
While other headers include a set of instructions for the receiving host on what to do with the data packet, this protocol proves that the data packets being received are legitimate and delivered by a trusted source. In essence, Authentication Header (AH) gives the connected computers assurance that the exchanged packets will not compromise the encrypted information.
2. Encapsulating Security Protocol (ESP)
Encapsulating Security Protocol (ESP) is another layer of encryption that specifically encrypts the IP header and the payload of packets. When transport mode is used, this protocol only encrypts the payload. ESP has a separate header and trailer for each data packet. At the receiving end of the packet, the first thing the system has to do is authenticate the packet with the authentication header. Once the process is successful, the operating system will decrypt the encrypted data.
3. Security Association (SA)
Security Association (SA) is the first protocol that IPSec establishes once the service is in operation. While regarded as a single type of protocol used in IPSec, SA refers to several protocols for negotiating encryption keys and algorithms between hosts. There are many different SA types, and the network administrator will have to configure and set up what kind of Security Association should be established between the computer and the network.
What Port Does IPSec Use?
A network port is where the data gets stored in a computer and where the computer keeps track of all the connections and information about the data packets. When a data packet gets stored in a certain port within a computer, that computer’s operating system can determine which process the packet belongs to. IPSec usually operates using port 500.
What are the IPSec Modes?
The two IPSec modes are listed and explained below.
- Tunnel Mode
- Transport Mode
1. Tunnel Mode
Tunnel mode is normally used by two routers, each with a dedicated usage to a host. This is also where the actual IP header is encrypted. The most common use of tunnel mode is for configurations. This mode is what IPSec uses to establish a secure connection between two different networks. While the tunnel mode will work without an IPSec VPN because two hosts are able to connect directly, IPSec here is used to enhance the security of the data.
2. Transport Mode
Meanwhile, the transport mode only encrypts the packet, not the IP header containing the packet’s original destination. The most common transport mode use case is when the user requires fast and secure end-to-end communication between two hosts. Compared to tunnel mode, transfer mode also has a bigger MTU capacity, so this mode doesn’t have to fragment a lot of packets during a session.
How does IPSec Impact MSS and MTU?
Maximum Segment Size (MSS) and Maximum Transmission Unit (MTU) are measurements of packet size. MSS only measures the size of the packet’s payload (the data the packet is carrying), while MTU measures the size of the whole packet, including the header. Packets that exceed the maximum size that the MTU can measure will be broken down into smaller fragments before being reassembled again. In contrast, packet payloads that exceed the MSS will simply be dropped and cannot proceed to the destination.
IPSec usually adds another set of layers in the packet payload or the whole packet. This can quickly take up several bytes, and both MTU and MSS will handle the transmission process noticeably slower than without IPSec. Each whole packet typically contains 1,500 bytes of data. With the standard headers and TCPs being about 20 bytes each, the usual amount of bytes that the payload contains is about 1,460. Another set of layers can add up to 40 to 60 bytes of data when IPSec is used. This excess will be fragmented by the MTU and then reassembled to complete the process.
Can IPSec be Hacked?
Yes. IPSec can be hacked using a method called the Bleichenbacher attack. The method was first demonstrated to be successful in hacking a network with an IPSec service by researchers from the Horst Görtz Institute for IT Security in Germany and Opole University in Poland.
The attack process involves purposely filling an encoded message with errors and then repeatedly sending that message to the server. In usual cases, the server’s response will contain hints about the contents of the encrypted information, and the hacker can study the responses with each send of the error-filled message. This way, the hacker can study the information based on the responses and gain increasingly accurate intelligence about the whole content of the encrypted data.
What are the Historical Aspects of IPSec?
The internet started small, and security wasn’t a concern back in the early-1980s since only a few dozen machines were connected. When the internet was first established, security was much easier for organizations and researchers with the simple implementation of physical locks and security systems to prevent someone from snooping around sensitive information. However, the internet quickly grew by the start of the 2000s. With this rapid growth in popularity, the security of internet networks also dissipated over time, as simply placing a physical security system to protect hardware that contained information wasn’t enough anymore. Hackers can execute operations remotely without going to the physical site.
What is the Difference between IPSec Tunnel Mode and IPSec Transport Mode?
The main difference between IPSec tunnel mode and IPSec transport mode is that tunnel mode refers to the tunneling or connecting of two hosts, and certain configurations of this mode create an additional layer of headers to encrypt. These headers include the information for the intermediary routers about where to forward the packets. At each end of the tunnel, the exchanged packets are decrypted so that those can arrive at the assigned destinations.
In transport mode, on the other hand, intermediary routers can access the information inside the header without decryption, except for the payload.
What is the Difference between IPSec VPN and SSL VPN?
An IPSec VPN is the more traditional approach to protecting privacy on the internet, while an SSL VPN was developed only recently, following the introduction of cloud-based services. The main difference lies in how each technology operates and the complexity of setting up the VPN.
IPSec VPNs are generally more complicated since users have to install third-party client software. SSL VPNs, on the other hand, are already supported by browsers. These two types of VPNs operate in different layers of the OSI Networking Model. IPSec VPNs work within the network layer of the OSI model, while SSL works in the application layer of the model.