Securing Traffic Across the Internet
VPN tunneling protocols that protect data in transit
Your company has two offices — HQ in Jacksonville and a branch in Atlanta. They need to communicate securely over the public Internet. You need to build a VPN tunnel. But what protocols make that tunnel work? Let's break down IPSec and GRE.
Authentication Header (51) and Encapsulating Security Payload (50)
Internet Key Exchange — the handshake. UDP ports 500 and 4500.
Generic Routing Encapsulation — Protocol 47. Tunneling without encryption.
10 Slides | N10-009 Objective 1.4 | Chapter 2
Data travels as plaintext across shared ISP infrastructure. Anyone on the path can intercept and read it — packet sniffing, man-in-the-middle attacks.
Even if traffic is captured on the public Internet, the payload is encrypted and cannot be read without the session keys.
GRE wraps packets inside IP headers, creating a virtual tunnel. It does NOT encrypt — data is still readable. But GRE can carry protocols IPSec alone cannot (multicast, IPv6, routing updates).
GRE handles encapsulation and multicast. IPSec provides encryption and authentication. Together they give you a fully functional, secure tunnel.
IPSec is a suite of protocols — not a single protocol. It operates at Layer 3 (Network layer) and protects IP packets end-to-end. It defines how to authenticate, encrypt, and integrity-check traffic between two endpoints.
Encrypts only the payload (data). The original IP header remains intact and visible. Used for host-to-host communication.
IP header visible — routing works normally
Encrypts the entire original packet (IP header + payload) and wraps it in a new outer IP header. Used for site-to-site VPN gateways.
Original addresses hidden — new outer IP routes through Internet
The firewalls at both sites use Tunnel Mode. Each firewall wraps the internal packet in a new IP header addressed to the other firewall's public IP. Internal RFC 1918 addresses stay hidden from the Internet.
No encryption. The payload is completely visible to anyone who captures the traffic. AH provides zero confidentiality.
AH Packet Structure
Payload is NOT encrypted — readable by anyone
AH authenticates the entire IP header including source and destination IP addresses. When NAT modifies those addresses, the authentication check fails. AH breaks behind NAT.
Exam Tip
AH = Authentication + Integrity, but NO encryption. If a question asks about confidentiality or encryption, AH is the wrong answer. AH cannot pass through NAT.
This is what virtually all real-world VPNs use — ESP in Tunnel Mode.
ESP does not authenticate the outer IP header (unlike AH), so NAT address modification does not break it. When NAT is detected between endpoints, NAT-T wraps ESP inside UDP port 4500, allowing it to pass through NAT devices.
ESP Packet Structure (Tunnel Mode)
Entire original packet is encrypted — payload cannot be read
Exam Tip
ESP does everything AH does, plus encryption. ESP works with NAT (via NAT-T on UDP 4500). Protocol number 50.
| Feature | AH Protocol 51 | ESP Protocol 50 |
|---|---|---|
| Encryption | NO | YES |
| Authentication | YES | YES |
| Data Integrity | YES | YES |
| Anti-Replay | YES | YES |
| NAT Compatible | NO — fails behind NAT | YES — with NAT-T (UDP 4500) |
| What it covers | Entire packet incl. IP header | Payload only (outer IP hdr not authenticated) |
| Protocol Number | 51 | 50 |
| Real-world use | Rarely deployed — mostly exam content | Standard in all production VPNs |
In practice, ESP has replaced AH in almost every deployment. AH is heavily tested on the exam because the encryption vs. no-encryption distinction is a classic distractor question. Know it cold: ESP = encryption + auth. AH = auth only, no encryption.
Before Jacksonville and Atlanta can pass any encrypted traffic, their firewalls must first agree on who they are and what encryption to use. That negotiation is IKE's job.
An SA is a one-way agreement between two peers defining the cryptographic parameters for that direction of traffic. Every IPSec tunnel has at minimum two SAs — one in each direction.
Exam Tip
IKE Phase 1 = authenticate + establish secure channel (UDP 500). IKE Phase 2 = negotiate the actual tunnel (still UDP 500, or UDP 4500 with NAT-T). If asked what port IPSec uses to negotiate: UDP 500.
GRE creates a virtual point-to-point tunnel by encapsulating packets inside IP. The original packet — regardless of its type — becomes the payload of a new IP packet.
GRE Packet Structure
+24 bytes overhead per packet (20B outer IP + 4B GRE)
GRE provides no encryption, no authentication, no integrity checking. Traffic inside a GRE tunnel is fully readable. GRE is a dumb pipe — it moves packets, nothing more.
IPSec alone cannot carry multicast traffic or dynamic routing protocols (OSPF, EIGRP use multicast). GRE can. The combination:
Jacksonville and Atlanta can run OSPF across the VPN because GRE carries the multicast OSPF hellos, and IPSec encrypts everything.
Exam Tip
GRE = tunneling without security. Protocol 47. If a question involves sending multicast or routing protocol traffic across a VPN, GRE over IPSec is the answer.
These appear in the Protocol field of the IP header — they tell the receiving host what is encapsulated inside the IP packet.
How this looks in Wireshark / a packet capture:
IKE Phase 1 authenticated both firewalls (UDP 500). Phase 2 negotiated ESP encryption (Protocol 50). GRE (Protocol 47) wraps the OSPF multicast traffic so routing updates flow between sites. The tunnel is live. All traffic between the two offices is encrypted and authenticated end-to-end.