Traffic Analysis Techniques

Network traffic analysis methods for threat detection

Week 4 TOPIC

Network Traffic Analysis

Network Traffic Analysis (NTA) involves monitoring network traffic to identify anomalies, threats, and operational issues. SOC analysts use NTA to detect intrusions that evade signature-based detection. According to NIST SP 800-86, network traffic is one of the four primary forensic data sources (alongside computer data, application data, and live response data).

Analysis Approaches

Flow Analysis

NetFlow/IPFIX metadata: who talked to whom, when, how much data. No payload inspection. Ideal for long-term trending, C2 beaconing detection, and identifying data exfiltration by volume. Low storage cost enables months of retention.

Deep Packet Inspection

Full packet capture and analysis. Content inspection for malware, data exfiltration, protocol violations. High storage cost (~100GB+/day on busy networks) but provides complete forensic evidence. Required for malware payload extraction.

Behavioral Analysis

ML-based anomaly detection. Baselines normal traffic patterns and alerts on deviations. Can detect zero-day attacks and insider threats without signatures. Requires tuning to minimize false positives during baselining period.

Protocol Analysis

Zeek/Bro style parsing. Extract metadata from application protocols (HTTP, DNS, SMB, TLS). Creates structured logs like conn.log, dns.log, http.log for SIEM ingestion. Best for protocol abuse detection and threat hunting.

NTA in the SOC Workflow

Traffic analysis sits at the intersection of detection, investigation, and hunting:

Phase 1

Baseline: Learn what "normal" looks like for each network segment

Phase 2

Detect: Alert on deviations from baseline or known-bad patterns

Phase 3

Investigate: Drill into alerts using full packet data when available

Phase 4

Hunt: Proactively search for threats that bypassed automated detection

Source: Eye House > Presentations > Network Traffic Analysis Open Presentation

Analysis Techniques

Technique	Data Required	Detects	Limitations
Baseline Deviation	Historical flow data	DDoS, data exfil, C2 beaconing	Requires tuning, slow attacks evade
Signature Matching	Full packets	Known exploits, malware	Zero-days, encrypted traffic
DNS Analysis	DNS queries/responses	DGA domains, tunneling, C2	DoH/DoT encryption
TLS Fingerprinting	TLS handshakes	Malware families, suspicious clients	Fingerprint spoofing
Beaconing Detection	Connection timing	C2 callbacks, implants	Jittered beacons
Entropy Analysis	Payload data	Encrypted exfil, packed malware	High entropy is also normal for compressed media
GeoIP Correlation	Connection metadata	Connections to hostile nations, impossible travel	VPNs and Tor mask true origin

Beaconing Detection Example

C2 implants call home at regular intervals. Even with jitter, statistical analysis reveals the pattern:

# Detect regular interval connections (C2 beaconing) # Splunk SPL query: index=firewall action=allow | bin _time span=1m | stats count by src_ip, dest_ip, dest_port, _time | streamstats current=f window=60 avg(count) as avg_count by src_ip, dest_ip | where count > 0 AND abs(count - avg_count) < 0.5 | stats count as beacon_intervals by src_ip, dest_ip, dest_port | where beacon_intervals > 30

DNS Tunneling Detection

Attackers encode data in DNS queries to bypass firewalls. Detection indicators include high query volume to a single domain, unusually long subdomain labels, and high entropy in query strings:

# Detect DNS tunneling via query length analysis index=dns query_type=A | eval query_len = len(query) | where query_len > 50 | stats count avg(query_len) as avg_len dc(query) as unique_queries by src_ip, answer | where count > 100 AND avg_len > 40 | table src_ip, answer, count, avg_len, unique_queries

Encrypted Traffic Blind Spot

Over 90% of web traffic is now encrypted (TLS). This means signature-based inspection cannot see payloads. SOC analysts must rely on metadata analysis: TLS fingerprinting (JA3/JA3S), certificate analysis, connection timing, data volume, and destination reputation. MITRE ATT&CK technique T1573 (Encrypted Channel) documents how adversaries use this to their advantage.

SOC Analyst Tip

JA3/JA3S Fingerprinting: TLS client fingerprints can identify malware even through encrypted traffic. The JA3 hash is generated from the TLS ClientHello message fields (TLS version, accepted ciphers, extensions). Known malware families produce consistent JA3 hashes. Tools like Zeek can extract JA3 hashes and SIEMs can correlate them against threat intelligence feeds. Example known-bad JA3: a0e9f5d64349fb13191bc781f81f42e1 (Cobalt Strike default).

Traffic Analysis Tools

Wireshark

Deep packet inspection, protocol dissection, conversation analysis. GUI-based. Supports 3,000+ protocols. Follow TCP/UDP streams, extract files, apply display filters. Industry standard for forensic packet analysis.

Zeek (Bro)

Network metadata extraction. Generates structured logs from traffic. Scriptable policy language for custom detection. Produces conn.log, dns.log, http.log, files.log, ssl.log, and more. The backbone of many NSM deployments.

tcpdump

Command-line packet capture. Filtering with BPF syntax. Lightweight, available on all Linux/Unix systems. Essential for IR when you need to capture traffic on a live system quickly. Write to pcap for later Wireshark analysis.

NetworkMiner

File extraction from pcaps. Host profiling. Credential detection. Passively reconstructs sessions and extracts images, documents, and executables transferred over the network.

Recognizing Malicious Traffic Patterns

As a SOC analyst, you need to recognize what malicious traffic looks like in your tools. These are the patterns you will encounter in real investigations:

Pattern	What You See	What It Means	MITRE ATT&CK
DNS Beaconing	Hundreds of queries to random-looking subdomains of one domain (e.g., `a8f3c2.evil.com`, `b7d1e4.evil.com`)	DNS tunneling or DGA-based C2 -- data encoded in DNS queries	T1071.004
Low-and-Slow Exfil	Small HTTPS connections (5-50 KB) to the same IP every 5 minutes, 24/7	APT-style exfiltration -- blends with normal traffic volume	T1048
Port Scanning	One source IP sends SYN packets to hundreds of destination ports on one host	Reconnaissance -- attacker mapping open services	T1046
Lateral Movement	Workstation A connects to Workstation B on port 445 (SMB) or 5985 (WinRM)	Workstation-to-workstation SMB is almost never legitimate -- investigate immediately	T1021.002
ICMP Tunnel	Unusually large ICMP packets (>100 bytes) or high ICMP volume to one destination	Data exfiltration hidden in ping packets	T1095
Certificate Anomaly	Self-signed cert on port 443, cert CN doesn't match domain, recently issued cert	Possible C2 infrastructure or man-in-the-middle	T1573.002

Common Wireshark Filters

# DNS queries to external servers dns && !ip.addr == 192.168.0.0/16 # HTTP POST requests (potential exfil) http.request.method == "POST" # Large outbound transfers tcp && ip.src == 192.168.1.0/24 && frame.len > 1400 # SMB traffic (lateral movement) smb || smb2 # TLS with self-signed certs (suspicious C2) tls.handshake.type == 11 && x509sat.printableString == "" # Unusual ports (common for reverse shells) tcp.port == 4444 || tcp.port == 8888 || tcp.port == 1337

Traffic Analysis Scenarios

Analyze the following network traffic samples and identify the threat. Each scenario presents real-world traffic patterns a SOC analyst would encounter during an investigation.