Incident Response

When breaches happen, respond fast. Collect evidence. Contain the threat.

The IR Lifecycle

Incident Response follows a structured methodology. Speed matters, but so does doing it right.

1⃣

Preparation

Tools, training, playbooks ready

2⃣

Identification

Detect and confirm the incident

3⃣

Containment

Stop the bleeding, limit damage

4⃣

Eradication

Remove the threat completely

5⃣

Recovery

Restore normal operations

Lessons Learned

Document and improve

First Responder Collection

Order of Volatility

Collect most volatile evidence first! Memory disappears on reboot, disk persists.

# === VOLATILE DATA (collect FIRST) ===

# Current date/time
date; uptime

# Who is logged in NOW
w
who

# Running processes
ps auxf > /evidence/processes.txt

# Network connections
netstat -tulpn > /evidence/connections.txt
ss -tulpn > /evidence/sockets.txt

# Open files
lsof > /evidence/open_files.txt

# Memory dump (if possible)
dd if=/dev/mem of=/evidence/memory.dump

Persistent Evidence

# === NON-VOLATILE DATA ===

# User accounts and recent changes
cat /etc/passwd
cat /etc/shadow # Requires root
lastlog

# Login history
last -f /var/log/wtmp
lastb -f /var/log/btmp # Failed logins

# Scheduled tasks (persistence)
crontab -l
ls -la /etc/cron.*
cat /etc/crontab

# Startup scripts
ls -la /etc/init.d/
systemctl list-unit-files --type=service

# Recent file modifications
find / -mtime -1 -type f 2>/dev/null > /evidence/recent_files.txt

Timeline Analysis

Building a timeline is critical. When did the attacker get in? What did they touch?

# Find files modified in last 24 hours
find / -mtime -1 -ls 2>/dev/null | sort -k9

# Find files by specific date range
find / -newermt "2025-12-24" ! -newermt "2025-12-25" -ls

# Check bash history timestamps (if HISTTIMEFORMAT set)
cat ~/.bash_history

# Auth log timeline
grep "Dec 25" /var/log/auth.log | head -50

# Create master timeline
find / -type f -printf "%T+ %p\n" 2>/dev/null | sort > timeline.txt

Evidence Integrity

Hash Everything

SHA256 hash all collected evidence immediately. Proves it wasn't tampered with.

Chain of Custody

Document who collected what, when, and how. Required for legal proceedings.

# Hash collected files
sha256sum /evidence/* > /evidence/hashes.txt

# Create evidence manifest
ls -la /evidence/ >> /evidence/manifest.txt
date >> /evidence/manifest.txt
whoami >> /evidence/manifest.txt

LAB: First Responder Scenario

ALERT: A potential breach has been detected on server 10.0.0.50 at 03:47 AM. Collect volatile and non-volatile evidence before it's lost!

☐Document date/time and uptime

☐Check who is logged in

☐Capture network connections

☐Hash evidence for integrity

ir-response

Incident Response Terminal
===========================
Server: 10.0.0.50 | Status: ALERT
Time of detection: 03:47 AM

Collect evidence in order of volatility!
Commands: date, uptime, w, who, ps, netstat, lsof, sha256sum

root@incident:~#

Ready to Test Your Skills?

Take the Quiz → Back to Script House

Completing CLH-013 to CLH-015 earns: CLI Architect