Web Enumeration

Probe web servers. Discover hidden paths. Extract valuable information from HTTP.

curl - The Web Swiss Army Knife

curl transfers data with URLs. Essential for probing web servers, APIs, and extracting information.

# Basic GET request
curl https://example.com

# Show response headers only
curl -I https://example.com
HTTP/2 200
server: nginx
x-powered-by: PHP/7.4  # Version leak!
set-cookie: session=abc123

# Follow redirects
curl -L http://example.com

# Include headers in output
curl -i https://example.com

# Send POST data
curl -X POST -d "user=admin&pass=test" https://example.com/login

wget - Download & Mirror

# Download a file
wget https://example.com/file.pdf

# Download with custom filename
wget -O report.pdf https://example.com/file.pdf

# Mirror entire website (careful!)
wget --mirror --convert-links --page-requisites https://target.com

# Download robots.txt (common recon target)
wget https://example.com/robots.txt
cat robots.txt
User-agent: *
Disallow: /admin/
Disallow: /backup/
Disallow: /config/  # Interesting paths!

HTTP Headers That Leak Info

Server Header

Reveals web server software and version (Apache, nginx, IIS)

X-Powered-By

Backend technology (PHP, ASP.NET, Express). Check for outdated versions.

Set-Cookie

Session handling details. Missing HttpOnly or Secure flags = vulnerability.

Security Headers

Missing CSP, X-Frame-Options, HSTS indicate weak security posture.

Common Files to Check

Path	What It Reveals
`/robots.txt`	Hidden directories the site doesn't want indexed
`/sitemap.xml`	Full site structure
`/.git/`	Exposed source code repository
`/.env`	Environment variables, credentials
`/backup/`	Database dumps, old files
`/admin/`	Admin panels
`/phpinfo.php`	PHP configuration details

# Quick check for common files
for path in robots.txt sitemap.xml .git/config .env; do
    status=$(curl -s -o /dev/null -w "%{http_code}" "https://target.com/$path")
    echo "$path: $status"
done

robots.txt: 200
sitemap.xml: 200
.git/config: 403
.env: 404

HTTP Response Codes

Code	Meaning	Recon Value
`200`	OK	Resource exists and is accessible
`301/302`	Redirect	Follow to find actual location
`401`	Unauthorized	Auth required - path exists!
`403`	Forbidden	Exists but blocked - interesting
`404`	Not Found	Doesn't exist (usually)
`500`	Server Error	May reveal error messages

LAB: Web Enumeration Simulator

Target: https://target-app.local - Enumerate this web server to find hidden paths and information leaks.

☐Check HTTP headers for server info

☐Retrieve robots.txt

☐Find a hidden admin path

☐Check for exposed .git directory

web-enum

Web Enumeration Lab
====================
Target: https://target-app.local

Commands: curl -I [url], curl [url], wget [url]
Try: curl -I https://target-app.local
     curl https://target-app.local/robots.txt

Ready to Test Your Skills?

Take the Quiz → Back to Script House

Completing CLH-010 to CLH-012 earns: CLI Engineer