System Administration & Automation Lab

1

Automation Fundamentals

Beginner

Before automating, learn to identify good candidates for automation and structure your scripts properly.

When to Automate?

Good candidates: Repetitive tasks, error-prone manual processes, scheduled maintenance, data collection/reporting.

Bad candidates: One-time tasks, tasks requiring human judgment, tasks that change frequently.

Rule of thumb: If you do it more than twice, automate it!

Script Structure Best Practices

Bash PowerShell Python

                    #!/bin/bash
# Script: system_health.sh
# Purpose: Daily system health check
# Author: SysAdmin
# Version: 1.0

set -e  # Exit on error
set -u  # Error on undefined vars

# Configuration
LOG_DIR="/var/log/health"
DATE=$(date +%Y-%m-%d)
LOG_FILE="${LOG_DIR}/health_${DATE}.log"

# Functions
log_message() {
    echo "[$(date '+%H:%M:%S')] $1" | tee -a "$LOG_FILE"
}

check_disk() {
    log_message "Checking disk usage..."
    df -h | tee -a "$LOG_FILE"
}

# Main execution
main() {
    log_message "=== Health Check Started ==="
    check_disk
    log_message "=== Health Check Complete ==="
}

main
                

                    # Script: System-Health.ps1
# Purpose: Daily system health check
# Author: SysAdmin

# Configuration
$LogDir = "C:\Logs\Health"
$Date = Get-Date -Format "yyyy-MM-dd"
$LogFile = "$LogDir\health_$Date.log"

# Ensure log directory exists
if (-not (Test-Path $LogDir)) {
    New-Item -ItemType Directory -Path $LogDir -Force
}

# Functions
function Write-Log {
    param([string]$Message)
    $Timestamp = Get-Date -Format "HH:mm:ss"
    "[$Timestamp] $Message" | Tee-Object -FilePath $LogFile -Append
}

function Get-DiskHealth {
    Write-Log "Checking disk usage..."
    Get-WmiObject Win32_LogicalDisk -Filter "DriveType=3" |
        Select-Object DeviceID,
            @{N='Size(GB)';E={[math]::Round($_.Size/1GB,2)}},
            @{N='Free(GB)';E={[math]::Round($_.FreeSpace/1GB,2)}} |
        Tee-Object -FilePath $LogFile -Append
}

# Main
Write-Log "=== Health Check Started ==="
Get-DiskHealth
Write-Log "=== Health Check Complete ==="
                

                    #!/usr/bin/env python3
"""
Script: system_health.py
Purpose: Daily system health check
Author: SysAdmin
"""

import os
import shutil
from datetime import datetime
from pathlib import Path

# Configuration
LOG_DIR = Path("/var/log/health")
DATE = datetime.now().strftime("%Y-%m-%d")
LOG_FILE = LOG_DIR / f"health_{DATE}.log"

def log_message(message: str) -> None:
    """Log a message with timestamp."""
    timestamp = datetime.now().strftime("%H:%M:%S")
    log_line = f"[{timestamp}] {message}"
    print(log_line)
    with open(LOG_FILE, 'a') as f:
        f.write(log_line + "\n")

def check_disk() -> None:
    """Check disk usage."""
    log_message("Checking disk usage...")
    total, used, free = shutil.disk_usage("/")
    log_message(f"Total: {total // (2**30)} GB")
    log_message(f"Used: {used // (2**30)} GB")
    log_message(f"Free: {free // (2**30)} GB")

if __name__ == "__main__":
    LOG_DIR.mkdir(parents=True, exist_ok=True)
    log_message("=== Health Check Started ===")
    check_disk()
    log_message("=== Health Check Complete ===")
                

Tasks

Identify 5 tasks in your environment that could be automated
Create a script template with proper header, configuration section, and functions
Implement logging that records timestamps and outputs to file
Add error handling to your script template
Document your script with comments explaining each section

Related Visualizers

Automation Introduction

2

Log Analysis & Monitoring

Intermediate

Automate log analysis to detect issues before they become problems.

Scenario

Your web server generates thousands of log entries daily. You need to automatically detect failed login attempts, 500 errors, and unusual traffic patterns.

Log Parsing Examples

Bash PowerShell Python

#!/bin/bash
# Analyze auth.log for failed SSH attempts

LOG_FILE="/var/log/auth.log"
THRESHOLD=5

echo "=== Failed SSH Attempts ==="

# Count failed attempts per IP
grep "Failed password" "$LOG_FILE" | \
    awk '{print $(NF-3)}' | \
    sort | uniq -c | sort -rn | \
    while read count ip; do
        if [ "$count" -ge "$THRESHOLD" ]; then
            echo "  ALERT: $ip has $count failed attempts"
        fi
    done

# Find 500 errors in web logs
echo ""
echo "=== HTTP 500 Errors (Last Hour) ==="
ONE_HOUR_AGO=$(date -d '1 hour ago' +'%d/%b/%Y:%H')
grep " 500 " /var/log/nginx/access.log | \
    grep "$ONE_HOUR_AGO" | \
    awk '{print $7}' | sort | uniq -c | sort -rn

# Analyze Windows Security Log for failed logins

$Hours = 24
$Threshold = 5
$StartTime = (Get-Date).AddHours(-$Hours)

Write-Host "=== Failed Login Attempts (Last $Hours Hours) ==="

# Event ID 4625 = Failed login
$FailedLogins = Get-WinEvent -FilterHashtable @{
    LogName = 'Security'
    Id = 4625
    StartTime = $StartTime
} -ErrorAction SilentlyContinue

$FailedLogins | Group-Object {
    $_.Properties[19].Value  # Source IP
} | Where-Object Count -ge $Threshold | ForEach-Object {
    Write-Host "  ALERT: $($_.Name) has $($_.Count) failed attempts"
}

# Check for locked accounts
Write-Host ""
Write-Host "=== Locked Accounts ==="
Search-ADAccount -LockedOut | Select-Object Name, LockedOut

#!/usr/bin/env python3
"""Analyze logs for security events."""

import re
from collections import Counter
from datetime import datetime, timedelta
from pathlib import Path

def analyze_auth_log(log_path: str, threshold: int = 5):
    """Find IPs with multiple failed SSH attempts."""
    pattern = re.compile(r'Failed password.*from (\d+\.\d+\.\d+\.\d+)')
    ip_counts = Counter()

    with open(log_path) as f:
        for line in f:
            match = pattern.search(line)
            if match:
                ip_counts[match.group(1)] += 1

    print("=== Failed SSH Attempts ===")
    for ip, count in ip_counts.most_common():
        if count >= threshold:
            print(f"  ALERT: {ip} has {count} failed attempts")

def analyze_web_log(log_path: str):
    """Find 500 errors in access logs."""
    pattern = re.compile(r'"[A-Z]+ (\S+).*" 500')
    error_paths = Counter()

    with open(log_path) as f:
        for line in f:
            match = pattern.search(line)
            if match:
                error_paths[match.group(1)] += 1

    print("\n=== HTTP 500 Errors ===")
    for path, count in error_paths.most_common(10):
        print(f"{count:4d} errors: {path}")

if __name__ == "__main__":
    analyze_auth_log("/var/log/auth.log")
    analyze_web_log("/var/log/nginx/access.log")

Tasks

Create a script that counts failed login attempts by IP
Add email alerting when threshold is exceeded
Parse web server logs to find top 10 requested URLs
Identify unique visitors by IP in the last hour
Generate a daily summary report of key metrics

Regular Expression Tips

Use regex101.com to test patterns. Common patterns: IP address: \d+\.\d+\.\d+\.\d+, HTTP status: " (\d{3}) "

Related Visualizers

Log Analysis

3

Scheduled Tasks & Cron Jobs

Intermediate

Run scripts automatically at specified times using schedulers.

Cron Syntax (Linux)

                    # Cron format: minute hour day month weekday command
# ┌───────────── minute (0 - 59)
# │ ┌───────────── hour (0 - 23)
# │ │ ┌───────────── day of month (1 - 31)
# │ │ │ ┌───────────── month (1 - 12)
# │ │ │ │ ┌───────────── day of week (0 - 6) (Sunday=0)
# │ │ │ │ │
# * * * * * command

# Examples:
0 2 * * * /scripts/backup.sh        # Daily at 2:00 AM
*/15 * * * * /scripts/check.sh       # Every 15 minutes
0 9 * * 1-5 /scripts/report.sh     # Weekdays at 9 AM
0 0 1 * * /scripts/monthly.sh      # First day of month

# Edit crontab:
crontab -e     # Edit current user's crontab
crontab -l     # List current crontab
                

Task Scheduler (Windows PowerShell)

                    # Create a scheduled task
$Action = New-ScheduledTaskAction `
    -Execute "PowerShell.exe" `
    -Argument "-File C:\Scripts\backup.ps1"

$Trigger = New-ScheduledTaskTrigger `
    -Daily -At "2:00AM"

$Settings = New-ScheduledTaskSettingsSet `
    -ExecutionTimeLimit (New-TimeSpan -Hours 1) `
    -RestartCount 3

Register-ScheduledTask `
    -TaskName "DailyBackup" `
    -Action $Action `
    -Trigger $Trigger `
    -Settings $Settings `
    -Description "Daily backup at 2 AM"

# View scheduled tasks
Get-ScheduledTask | Where-Object State -eq 'Ready'
                

Tasks

Schedule your health check script to run daily
Create a weekly cleanup job for temporary files
Set up an hourly monitoring script with output logging
Configure email notifications for task failures
Document all scheduled tasks in a tracking sheet

Reflection

What time should maintenance tasks run? Consider user activity, system load, and dependencies.

4

Backup & Recovery Automation

Intermediate

Implement automated backup solutions with verification and rotation.

Backup Script Example

Bash PowerShell

#!/bin/bash
# Incremental backup with rotation

SOURCE="/home"
DEST="/backup"
DATE=$(date +%Y-%m-%d)
RETENTION=7  # Keep 7 days

# Create backup with rsync
rsync -avz --delete \
    --link-dest="${DEST}/latest" \
    "${SOURCE}" \
    "${DEST}/${DATE}"

# Update 'latest' symlink
rm -f "${DEST}/latest"
ln -s "${DEST}/${DATE}" "${DEST}/latest"

# Remove old backups
find "${DEST}" -maxdepth 1 -type d -mtime +${RETENTION} -exec rm -rf {} \;

# Verify backup
if [ -d "${DEST}/${DATE}" ]; then
    echo " Backup completed: ${DEST}/${DATE}"
    du -sh "${DEST}/${DATE}"
else
    echo " Backup FAILED!"
    exit 1
fi

# Windows backup with robocopy

$Source = "C:\Users"
$Dest = "D:\Backup"
$Date = Get-Date -Format "yyyy-MM-dd"
$Retention = 7

$BackupPath = "$Dest\$Date"

# Create backup with robocopy
robocopy $Source $BackupPath /MIR /XA:SH /R:3 /W:5 /LOG:"$Dest\backup_$Date.log"

# Check robocopy exit code
if ($LASTEXITCODE -le 3) {
    Write-Host " Backup completed: $BackupPath" -ForegroundColor Green
    (Get-ChildItem $BackupPath -Recurse | Measure-Object -Property Length -Sum).Sum / 1GB
} else {
    Write-Host " Backup FAILED! Exit code: $LASTEXITCODE" -ForegroundColor Red
    exit 1
}

# Remove old backups
Get-ChildItem $Dest -Directory |
    Where-Object { $_.CreationTime -lt (Get-Date).AddDays(-$Retention) } |
    Remove-Item -Recurse -Force

Tasks

Create a backup script with configurable source/destination
Implement backup rotation (keep last N backups)
Add backup verification (check file count, size)
Create a restore script that can recover from any backup
Add notification on backup success/failure

Related Visualizers

Backup Strategies

5

User & Permission Management

Advanced

Automate user account lifecycle: creation, modification, and deprovisioning.

Bulk User Management

Bash PowerShell

#!/bin/bash
# Create users from CSV: username,fullname,department

CSV_FILE="new_users.csv"
DEFAULT_PASS="ChangeMeNow123!"

while IFS=',' read -r username fullname department; do
    # Skip header
    [ "$username" = "username" ] && continue

    # Create user
    if ! id "$username" &>/dev/null; then
        useradd -m -c "$fullname" -G "$department" "$username"
        echo "$username:$DEFAULT_PASS" | chpasswd
        chage -d 0 "$username"  # Force password change
        echo " Created: $username"
    else
        echo "  Exists: $username"
    fi
done < "$CSV_FILE"

# Disable inactive users (no login in 90 days)
lastlog -b 90 | awk 'NR>1 {print $1}' | while read user; do
    usermod -L "$user"
    echo " Locked: $user (inactive)"
done

# Create AD users from CSV

$CsvFile = "new_users.csv"
$DefaultPass = ConvertTo-SecureString "ChangeMeNow123!" -AsPlainText -Force
$Domain = "contoso.com"

Import-Csv $CsvFile | ForEach-Object {
    $Username = $_.username
    $Name = $_.fullname
    $Dept = $_.department

    try {
        New-ADUser `
            -Name $Name `
            -SamAccountName $Username `
            -UserPrincipalName "$Username@$Domain" `
            -Department $Dept `
            -AccountPassword $DefaultPass `
            -ChangePasswordAtLogon $true `
            -Enabled $true

        Write-Host " Created: $Username" -ForegroundColor Green
    } catch {
        Write-Host " Failed: $Username - $_" -ForegroundColor Red
    }
}

# Find and disable inactive users
$InactiveDate = (Get-Date).AddDays(-90)
Get-ADUser -Filter {LastLogonDate -lt $InactiveDate} | ForEach-Object {
    Disable-ADAccount $_
    Write-Host " Disabled: $($_.SamAccountName)"
}

Tasks

Create a script to provision new users from a CSV file
Implement password policy enforcement
Build a user deprovisioning script (disable, archive, remove)
Create a permission audit report
Automate group membership based on department

6

Integration & Automated Reporting

Advanced

Combine multiple automation scripts into comprehensive solutions with reporting.

Capstone Project

Build a "Daily Operations Dashboard" that collects data from multiple sources and generates an HTML report sent via email.

Report Generation

                    #!/usr/bin/env python3
"""Generate HTML system report."""

import os
import psutil
import smtplib
from email.mime.text import MIMEText
from email.mime.multipart import MIMEMultipart
from datetime import datetime

def generate_html_report():
    cpu = psutil.cpu_percent(interval=1)
    memory = psutil.virtual_memory()
    disk = psutil.disk_usage('/')

    html = f"""
    <html>
    <head><style>
        body {{ font-family: Arial; padding: 20px; }}
        .metric {{ margin: 10px 0; padding: 15px; background: #f5f5f5; }}
        .warning {{ background: #fff3cd; }}
        .critical {{ background: #f8d7da; }}
    </style></head>
    <body>
        <h1>System Report - {datetime.now().strftime('%Y-%m-%d %H:%M')}</h1>
        <div class="metric {'warning' if cpu > 70 else ''}">
            <h3>CPU: {cpu}%</h3>
        </div>
        <div class="metric {'warning' if memory.percent > 80 else ''}">
            <h3>Memory: {memory.percent}%</h3>
        </div>
        <div class="metric {'critical' if disk.percent > 90 else ''}">
            <h3>Disk: {disk.percent}%</h3>
        </div>
    </body>
    </html>
    """
    return html

def send_report(html_content, to_email):
    msg = MIMEMultipart('alternative')
    msg['Subject'] = f"Daily System Report - {datetime.now().strftime('%Y-%m-%d')}"
    msg['From'] = "reports@example.com"
    msg['To'] = to_email
    msg.attach(MIMEText(html_content, 'html'))

    with smtplib.SMTP('smtp.example.com') as server:
        server.send_message(msg)

if __name__ == "__main__":
    report = generate_html_report()
    # send_report(report, "admin@example.com")
    print(report)
                

Tasks

Create an HTML report template with CSS styling
Integrate data from health checks, backups, and security logs
Add conditional formatting (green/yellow/red) for metrics
Implement email delivery of the report
Schedule the report to run daily and archive historical reports

Related Visualizers

Reporting Automation

Final Reflection

What repetitive tasks in your current role could you automate? Estimate the time savings per week.

Learning Objectives

Automation Fundamentals

When to Automate?

Script Structure Best Practices

Tasks

Related Visualizers

Log Analysis & Monitoring

Scenario

Log Parsing Examples

Tasks

Related Visualizers

Scheduled Tasks & Cron Jobs

Cron Syntax (Linux)

Task Scheduler (Windows PowerShell)

Tasks

Reflection

Backup & Recovery Automation

Backup Script Example

Tasks

Related Visualizers

User & Permission Management

Bulk User Management

Tasks

Integration & Automated Reporting

Capstone Project

Report Generation

Tasks

Related Visualizers

Final Reflection