sudo is not needed for any command in this workbook. Python 3 is pre-installed but uses an externally-managed environment, so all pip install commands require the --break-system-packages flag. All terminal blocks in this workbook already reflect this.These tools ship with Kali and are ready to use immediately. Confirm each is present by running the version check:
# Steganography tools # steghide --version steghide version 0.5.1 # binwalk --help | head -1 Binwalk v2.3.4 # foremost -V foremost version 1.5.7 # exiftool -ver 12.76 # wireshark --version | head -1 Wireshark 4.2.x # strings --version | head -1 GNU strings (GNU Binutils for Kali) 2.42 # Network tools # python3 -c "from scapy.all import IP; print('scapy ok')" scapy ok # Python version check # python3 --version Python 3.11.x
Java is not pre-installed on Kali. StegExpose and Stegsolve both require it.
# apt update && apt install -y default-jdk Reading package lists... Done Setting up default-jdk ... # java -version openjdk version "17.0.x" 2024-xx-xx # wget https://github.com/b3dk7/StegExpose/releases/download/StegExpose/StegExpose.jar # java -jar StegExpose.jar StegExpose - steganalysis tool for detecting steganography in lossless images
# Install all required Python packages for the full module # pip install --break-system-packages pillow numpy scipy matplotlib jpegio opencv-python peepdf Successfully installed pillow-10.x numpy-1.26.x scipy-1.12.x ... # Verify key imports work # python3 -c "import PIL, numpy, scipy, matplotlib, cv2; print('all imports OK')" all imports OK # jpegio requires a separate check (needs libjpeg) # python3 -c "import jpegio; print('jpegio OK')" jpegio OK
apt install -y libjpeg-dev then reinstall: pip install jpegio --break-system-packages --force-reinstall# gem install zsteg Successfully installed zsteg-0.2.13 # zsteg --version zsteg 0.2.13
# pdfid and pdf-parser are included in Kali's pdfid package # apt install -y pdfid # Verify # pdfid.py --version 2>&1 | head -1 PDFiD 0.2.8 # pdf-parser.py --version 2>&1 | head -1 pdf-parser.py by Didier Stevens # peepdf (advanced interactive PDF shell) # pip install peepdf --break-system-packages
Lesson 11 requires two machines on an isolated network. The recommended setup for Kali is:
| Setup option | How to configure | Notes |
|---|---|---|
| Two Kali VMs — VirtualBox | Set both VMs to "Host-Only Adapter" (vboxnet0) | Simplest. No internet traffic leaves the host. |
| Two Kali VMs — VMware | Set both to VMnet1 (Host-only) | Same isolation as VirtualBox approach. |
| Single machine with network namespaces | ip netns add ns1 && ip netns add ns2 + veth pair |
Advanced — no second VM needed. Uses Linux namespaces. |
| Kali + Docker | Two containers on a custom bridge network | Quick but raw socket support requires --privileged flag. |
# Create two isolated network namespaces connected by a virtual cable # ip netns add sender # ip netns add receiver # ip link add veth0 type veth peer name veth1 # ip link set veth0 netns sender # ip link set veth1 netns receiver # ip netns exec sender ip addr add 10.0.0.1/24 dev veth0 # ip netns exec receiver ip addr add 10.0.0.2/24 dev veth1 # ip netns exec sender ip link set veth0 up # ip netns exec receiver ip link set veth1 up # Run sender script in sender namespace # ip netns exec sender python3 covert_sender.py 10.0.0.2 "COVERT MESSAGE" # Run receiver script in receiver namespace (separate terminal) # ip netns exec receiver python3 covert_receiver.py # Clean up after lab # ip netns del sender && ip netns del receiver
# python3 - << 'PYEOF'
tools = {
"java": "java -version",
"steghide": "steghide --version",
"binwalk": "binwalk --help",
"foremost": "foremost -V",
"exiftool": "exiftool -ver",
"zsteg": "zsteg --version",
"pdfid": "pdfid.py --version",
"strings": "strings --version",
"wireshark": "wireshark --version",
}
import subprocess, importlib
for name, cmd in tools.items():
r = subprocess.run(cmd.split(), capture_output=True)
status = "✓" if r.returncode == 0 else "✗ MISSING"
print(f" {status} {name}")
for lib in ["PIL","numpy","scipy","matplotlib","jpegio","cv2","scapy"]:
try:
importlib.import_module(lib)
print(f" ✓ python:{lib}")
except ImportError:
print(f" ✗ python:{lib} ← pip install needed")
PYEOF
# as the prompt (Kali root) rather than $. No sudo prefix is needed anywhere.In real-world digital forensics, you rarely know which files in a suspect's folder contain hidden data. StegExpose is a Java-based batch steganalysis tool that applies multiple detection algorithms simultaneously — SamplePairs analysis, RS analysis, Primary Sets, and LSB analysis — to rank images by their probability of containing hidden content.
It was developed as an academic research tool and remains one of the most effective open-source detectors for LSB-based image steganography in PNG and BMP formats.
How StegExpose works
StegExpose fuses four statistical tests into a combined score:
- SamplePairs analysis: measures correlation asymmetry between adjacent pixel pairs — LSB embedding disrupts natural image statistics.
- RS (Regular-Singular) analysis: counts "regular" and "singular" pixel groups; embedding flips these counts in a detectable way.
- Primary Sets: analyses the distribution of the last bit plane.
- LSB analysis: direct inspection of bit-plane randomness using chi-square statistics.
A clean image scores near 0.0. A stego image scores above the detection threshold (default: 0.2).
apt install default-jdk then verify with java -version.# wget https://github.com/b3dk7/StegExpose/releases/download/StegExpose/StegExpose.jar --2024-03-01 09:14:02-- https://github.com/b3dk7/StegExpose/... Saving to: 'StegExpose.jar' StegExpose.jar 100% [==============>] 847K # java -version openjdk version "11.0.22" 2024-01-16 # java -jar StegExpose.jar StegExpose - steganalysis tool for detecting steganography in lossless images Usage: java -jar StegExpose.jar [directory] [threshold] [default|quiet|csv] [output.csv]
The basic usage scans every PNG and BMP in a directory and reports a suspicion score for each. Images above the threshold are flagged.
# java -jar StegExpose.jar ./test_images/ 0.2 default Analysing images in ./test_images/ ... Image | Combined Score | Flagged -----------------------+----------------+-------- beach_holiday.png | 0.043 | No corporate_logo.png | 0.067 | No invoice_scan.png | 0.312 | YES ◄ profile_photo.png | 0.091 | No team_photo.png | 0.447 | YES ◄ company_banner.png | 0.029 | No Flagged: 2 of 6 images Analysis complete in 1.24s
For forensic reporting, use CSV mode to export results for further processing in Python or a spreadsheet:
# java -jar StegExpose.jar ./test_images/ 0.2 csv results.csv # cat results.csv filename,score,flagged beach_holiday.png,0.043,false corporate_logo.png,0.067,false invoice_scan.png,0.312,true profile_photo.png,0.091,false team_photo.png,0.447,true company_banner.png,0.029,false
StegExpose is not infallible. Certain types of clean image can produce false positives:
- Images with high-frequency noise (film grain, sensor noise)
- Heavily compressed images re-saved multiple times
- Procedurally generated images (e.g. pixel art, screenshots of code)
- Images that have been put through image editors that modify LSBs
A flagged score is evidence to investigate further, not proof of guilt. Use it as a triage tool before deeper manual analysis.
Blind Detection Challenge
Work in pairs. Person A uses Steghide or OpenStego to embed secret messages in 5 of 20 downloaded images, then passes the full folder of 20 to Person B without revealing which are stego images.
- Download 20 varied PNG images (landscapes, portraits, logos, screenshots — mix matters)
- Person A: embed messages of varying lengths (50 chars, 500 chars, 5000 chars) in 5 different images using Steghide with password "steg123"
- Person B: run StegExpose at threshold 0.2 and record your results — do not look at the images beforehand
- Compare your flagged list against the actual stego images — calculate precision and recall
- Re-run at threshold 0.1 and 0.3 — how does precision/recall change?
- Discussion question: why does message length affect detectability? At what payload size does StegExpose reliably flag the image?
When an image is unmodified, its pixel values follow natural distributions. In any natural image, certain pairs of pixel values — called PoVs (Pairs of Values) — have a natural statistical relationship: the value 2k and its partner 2k+1 appear in roughly equal proportions.
For example: pixels with value 200 and pixels with value 201 form a PoV. In a clean image, these counts are not necessarily equal — 200 might appear 1,200 times and 201 might appear 900 times. But sequential LSB embedding equalises these counts, because it replaces the LSB of every pixel with message bits, which are approximately 50% 0s and 50% 1s.
The chi-square test measures how much the actual distribution deviates from the expected "equalised" distribution. Low chi-square = equalised = stego image likely.
where: expected_k = (count(2k) + count(2k+1)) / 2
and the sum is over all value pairs k = 0..127
import numpy as np from PIL import Image from scipy import stats import matplotlib.pyplot as plt import sys def get_pixel_values(image_path: str, channel: int = 0) -> np.ndarray: """Extract pixel values for a single channel (0=R, 1=G, 2=B).""" img = Image.open(image_path).convert('RGB') arr = np.array(img) return arr[:, :, channel].flatten() def chi_square_lsb(values: np.ndarray) -> tuple[float, float]: """ Perform the chi-square attack on a 1-D array of pixel values. Returns (chi2_statistic, p_value). High p-value (close to 1.0) suggests steganography. """ # Count occurrences of each value 0..255 counts = np.bincount(values, minlength=256) # Build PoV (Pair of Values) lists # observed: actual counts of each pair member # expected: mean of the pair (what LSB embedding produces) observed = [] expected = [] for k in range(128): n0 = counts[2 * k] # count of value 2k n1 = counts[2 * k + 1] # count of value 2k+1 total = n0 + n1 if total == 0: continue mean = total / 2.0 observed.extend([n0, n1]) expected.extend([mean, mean]) observed = np.array(observed, dtype=float) expected = np.array(expected, dtype=float) # Remove pairs where expected == 0 (avoids division by zero) mask = expected > 0 chi2 = np.sum((observed[mask] - expected[mask]) ** 2 / expected[mask]) dof = mask.sum() - 1 p_value = 1 - stats.chi2.cdf(chi2, dof) return chi2, p_value def sliding_window_chi2(values: np.ndarray, window: int = 512) -> np.ndarray: """ Run chi-square over sliding windows of pixel values. Returns array of p-values — shows where in the image embedding starts. Crucial for visualising partial embedding (not all images are fully embedded). """ p_values = [] for i in range(0, len(values) - window, window // 2): segment = values[i:i + window] _, p = chi_square_lsb(segment) p_values.append(p) return np.array(p_values) def analyse_and_plot(clean_path: str, stego_path: str): """Full analysis with side-by-side sliding window plots.""" fig, axes = plt.subplots(2, 2, figsize=(14, 8)) fig.patch.set_facecolor('#0d1117') for ax in axes.flat: ax.set_facecolor('#161b22') ax.tick_params(colors='#8b949e') for spine in ax.spines.values(): spine.set_color('#30363d') for idx, (path, label) in enumerate([(clean_path, 'CLEAN'), (stego_path, 'STEGO')]): values = get_pixel_values(path, channel=0) # Red channel chi2_global, p_global = chi_square_lsb(values) p_values_sliding = sliding_window_chi2(values) # Top row: value pair distribution counts = np.bincount(values, minlength=256) pairs = [(counts[2*k], counts[2*k+1]) for k in range(64)] evens = [p[0] for p in pairs] odds = [p[1] for p in pairs] x = range(64) axes[0][idx].bar(x, evens, width=0.4, label='2k', color='#00d8a0', alpha=0.7) axes[0][idx].bar([i+0.4 for i in x], odds, width=0.4, label='2k+1', color='#e05252', alpha=0.7) axes[0][idx].set_title(f'{label} — Pair Counts (lower 64 pairs)\n' f'χ²={chi2_global:.1f} p={p_global:.4f}', color='#e6edf3', fontsize=10) axes[0][idx].legend(facecolor='#21262d', labelcolor='#8b949e', fontsize=8) # Bottom row: sliding window p-value chart color = '#e05252' if idx == 1 else '#00d8a0' axes[1][idx].plot(p_values_sliding, color=color, linewidth=1.5) axes[1][idx].axhline(0.95, color='#d29922', linestyle='--', alpha=0.6, label='p=0.95 threshold') axes[1][idx].set_ylim(0, 1) axes[1][idx].set_title(f'{label} — Sliding Window p-values', color='#e6edf3', fontsize=10) axes[1][idx].set_xlabel('Window index', color='#8b949e') axes[1][idx].set_ylabel('p-value', color='#8b949e') axes[1][idx].legend(facecolor='#21262d', labelcolor='#8b949e', fontsize=8) plt.tight_layout(pad=2.0) plt.savefig('chi_square_analysis.png', dpi=150, bbox_inches='tight', facecolor='#0d1117') plt.show() print("\nPlot saved: chi_square_analysis.png") if __name__ == '__main__': if len(sys.argv) != 3: print("Usage: python chi_square_attack.py clean.png stego.png") sys.exit(1) analyse_and_plot(sys.argv[1], sys.argv[2])
# pip install pillow numpy scipy matplotlib --break-system-packages # python chi_square_attack.py beach_clean.png beach_stego.png CLEAN image: χ² = 847.3 p-value = 0.0031 → likely clean STEGO image: χ² = 142.1 p-value = 0.9987 → STEGO DETECTED Plot saved: chi_square_analysis.png
The chi-square attack is powerful against sequential LSB embedding, but it is defeated by:
- Random-key LSB embedding: if pixels are selected pseudorandomly using a password, the equalization is spread non-sequentially and harder to detect in sliding windows
- JPEG images: lossy compression destroys the LSB statistics entirely
- Very short payloads: if only 2% of pixels are modified, the statistical disturbance is too small to detect reliably
- Adaptive steganography: hiding only in high-complexity regions avoids creating detectable uniformity in flat regions
Implement and Stress-test the Chi-square Detector
Run the chi-square analysis on a set of images, then try to evade your own detector.
- Install dependencies and run chi_square_attack.py on one clean and one stego image pair. Confirm you see the expected p-value difference.
- Embed messages of different lengths (100, 1000, 10000 chars) in the same image. Plot p-value vs payload size — at what size does it become reliably detectable?
- Modify the embedding code to use random pixel selection (seed a Python random generator with a password, then pick pixel indices). Rerun chi-square — does the detector still work?
- Bonus: adapt the sliding window function to return the estimated payload start position and estimated message length from the p-value transition point.
JPEG compression does not store pixel values directly. Instead, the image is divided into 8×8 pixel blocks, each transformed into frequency components using the Discrete Cosine Transform (DCT). The result is a matrix of 64 DCT coefficients per block — low-frequency components hold the broad shapes and colours; high-frequency components hold fine detail. JPEG quantises these coefficients (throws away precision) to achieve compression.
JPEG steganography tools like JSteg and F5 hide data by modifying specific DCT coefficients — typically the AC coefficients (all except the top-left DC coefficient). This leaves statistical fingerprints.
JSteg embedding and its statistical signature
JSteg replaces the LSBs of non-zero, non-one DCT coefficients with message bits. This has a predictable effect: in natural images, the number of DCT coefficients equal to +1 and -1 follows a natural ratio. JSteg embedding systematically reduces this asymmetry — a detectable signature.
The PoV (Pair of Values) histogram of DCT coefficients, particularly the ratio f(+1)/f(-1) and f(+2)/f(-2), is the primary detection signal.
jpegio Python library, which reads raw DCT coefficients from a JPEG file without decompressing to pixels first. Install with: pip install jpegio --break-system-packagesimport jpegio as jio import numpy as np import matplotlib.pyplot as plt from collections import Counter import sys def extract_dct_coefficients(jpeg_path: str) -> np.ndarray: """ Read raw quantised DCT coefficients directly from JPEG file. Returns a flattened array of all AC coefficients (excludes DC at position [0,0]). """ struct = jio.read(jpeg_path) coeffs = [] for channel_coeffs in struct.coef_arrays: h, w = channel_coeffs.shape # Process each 8x8 block for row in range(0, h, 8): for col in range(0, w, 8): block = channel_coeffs[row:row+8, col:col+8] # Flatten block, skip DC coefficient at [0,0] ac = block.flatten()[1:] coeffs.extend(ac.tolist()) return np.array(coeffs) def pov_histogram(coeffs: np.ndarray, rng: int = 10) -> dict: """ Build Pair of Values (PoV) histogram of DCT coefficients. For JSteg detection: compare frequency of +k vs -k. Natural images have |f(+k)| ≈ |f(-k)| but not exactly equal. JSteg embedding equalises them — specifically at k=1 and k=2. """ counts = Counter(coeffs.tolist()) pov = {} for k in range(1, rng + 1): pov[k] = { 'positive': counts.get(k, 0), 'negative': counts.get(-k, 0), 'ratio': counts.get(k, 1) / max(counts.get(-k, 1), 1) } return pov def jsteg_detection_score(coeffs: np.ndarray) -> float: """ Simple JSteg detection heuristic. Returns score 0..1 where >0.7 suggests JSteg embedding. Based on the deviation of f(1)/f(-1) from expected natural ratio. Natural images: ratio typically 0.95-1.05 JSteg images: ratio approaches 1.00 (equalised) """ counts = Counter(coeffs.tolist()) f_pos1 = counts.get(1, 0) f_neg1 = counts.get(-1, 0) f_pos2 = counts.get(2, 0) f_neg2 = counts.get(-2, 0) if f_neg1 == 0 or f_neg2 == 0: return 0.0 ratio1 = f_pos1 / f_neg1 ratio2 = f_pos2 / f_neg2 # Natural images: these ratios have specific biases # JSteg drives them toward 1.0 independently # Score based on how close both ratios are to 1.0 deviation1 = abs(ratio1 - 1.0) deviation2 = abs(ratio2 - 1.0) # Lower deviation = more suspicious score = 1.0 - min((deviation1 + deviation2) / 2.0, 1.0) return round(score, 4) def analyse_jpeg(path: str): """Full DCT analysis with visualisation.""" print(f"\n{'='*50}") print(f"Analysing: {path}") print(f"{'='*50}") coeffs = extract_dct_coefficients(path) pov = pov_histogram(coeffs, rng=12) score = jsteg_detection_score(coeffs) print(f"Total AC coefficients analysed: {len(coeffs):,}") print(f"\nPoV Histogram (|k| vs |-k| counts):") print(f"{'k':>4} {'f(+k)':>8} {'f(-k)':>8} {'ratio':>8}") print("-" * 36) for k, v in pov.items(): flag = " ◄ suspicious" if abs(v['ratio']-1.0) < 0.02 and k <= 3 else "" print(f"{k:>4} {v['positive']:>8,} {v['negative']:>8,} {v['ratio']:>8.4f}{flag}") print(f"\nJSteg detection score: {score:.4f}") if score > 0.85: print("→ HIGH confidence: JSteg steganography likely present") elif score > 0.65: print("→ MODERATE confidence: anomalies detected, further analysis needed") else: print("→ LOW confidence: image likely clean") # Plot DCT coefficient distribution fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 5)) fig.patch.set_facecolor('#0d1117') for ax in [ax1, ax2]: ax.set_facecolor('#161b22') ax.tick_params(colors='#8b949e') for s in ax.spines.values(): s.set_color('#30363d') # Left: overall DCT histogram (trimmed to ±30) trimmed = coeffs[(abs(coeffs) <= 30)] ax1.hist(trimmed, bins=range(-30, 31), color='#00d8a0', alpha=0.8, edgecolor='none') ax1.set_title('DCT Coefficient Distribution (±30)', color='#e6edf3') ax1.set_xlabel('DCT coefficient value', color='#8b949e') # Right: PoV bar chart for k=1..8 ks = list(pov.keys())[:8] pos_counts = [pov[k]['positive'] for k in ks] neg_counts = [pov[k]['negative'] for k in ks] x = np.arange(len(ks)) ax2.bar(x - 0.2, pos_counts, 0.4, label='f(+k)', color='#00d8a0', alpha=0.8) ax2.bar(x + 0.2, neg_counts, 0.4, label='f(-k)', color='#e05252', alpha=0.8) ax2.set_xticks(x); ax2.set_xticklabels([f'k={k}' for k in ks]) ax2.set_title('PoV Histogram: f(+k) vs f(-k)', color='#e6edf3') ax2.legend(facecolor='#21262d', labelcolor='#8b949e') plt.tight_layout() outname = path.replace('.jpg', '_dct.png').replace('.jpeg', '_dct.png') plt.savefig(outname, dpi=150, bbox_inches='tight', facecolor='#0d1117') print(f"Plot saved: {outname}") if __name__ == '__main__': for arg in sys.argv[1:]: analyse_jpeg(arg)
JPEG DCT Detection Lab
-
Install jpegio:
pip install jpegio --break-system-packages. Run dct_analysis.py on a clean JPEG photo. Record the f(+1)/f(-1) ratio. - Download Jphide or use an online JSteg tool to embed a message in the same JPEG. Rerun dct_analysis.py and compare the ratio — does it move closer to 1.0?
- Try steganalysis on a JPEG that has simply been re-saved at different quality levels (no steganography). Does the score falsely increase? What does this tell you about false positives in JPEG analysis?
- Research: how does the F5 algorithm improve on JSteg to reduce this statistical signature? Write a 200-word summary.
IP and TCP headers contain several fields that are either ignored by most routers, semantically unused, or only partially validated. These become covert channels — mechanisms for hiding data that pass through network monitoring that only inspects payload content.
| Header field | Normal use | Covert channel capacity | Detectability |
|---|---|---|---|
| IP ID field | Fragment reassembly identifier | 16 bits per packet | Medium — sequential IDs expected |
| IP TTL field | Hop-count decay | 8 bits per packet (with constraints) | Medium — unusual TTL values stand out |
| TCP sequence number | Byte-stream ordering | 32 bits (randomised by OS — hard to abuse) | High — ISN prediction attacks exist |
| IP Reserved bit | Must be zero (RFC 791) | 1 bit per packet | Very high — any set bit is anomalous |
| TCP URG/PSH flags | Priority/push signalling | Timing-based encoding | Low — flags used legitimately |
""" Covert channel: encodes a message in the IP ID field. Run as root. Use only in isolated lab environments. Protocol: - Each packet carries 1 byte of the message in the IP ID field (low 8 bits) - High byte of IP ID is used as a sequence number (0-255) - Termination: IP ID = 0xFFFF signals end of message """ from scapy.all import IP, ICMP, send, conf import time import sys conf.verb = 0 # suppress Scapy output def encode_message(message: str, dst: str, delay: float = 0.1): """ Encode a string message in IP ID fields of ICMP packets. Each byte of the message becomes the low byte of the IP ID. """ encoded = message.encode('utf-8') print(f"Sending {len(encoded)} bytes to {dst}...") for seq, byte_val in enumerate(encoded): # Encode: high byte = sequence number, low byte = message byte ip_id = ((seq % 256) << 8) | byte_val pkt = IP(dst=dst, id=ip_id) / ICMP() send(pkt) time.sleep(delay) # Termination packet send(IP(dst=dst, id=0xFFFF) / ICMP()) print("Transmission complete. Termination packet sent.") if __name__ == '__main__': if len(sys.argv) < 3: print("Usage: python3 covert_sender.py <dst_ip> <message>") sys.exit(1) encode_message(sys.argv[2], sys.argv[1])
""" Receiver: reconstructs the message from IP ID fields. Run as root on the destination machine. """ from scapy.all import sniff, IP, ICMP import sys message_bytes = {} def process_packet(pkt): if IP in pkt and ICMP in pkt: ip_id = pkt[IP].id if ip_id == 0xFFFF: # Termination — reconstruct message if message_bytes: sorted_bytes = [message_bytes[k] for k in sorted(message_bytes)] msg = bytes(sorted_bytes).decode('utf-8', errors='replace') print(f"\n[+] Covert message received ({len(message_bytes)} bytes):") print(f" {msg}") message_bytes.clear() return seq = (ip_id >> 8) & 0xFF # high byte = sequence byte_val = ip_id & 0xFF # low byte = message byte message_bytes[seq] = byte_val print(f"[>] Packet {seq}: byte={byte_val} ('{chr(byte_val) if 32 <= byte_val < 127 else '?'}')", end='\r') print("[*] Listening for covert channel packets (Ctrl+C to stop)...") sniff(filter="icmp", prn=process_packet, store=0)
Normal ICMP traffic has predictable, incrementing IP ID values. A covert channel using the IP ID field will show apparently random, non-sequential values. Here is a Wireshark display filter to flag anomalous IP ID patterns:
# Flag ICMP packets — examine IP ID field manually Filter: icmp # Look for packets where the IP ID high byte > 0 # (in normal pings, IP ID is typically sequential 1, 2, 3...) Filter: ip.id > 0x0100 # Column trick: add "ip.id" as a custom column to see all IP ID values # Right-click column header → Column Preferences → Add: ip.id # Python script to detect from a PCAP file Filter script: python detect_covert.py capture.pcap
""" Detect IP ID-based covert channels in a PCAP file. Analyses IP ID entropy and sequential patterns. """ from scapy.all import rdpcap, IP import numpy as np from collections import defaultdict import sys def analyse_ip_ids(pcap_path: str): pkts = rdpcap(pcap_path) ip_ids_by_src = defaultdict(list) for pkt in pkts: if IP in pkt: ip_ids_by_src[pkt[IP].src].append(pkt[IP].id) print(f"{'Source IP':<20} {'Pkts':>5} {'Entropy':>8} {'Max jump':>10} {'Verdict'}") print("-"*65) for src, ids in ip_ids_by_src.items(): if len(ids) < 5: continue ids_arr = np.array(ids) # Entropy of IP ID values (higher = more random = suspicious) _, cnts = np.unique(ids_arr, return_counts=True) probs = cnts / cnts.sum() entropy = -np.sum(probs * np.log2(probs + 1e-10)) # Max jump between consecutive IDs diffs = np.abs(np.diff(ids_arr.astype(int))) max_jump = diffs.max() if len(diffs) > 0 else 0 verdict = 'SUSPICIOUS' if entropy > 6.0 and max_jump > 500 else 'normal' print(f"{src:<20} {len(ids):>5} {entropy:>8.3f} {max_jump:>10} {verdict}") if __name__ == '__main__': analyse_ip_ids(sys.argv[1])
Build and Detect a Covert Channel
-
Set up two VMs on a host-only network. Install Scapy on both:
pip install scapy --break-system-packages - Run covert_receiver.py on VM-B. On VM-A, run covert_sender.py to send the message "COVERT CHANNEL TEST 12345"
- Simultaneously capture the traffic in Wireshark on VM-B. Save the PCAP file.
- Run detect_covert_channel.py on the PCAP — confirm it flags the covert traffic as SUSPICIOUS
- Extension: modify the sender to encode data in the TTL field instead (valid range 1-255; use values 65-128 to avoid triggering TTL-too-low alerts). Update the receiver and detector accordingly.
PDFs are richly structured formats with many hiding surfaces. Understanding these is essential for both creating and detecting document-based exfiltration:
| Technique | How it works | Detection method |
|---|---|---|
| White-on-white text | Text objects with colour matching background — visually invisible but searchable and extractable | pdf-parser: look for /Font + white colour operators; select-all in Adobe reveals it |
| Zero-opacity layers | Content placed on an OCG (Optional Content Group) layer set to invisible | pdfid flags /OCG; layer management panel in PDF viewers |
| Metadata streams | XMP metadata or custom Info dictionary entries can hold arbitrary data | exiftool -a -u; pdf-parser for /Metadata objects |
| Embedded file streams | Files attached with /EmbeddedFile — may be compressed, encrypted, or disguised | pdfid flags /EmbeddedFile; pdf-parser extracts streams |
| Comment fields | PDF supports comment annotations with hidden author/content fields | pdf-parser for /Annots; check /Contents of annotation objects |
| Incremental updates | PDFs allow appending new versions; old content may be hidden beneath newer revisions | Look for multiple %%EOF markers; extract each revision separately |
# pdfid and pdf-parser by Didier Stevens — industry standard tools # pip install pdfid pdf-parser --break-system-packages # Or download directly from: # https://blog.didierstevens.com/programs/pdf-tools/ # exiftool — reads all metadata # apt install libimage-exiftool-perl # peepdf — advanced PDF analysis with interactive shell # pip install peepdf --break-system-packages
Always start with pdfid. It counts the occurrences of suspicious PDF keywords in the file, giving you a threat profile at a glance:
# pdfid.py suspicious_invoice.pdf PDFiD 0.2.8 suspicious_invoice.pdf PDF Header: %PDF-1.7 obj 47 endobj 47 stream 12 endstream 12 xref 1 trailer 1 startxref 2 /Page 8 /Encrypt 0 /ObjStm 3 ← object streams (can hide content) /JS 1 ← JavaScript present /JavaScript 1 /AA 0 /OpenAction 0 /EmbeddedFile 2 ← embedded files present /XFA 1 ← XML form data /Colors > 2^24 0 /URI 0
# List all objects and their types # pdf-parser.py suspicious_invoice.pdf | head -80 # Find embedded file streams # pdf-parser.py -s EmbeddedFile suspicious_invoice.pdf obj 23 0 Type: /EmbeddedFile Contains stream <</Type /EmbeddedFile /Length 4820 /Filter /FlateDecode>> # Extract and decompress an object's stream to a file # pdf-parser.py -o 23 -d extracted_file.bin suspicious_invoice.pdf # Determine what type of file was embedded # file extracted_file.bin extracted_file.bin: Zip archive data, at least v2.0 to extract # Search for suspicious colour operators (white text hiding) # pdf-parser.py -f -s /Font suspicious_invoice.pdf # Then manually inspect the page content streams for: # "1 1 1 rg" (white fill RGB), "1 1 1 RG" (white stroke RGB) # "0 g" (black fill), followed by text with colour override # All metadata fields # exiftool -a -u suspicious_invoice.pdf ... Author : Finance Dept Subject : Q3 Invoice Comment : PAYLOAD:SG Comment : PAYLOAD:SGVsbG8gV29ybGQ= ← base64 data hidden in Comment # echo "SGVsbG8gV29ybGQ=" | base64 -d Hello World
""" Automated PDF forensics: scans for steganographic indicators. Outputs a structured report of suspicious findings. """ import re import sys import zlib import base64 from pathlib import Path def read_pdf_raw(path: str) -> bytes: with open(path, 'rb') as f: return f.read() def count_eof_markers(data: bytes) -> int: """Count %%EOF markers — multiple EOFs = incremental updates.""" return data.count(b'%%EOF') def find_white_text_indicators(data: bytes) -> list: """Find PDF colour operators that set white fill (potential invisible text).""" findings = [] # Decompress FlateDecode streams first streams = re.findall(rb'stream\r?\n(.*?)\r?\nendstream', data, re.DOTALL) for i, stream in enumerate(streams): try: decoded = zlib.decompress(stream) except: decoded = stream # "1 1 1 rg" = white RGB fill in PDF content stream if b'1 1 1 rg' in decoded or b'1 1 1 RG' in decoded: findings.append(f"Stream {i}: white fill colour operator found") if b'0 0 0 0 k' in decoded: findings.append(f"Stream {i}: white CMYK fill found") return findings def find_suspicious_keywords(data: bytes) -> dict: """Count suspicious PDF keywords.""" keywords = [ b'/EmbeddedFile', b'/ObjStm', b'/JavaScript', b'/JS', b'/OCG', b'/XFA', b'/AA', b'/OpenAction' ] return {kw.decode(): data.count(kw) for kw in keywords} def find_base64_in_metadata(data: bytes) -> list: """Look for base64-encoded strings in Info dictionary fields.""" findings = [] # Rough regex for Info dict values that look like base64 matches = re.findall(rbr'\(([A-Za-z0-9+/]{20,}={0,2})\)', data) for m in matches: try: decoded = base64.b64decode(m) if all(32 <= b < 127 for b in decoded[:20]): findings.append(f"Possible base64 payload: {decoded[:60]}...") except: pass return findings def analyse_pdf(path: str): """Run all checks and print a forensic report.""" data = read_pdf_raw(path) print(f"\n{'='*55}") print(f"PDF FORENSIC REPORT: {Path(path).name}") print(f"{'='*55}") print(f"File size : {len(data):,} bytes") print(f"%%EOF markers: {count_eof_markers(data)}", "← MULTIPLE (incremental updates present)" if count_eof_markers(data) > 1 else "") kw = find_suspicious_keywords(data) print("\nSuspicious keyword counts:") for k, v in kw.items(): flag = " ◄ FLAGGED" if v > 0 else "" print(f" {k:<20} {v:>3}{flag}") white = find_white_text_indicators(data) if white: print(f"\nWhite text indicators found:") for w in white: print(f" ◄ {w}") else: print("\nNo white text indicators found.") b64 = find_base64_in_metadata(data) if b64: print(f"\nPossible base64 payloads in string fields:") for b in b64[:5]: print(f" ◄ {b}") if __name__ == '__main__': analyse_pdf(sys.argv[1])
PDF Forensics Lab
- Create a test PDF using LibreOffice Writer. Add a text box, colour the text white, and save as PDF. Run pdfid.py on it — do you see anything suspicious?
- Run pdf_forensics.py on your crafted PDF. Confirm it detects the white text operator.
-
Use exiftool to add a base64-encoded message to the Comment field:
exiftool -Comment="$(echo 'SECRET' | base64)" your.pdf. Run pdf_forensics.py again — does it surface the hidden data? - Download a sample malicious PDF from MalwareBazaar (PDF category, sandboxed) and run pdfid.py on it. What steganographic indicators are present alongside the malicious content?
A 1080p video at 30fps contains 1,920 × 1,080 × 3 = over 6 million bytes of pixel data per frame. At 1-bit LSB embedding, a single second of video can hide over 22 MB of data while remaining visually identical to the original. This enormous capacity, combined with the fact that video files pass through most DLP tools uninspected (they're too large to analyse in real time), makes video an attractive exfiltration channel.
Real-world APT groups have used this technique to embed C2 commands in thumbnail images on YouTube and Imgur, and to exfiltrate data in video uploads that appear to be ordinary screen recordings.
When data is embedded uniformly across all frames of a video, each frame will show elevated LSB randomness. But the steganographer may only embed data in specific frames — a smarter approach. The detection strategy is to measure the LSB entropy of each frame and look for statistical discontinuities.
""" Video steganography detector. Analyses LSB entropy and frame-difference metrics per frame. Plots anomalies to identify stego frames. Install: pip install opencv-python numpy matplotlib --break-system-packages --break-system-packages """ import cv2 import numpy as np import matplotlib.pyplot as plt from scipy.stats import entropy as scipy_entropy import sys def lsb_entropy(frame: np.ndarray) -> float: """ Compute Shannon entropy of the LSB plane of a frame. Clean frames: ~1.0 (near-random LSBs from natural image detail). Stego frames: approaches 1.0 very precisely (truly random embedded bits). The signature is *consistency* — stego frames cluster tightly at high entropy. """ lsbs = frame & 1 # extract least significant bits lsb_flat = lsbs.flatten() ones = np.sum(lsb_flat) zeros = len(lsb_flat) - ones total = len(lsb_flat) if total == 0: return 0.0 p1 = ones / total p0 = zeros / total if p0 == 0 or p1 == 0: return 0.0 return -p0 * np.log2(p0) - p1 * np.log2(p1) def frame_difference(f1: np.ndarray, f2: np.ndarray) -> float: """Mean absolute difference between consecutive frames (motion metric).""" return np.mean(np.abs(f1.astype(int) - f2.astype(int))) def analyse_video(video_path: str, sample_rate: int = 1): """ Analyse video for steganographic anomalies. sample_rate: analyse every Nth frame (1 = every frame, 5 = every 5th). """ cap = cv2.VideoCapture(video_path) if not cap.isOpened(): print(f"Error: cannot open {video_path}") return fps = cap.get(cv2.CAP_PROP_FPS) total_frames = int(cap.get(cv2.CAP_PROP_FRAME_COUNT)) print(f"Video: {video_path}") print(f"Frames: {total_frames} FPS: {fps:.1f} Duration: {total_frames/fps:.1f}s") print(f"Analysing every {sample_rate} frame(s)...") entropies = [] diffs = [] frame_indices = [] prev_frame = None frame_num = 0 while True: ret, frame = cap.read() if not ret: break if frame_num % sample_rate == 0: gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) ent = lsb_entropy(gray) entropies.append(ent) frame_indices.append(frame_num) if prev_frame is not None: diff = frame_difference(gray, prev_frame) diffs.append(diff) else: diffs.append(0) prev_frame = gray frame_num += 1 if frame_num % 100 == 0: print(f" Processed {frame_num}/{total_frames} frames...", end='\r') cap.release() print(f"\nComplete. Analysed {len(entropies)} frames.") # Detect anomalous frames: entropy > mean + 2*std ent_arr = np.array(entropies) threshold = ent_arr.mean() + 2 * ent_arr.std() flagged = [frame_indices[i] for i, e in enumerate(entropies) if e > threshold] print(f"\nEntropy stats: mean={ent_arr.mean():.4f} std={ent_arr.std():.4f}") print(f"Detection threshold: {threshold:.4f}") if flagged: print(f"Flagged frames: {flagged[:20]}{'...' if len(flagged)>20 else ''}") print(f"Total flagged: {len(flagged)} of {len(entropies)}") else: print("No anomalous frames detected.") # Plot fig, (ax1, ax2) = plt.subplots(2, 1, figsize=(14, 6), sharex=True) fig.patch.set_facecolor('#0d1117') for ax in [ax1, ax2]: ax.set_facecolor('#161b22') ax.tick_params(colors='#8b949e') for s in ax.spines.values(): s.set_color('#30363d') ax1.plot(frame_indices, entropies, color='#00d8a0', linewidth=0.8, label='LSB entropy') ax1.axhline(threshold, color='#e05252', linestyle='--', alpha=0.7, label=f'threshold ({threshold:.3f})') for f in flagged: ax1.axvline(f, color='#d29922', alpha=0.3, linewidth=1) ax1.set_ylabel('LSB entropy', color='#8b949e') ax1.set_title(f'Video Steganalysis: {video_path}', color='#e6edf3') ax1.legend(facecolor='#21262d', labelcolor='#8b949e', fontsize=9) ax2.plot(frame_indices, diffs, color='#58a6ff', linewidth=0.8, label='Frame difference') ax2.set_ylabel('Frame difference', color='#8b949e') ax2.set_xlabel('Frame number', color='#8b949e') ax2.legend(facecolor='#21262d', labelcolor='#8b949e', fontsize=9) plt.tight_layout() plt.savefig('video_analysis.png', dpi=150, bbox_inches='tight', facecolor='#0d1117') plt.show() if __name__ == '__main__': path = sys.argv[1] if len(sys.argv) > 1 else 'test_video.mp4' rate = int(sys.argv[2]) if len(sys.argv) > 2 else 1 analyse_video(path, rate)
To test your detector, you need a video with steganography embedded in known frames. Use this helper to embed data into specific frames of an existing video:
""" Embed a message into specific frames of a video using LSB steganography. For detection testing only. Usage: python video_embed.py input.mp4 output.mp4 "secret message" 10 20 30 (embed message into frames 10, 20, 30) """ import cv2 import numpy as np import sys def embed_in_frame(frame: np.ndarray, message: str) -> np.ndarray: """Embed message bytes into LSB of frame pixels (blue channel).""" data = message.encode() + b'\x00' # null terminator bits = ''.join(format(b, '08b') for b in data) flat = frame[:, :, 0].flatten().copy() if len(bits) > len(flat): raise ValueError("Message too long for frame") for i, bit in enumerate(bits): flat[i] = (flat[i] & ~1) | int(bit) result = frame.copy() result[:, :, 0] = flat.reshape(frame.shape[:2]) return result def embed_video(src: str, dst: str, message: str, target_frames: list): cap = cv2.VideoCapture(src) fps = cap.get(cv2.CAP_PROP_FPS) w = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH)) h = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT)) fourcc = cv2.VideoWriter_fourcc(*'mp4v') out = cv2.VideoWriter(dst, fourcc, fps, (w, h)) frame_num = 0 target_set = set(target_frames) embedded = [] while True: ret, frame = cap.read() if not ret: break if frame_num in target_set: frame = embed_in_frame(frame, message) embedded.append(frame_num) out.write(frame) frame_num += 1 cap.release(); out.release() print(f"Embedded in frames: {embedded}") print(f"Output: {dst}") if __name__ == '__main__': src, dst, msg = sys.argv[1], sys.argv[2], sys.argv[3] frames = [int(x) for x in sys.argv[4:]] embed_video(src, dst, msg, frames)
Video Steganalysis Lab
- Download a short (10-30 second) Creative Commons MP4 clip from Pixabay or Pexels. Run video_embed.py to embed "OPERATION NIGHTFALL" into frames 50, 100, and 150.
- Run video_steg_detector.py on both the clean and stego versions. Do frames 50, 100, 150 get flagged?
- Modify the embed script to embed data into every frame. Rerun the detector — does it still detect anomalies? Why is this a more effective evasion?
- Discussion: A threat actor uploads 5-minute "screen recording tutorial" videos to YouTube daily. Each contains a C2 command embedded in frame 1000. Design a detection pipeline for this scenario. What data sources would you need? What SIEM rule would you write?
In a CTF, you receive a file and must extract a hidden flag. The critical skill is not knowing all tools — it's having a systematic methodology to work through unknown files. Always follow this sequence before reaching for specialised tools:
file mystery_file and xxd mystery_file | head -4 to read the magic bytes. A JPEG starts with FF D8 FF; a PNG with 89 50 4E 47; a ZIP with 50 4B 03 04.FF D9. For PNG: after 49 45 4E 44 AE 42 60 82. Run binwalk mystery_file to find embedded files and trailers.exiftool mystery_file and look at every single field. Flags are frequently hidden in Comment, UserComment, Artist, Copyright, GPS coordinates, or custom fields.strings mystery_file | grep -i flag and look for readable text. Run binwalk -E mystery_file to plot entropy — high-entropy regions suggest compression or encryption; sharp transitions suggest embedded encrypted payloads.steghide extract -sf file.jpg; PNG/BMP → zsteg file.png; audio → Audacity spectrogram + stegolsb; any image → Stegsolve bit plane viewer.password, steghide, ctf), or any prominent word in the challenge.Challenge 1 — The suspicious JPEG
Scenario: You receive a JPEG of a cat. The challenge hint says: "Sometimes the best place to hide something is in plain sight."
Approach:
- Run
file cat.jpg→ confirms JPEG - Run
exiftool cat.jpg→ check all fields → nothing obvious - Run
binwalk cat.jpg→ detects a ZIP archive appended after the JPEG end marker - Run
binwalk -e cat.jpg→ extracts the embedded ZIP to_cat.jpg.extracted/ - Unzip the extracted archive → contains
flag.txt
# binwalk cat.jpg DECIMAL HEXADECIMAL DESCRIPTION --------------------------------------------------------------------------- 0 0x0 JPEG image data, JFIF standard 52341 0xCC75 Zip archive data, "flag.txt" # binwalk -e cat.jpg # cat _cat.jpg.extracted/flag.txt FLAG{zip_inside_a_jpeg_classic}
Challenge 2 — The silent WAV
Scenario: A 5-second WAV file of silence. Challenge hint: "You need to see it to hear it."
Approach:
- Open in Audacity → plays as silence, no visible waveform
- Change view: Spectrogram (View → Show Spectrogram Scale)
- Zoom in on the high-frequency range (8–16 kHz) → text appears in the spectrogram image
- The flag is written in the frequency domain, readable only in spectrogram view
Sonic Visualiser or Python's librosa can generate audio where specific frequencies encode an image or text. The data is inaudible because it sits above or at the threshold of human hearing, but the spectrogram reveals it visually.Challenge 3 — The password-protected JPEG (steghide)
Scenario: A JPEG of a beach. Steghide is involved. No password given. Challenge title: "Tropical".
Approach:
- Try
steghide extract -sf beach.jpgwith no password → fails - Try common passwords:
password,steghide,ctf,tropical→ "tropical" works - Extracted
secret.txtcontains the flag
# steghide extract -sf beach.jpg -p tropical wrote extracted data to "secret.txt". # cat secret.txt FLAG{steghide_password_in_title}
Challenge 4 — The PNG with hidden bit planes (zsteg)
Scenario: A PNG image that looks like a normal screenshot. No obvious hint.
Approach:
- Run
zsteg challenge.png→ zsteg automatically scans all channel/bit combinations - Output includes a line for
b1,rgb,lsb,xythat shows readable text - Extract that specific channel:
zsteg -e b1,rgb,lsb,xy challenge.png > out.bin - Read out.bin — contains the flag, possibly with some junk bytes at the end
# zsteg challenge.png imagedata .. text: "KJKJKJKJKJ" b1,r,lsb,xy .. text: "random noise" b1,rgb,lsb,xy .. text: "FLAG{lsb_rgb_all_channels}" b1,bgr,lsb,xy .. text: "junk..." b4,r,lsb,xy .. text: "..."
Challenge 5 — The mystery file (binwalk + strings + file carving)
Scenario: A file with no extension called data. No hints at all.
Approach:
file data→ reports "data" (unknown)xxd data | head -10→ magic bytes don't match any known formatstrings data→ mostly garbage, but some readable fragmentsbinwalk -E data→ entropy plot shows two distinct regions — high entropy (compressed/encrypted) and a low-entropy region at offset ~4000binwalk data→ detects a PNG image at offset 4096dd if=data of=extracted.png bs=1 skip=4096→ extract the PNG- Open extracted.png → contains the flag as visible text in the image
# xxd data | head -3 00000000: dead beef cafe babe 0000 0000 0000 0000 ................ # binwalk data DECIMAL HEXADECIMAL DESCRIPTION --------------------------------------------------------------------------- 0 0x0 Unknown header bytes 4096 0x1000 PNG image, 640x480 # dd if=data of=extracted.png bs=1 skip=4096 # file extracted.png extracted.png: PNG image data, 640 x 480, 8-bit/color RGB
Module Capstone Challenge
Your instructor will provide a folder of 5 challenge files. Each uses a different technique from this module. Work through the methodology for each file without any hints. Document your approach for every step — including dead ends.
- Challenge A: extract the hidden flag from the image file
- Challenge B: identify the covert data in the network capture (PCAP provided)
- Challenge C: find the hidden content in the PDF document
- Challenge D: apply chi-square or DCT analysis to confirm whether the provided JPEG contains steganography, and estimate the payload size
- Challenge E: the mystery file — identify, extract, analyse
- Write a 1-page forensic report for each challenge: tools used, methodology, findings, and any false leads
You have now worked through the core techniques that security analysts use to detect and break steganographic hiding. Here is a consolidated view of what each lesson contributed:
| Lesson | Core technique | Key tool(s) | Real-world application |
|---|---|---|---|
| 08 | Automated batch scanning | StegExpose | Email attachment triage, DLP |
| 09 | Chi-square statistical attack | Python / scipy | Confirming LSB embedding |
| 10 | DCT coefficient analysis | jpegio / Python | JPEG forensics, APT malware analysis |
| 11 | Network covert channels | Scapy / Wireshark | NDR, SIEM detection rules |
| 12 | Document forensics | pdfid, pdf-parser, exiftool | Malicious document analysis |
| 13 | Video frame analysis | OpenCV / Python | C2 channel detection, DLP |
| 14 | Unknown file methodology | binwalk, zsteg, steghide, strings | Incident response, CTF, forensics |