flowscope

A pcap-driven TCP flow reconstructor and reliability-anomaly detector — built as burstgen's independent verification partner.

What this is for

burstgen generates traffic and reports what it believes it sent, plus (optionally) what the target's kernel counters show happened. Both of those are self-reports. flowscope closes the loop: it reads the actual packets that crossed the wire — captured independently with tcpdump, parsed with no code shared with burstgen — and reconstructs its own answer to "what happened here." When the two agree, that agreement means something, because they have no way to be fooling each other.

What it does

Given a .pcap file, flowscope:

1. Parses every Ethernet/IP/TCP/UDP frame into a structured packet record
2. Groups packets into per-connection flows (both directions of one conversation, correctly reassembled even if the capture starts mid-flow)
3. Runs three anomaly detectors over every flow:
   • Retransmit storms — several retransmitted segments clustered in time on one flow
   • Unanswered SYNs — a connection attempt (including the kernel's own retries) that never got a SYN-ACK — the packet-level signature of a target silently dropping connections
   • Out-of-order bursts — segments arriving out of sequence order, clustered in time
4. Optionally cross-checks its own independently-derived counts against a burstgen events.json from the same run, and reports whether they agree

Usage

cargo build --release

# Capture traffic while burstgen runs against a target
sudo tcpdump -i lo -n port 8080 -w capture.pcap &
burstgen conn-rate --target 127.0.0.1:8080 --rate 2000 --duration-secs 5

# Analyze it
./target/release/flowscope analyze --pcap capture.pcap \
    --burstgen-events burstgen-out/events.json
cat flowscope-out/report.md

Real test results (this build)

Tested against two real, independently-captured scenarios during development — not synthetic/fabricated data:

Scenario 1 — healthy listener, 671 connections: flowscope reconstructed exactly 671 flows from the raw packets. Correlation against burstgen's own report: both connection-attempt and success counts matched exactly (671=671, 671=671, zero diff). Zero findings — correctly reported no anomalies on a healthy run.

Scenario 2 — deliberately misconfigured listener (backlog=1), 1816 attempts: flowscope reconstructed 1816 flows and flagged 1814 "unanswered SYN" findings — 1816 attempts − 2 successes = 1814, matching burstgen's own success count exactly. Correlation: both counts agreed exactly with burstgen's self-report (1816=1816, 2=2).

A real bug found and fixed while building this

The first version of the unanswered-SYN detector counted every SYN packet, not every connection attempt. Under sustained backlog overflow, Linux's kernel retransmits an unanswered SYN several times (observed ~6x on average, with exponential backoff) before giving up — all on the same flow, since the client doesn't open a new port to retry. The first version flagged each retransmission as a separate finding and counted each one separately in the correlation table, which inflated the finding count by roughly the retry factor and made the correlation against burstgen's per-attempt count disagree for a reason that had nothing to do with an actual measurement discrepancy — it was double-counting retries, not measuring anything wrong with the target.

Fixed by grouping to one finding per flow (the first SYN through the last retry, with a count field showing how many retries occurred) rather than one finding per SYN packet. Re-verified against the same capture after the fix: the correlation now agrees exactly, and the finding count (1814) is now internally consistent with the attempts-minus-successes arithmetic, which it wasn't before.

Known limitations (v1)

• Retransmission detection uses exact-sequence-number matching, which catches full retransmits (the common case for bulk transfers like burstgen's large-flow profile) but not partial/overlapping retransmits with different segment boundaries.
• IPv6 is parsed but untested end-to-end (no IPv6 test capture was available in this environment).
• No live capture mode — flowscope only reads .pcap files already written by tcpdump/Wireshark, it does not capture traffic itself, deliberately, to avoid needing libpcap or elevated capture permissions as a build/runtime dependency.