skills and scripts to help diagnose regressions

Author: hmottestad

.codex/skills/query-plan-snapshot-cli/SKILL.md

@@ -5,10 +5,62 @@ description: Use QueryPlanSnapshotCli to capture and compare RDF4J query plans,
 
 # query-plan-snapshot-cli
 
-Use this skill to run reproducible query-plan captures and classify likely regression/improvement signals.
+Use this skill to run reproducible query-plan captures, triage historical theme-query benchmark results, and classify likely regression/improvement signals.
 
 ## Fast workflow
 
+1. Capture raw benchmark output into a normalized result file when needed.
+2. Analyze the newest dated run against historical results.
+3. Drill into the fastest known runs for a specific theme/query.
+4. If needed, capture baseline/candidate plan snapshots and diff them semantically.
+
+## History triage
+
+Result files live in:
+
+- `core/sail/lmdb/src/test/java/org/eclipse/rdf4j/sail/lmdb/benchmark/theme-query-benchmark-results`
+
+Normalize raw JMH output into a new result file:
+
+- `pbpaste | scripts/theme-query-benchmark-results.sh capture`
+- `scripts/theme-query-benchmark-results.sh capture raw-jmh.txt`
+
+Analyze only the queries that are more than 20% slower than history:
+
+- `core/sail/lmdb/src/test/java/org/eclipse/rdf4j/sail/lmdb/benchmark/theme-query-benchmark-results/analyze-theme-query-history.sh`
+
+Sort regressions from biggest to smallest:
+
+- `core/sail/lmdb/src/test/java/org/eclipse/rdf4j/sail/lmdb/benchmark/theme-query-benchmark-results/analyze-theme-query-history.sh --sort-regressions`
+
+Only print the top N regressions:
+
+- `core/sail/lmdb/src/test/java/org/eclipse/rdf4j/sail/lmdb/benchmark/theme-query-benchmark-results/analyze-theme-query-history.sh --top 10`
+
+Analyze every latest query, including current-run wins over previous best:
+
+- `core/sail/lmdb/src/test/java/org/eclipse/rdf4j/sail/lmdb/benchmark/theme-query-benchmark-results/analyze-theme-query-history.sh --all`
+
+Drill into the three fastest known runs for one theme/query and print optimized plan/query when present:
+
+- `core/sail/lmdb/src/test/java/org/eclipse/rdf4j/sail/lmdb/benchmark/theme-query-benchmark-results/analyze-theme-query-history.sh --theme PHARMA --query-index 10`
+
+Interpretation:
+
+- Default mode: newest dated file only for the “latest” baseline; compares against all other `results-*.md`, including `results-develop.md` and `results-main-branch.md`, but prints only queries where latest is more than 20% slower than historical best.
+- `--sort-regressions`: flat regression list, biggest slowdown first.
+- `--top N`: top N regressions only; implies regression sorting.
+- `--all`: prints every latest query; if latest is a new best it prints how much faster it is than the previous best.
+- Query detail mode: top three runs sorted by score ascending; ties prefer richer files with plan/query content.
+- `plan no | query yes`: optimized query rendered, no physical plan block in that result file.
+- `plan no | query no`: summary-only run or no per-query capture in that file.
+
+Use this path when the goal is optimizer-loop work: find the fastest known plan/query for a theme/query, then compare new runs back to that history before touching production logic.
+
+## Snapshot diff workflow
+
+Use this when you need semantic plan diffs between two controlled captures of the same query.
+
 1. Capture baseline run (main/reference commit).
 2. Capture candidate run (changed commit) with same query selector + `--query-id`.
 3. Produce semantic diff (`--compare-existing`).