Skip to content

Add the supervised YOLO baseline to the comparison harness (#51) - #76

Merged
jonfroehlich merged 2 commits into
mainfrom
feat/yolo-baseline-51
Jul 28, 2026
Merged

Add the supervised YOLO baseline to the comparison harness (#51)#76
jonfroehlich merged 2 commits into
mainfrom
feat/yolo-baseline-51

Conversation

@jonfroehlich

@jonfroehlich jonfroehlich commented Jul 28, 2026

Copy link
Copy Markdown
Member

The issue #51 architecture-vs-data ablation: does a generic detector trained on
the RampNet dataset also beat the zero-shot field, or is RampNet's keypoint
architecture doing the work?

This is the code half only. The training record it produced is #77, opened
concurrently by another session — the two are disjoint after a rewrite of this
branch (see "Overlap with #77" below).

What's here

  • YoloDetector + yolo_results_to_boxes in detectors.py, plus the yolo
    branch in build_detector. Identity is the weights file stem, so the
    detection cache key is machine-independent; signature() hashes the weights
    bytes, so a retrain busts the cache. It emits scored boxes, so YOLO gets AP,
    PR curves and --sweep like the other open detectors.
  • compare.py: --yolo-model, --yolo-conf (0.05 floor), --yolo-iou,
    --yolo-imgsz. requirements-vlm.txt: ultralytics.
  • prepare_yolo_dataset.py — tiles+pano dataset builder with pluggable box-size
    strategies and deterministic background thinning.
  • run_yolo_{data,prep,train}.slurm and hyak_yolo_runbook.sh (repo root,
    beside hyak_qwen_runbook.sh).
  • +10 tests (187 pass, up from main's 177).

Box size is a train-only knob: evaluation reduces predictions to centre
points and matches at radius 0.022, so width and height are never scored.

Overlap with #77 — one thing to fix at merge time

This branch originally also carried the training record; I dropped it once #77
appeared, so the two PRs now touch disjoint files. One conflict survives and
is worth catching whichever merges second:

#77 adds scripts/model_comparison/yolo_baseline/run_yolo_train.slurm and
.../yolo_baseline/hyak_yolo_runbook.sh. This PR adds the canonical
scripts/model_comparison/run_yolo_train.slurm and the repo-root
hyak_yolo_runbook.sh. Git will not flag it — different paths — but merging
both leaves two copies of each script, and:

  • the two run_yolo_train.slurm files are byte-identical;
  • the two runbooks are not: Preserve the supervised-YOLO baseline training record (#51) #77's copy is three lines older, predating the
    ARG_MAX fix to the dataset count display (ls "$DATA/$s"/*.jpg | wc -l
    overflowed on the ~150k-file train split and printed a false train: 0).

Suggested resolution: keep the canonical copies from this PR, drop the two under
yolo_baseline/, and have that README point at them. The provenance those copies
existed to preserve is already in git history. The difference is display-only and
cannot affect the recorded runs, but shipping the older text as a second live copy
would be a real trap.

Status

Training is still running; nothing here is a result. The benchmark-city
evaluation against RampNet has not been run — see #77 for what the runs so far
do and do not support.

🤖 Generated with Claude Code

The architecture-vs-data ablation for issue #51: does a generic detector
trained on the RampNet dataset also beat the zero-shot field, or is
RampNet's keypoint architecture doing the work?

- YoloDetector + yolo_results_to_boxes in detectors.py, plus the `yolo`
  branch in build_detector. Identity is the weights file stem, so the
  detection cache key is machine-independent; signature() hashes the
  weights bytes, so a retrain busts the cache. It emits scored boxes, so
  YOLO gets AP, PR curves and --sweep like the other open detectors.
- compare.py: --yolo-model / --yolo-conf (0.05 floor) / --yolo-iou /
  --yolo-imgsz. requirements-vlm.txt: ultralytics.
- prepare_yolo_dataset.py: tiles+pano dataset builder with pluggable
  box-size strategies and deterministic background thinning.
- run_yolo_{data,prep,train}.slurm and hyak_yolo_runbook.sh (repo root,
  beside hyak_qwen_runbook.sh).
- +10 tests.

The training record these scripts produced is PR #77, kept separate so
this PR is reviewable as code.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@jonfroehlich
jonfroehlich force-pushed the feat/yolo-baseline-51 branch from 234c1b1 to 29449c6 Compare July 28, 2026 21:17
@jonfroehlich jonfroehlich changed the title Add the supervised YOLO baseline: harness wiring, cluster pipeline, and training record (#51) Add the supervised YOLO baseline to the comparison harness (#51) Jul 28, 2026
jonfroehlich added a commit that referenced this pull request Jul 28, 2026
- Config table: record the as-run batches from args.yaml (y11x_tiles 3->12
  across its two attempts, y26_tiles 6) -- batch is the load-bearing variable
  in the collapse hypothesis, so the table must be exact
- Provenance: box-size corrected to pitch/1.8m per run_yolo_prep.slurm's as-run
  defaults, with a TODO to confirm from the klone prep log before scratch
  purges; add y11x_tiles's inferred job ID 37745359
- Drop the run_yolo_train.slurm / hyak_yolo_runbook.sh snapshots: the canonical
  copies land via #76 (launcher byte-identical; runbook drift was display-only,
  documented in #76). Removes the two-live-copies trap at merge time.
- .gitignore: enforce the stated weights-out policy structurally (*.pt, *.pth)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Runbook: default ACCOUNT to ckpt-makelab (run_yolo_train.slurm pins the ckpt
  partition, which the old gpu-l40s default cannot submit to) and pin per-config
  BATCH in the train stage -- unpinned, jobs ran batch=-1 (auto), defeating the
  identical-LR-schedule rationale the .slurm header states
- Runbook: align BOX_SIZE default with the as-run prep (pitch, ramp 1.8m, per
  run_yolo_prep.slurm) and point the multi-hour data/prep stages at their
  compute-node sbatch wrappers (login nodes reap heavy processes)
- prepare_yolo_dataset.py: stamp the full prep config into data.yaml as comments
  -- box size et al. are train-only knobs no downstream args.yaml records, and
  exactly that gap made the #77 provenance question unanswerable from disk;
  regression test added (188 pass)
- prepare_yolo_dataset.py: render each overlay view once, not twice; --overlay-n
  help now says PER SPLIT
- detectors.py: hoist the hashlib import to module top

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@jonfroehlich

Copy link
Copy Markdown
Member Author

Review fixes pushed in ebbc27b:

  • Runbook train: ACCOUNT now defaults to ckpt-makelab (the .slurm pins -p ckpt-g2 -q ckpt-gpu, which the old gpu-l40s-makelab default cannot submit to), and sub() now pins per-config BATCH (6/12/6/4/2/4) — unpinned, runbook-launched jobs ran batch=-1 (auto), defeating the identical-LR-schedule rationale the .slurm header states.
  • Runbook: BOX_SIZE default is now pitch + RAMP_SIZE_M=1.8, matching run_yolo_prep.slurm's as-run defaults, and the multi-hour data/prep stages now point at their compute-node sbatch wrappers (login nodes reap heavy processes).
  • prepare_yolo_dataset.py now stamps the full prep config into data.yaml as comments, with a regression test (188 pass). Box size is a train-only knob that no downstream args.yaml records — exactly the gap that made the Preserve the supervised-YOLO baseline training record (#51) #77 provenance question unanswerable from disk.
  • Minor: overlay views render once instead of twice; --overlay-n help says PER SPLIT; hashlib import hoisted to module top.

The "Overlap with #77" merge-time task is resolved ahead of time: #77 dropped its two script copies and its README points at the canonical paths here, so no dedup is needed whichever merges second.

🤖 Generated with Claude Code

@jonfroehlich
jonfroehlich merged commit 795a26b into main Jul 28, 2026
2 checks passed
jonfroehlich added a commit that referenced this pull request Jul 28, 2026
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
jonfroehlich added a commit that referenced this pull request Jul 28, 2026
Lands the YOLO baseline training record with the corrected instability diagnosis, four regenerable figures, and the deletion of the duplicate script snapshots that #76 now carries canonically.
@jonfroehlich
jonfroehlich deleted the feat/yolo-baseline-51 branch July 28, 2026 22:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant