contribution: 60fps restamp fix (HW-verified) + IN-005 registry/soak + IS-05 shim + ANC data-flow + v1.1.0 pin

[guycochran]

Review guide

This branch is the Oct 7–8 contribution-seam work: the 60fps restamp fix (the headline, HW-verified), plus the Tier-2/3 roadmap items and the NMOS control-plane direction. 10 commits, ~1400 lines, mostly additive. Everything is HW-verified, unit-tested, or dormant/opt-in — nothing changes a running path by default. Clean on master (no rebase).

Suggested review order (each group is independent; skip to what you care about):

1. 60fps restamp fix — the core change ⭐ HW-VERIFIED

tools/contribution_core.py · commits e7da4e3→13fcc0c

• The bug: on a 60fps source the restamp servo's err ran away / hard-relock-looped (-41s, then -4.7s churn, then a death loop). Root cause: the 1000ms SRT jitter buffer flushes a burst ~1s ahead of the just-started wall clock, so any offset-follows-source design breaks.
• The fix (489b115): stop following buf.pts; drive a local wall-clock grain grid + drop ahead-of-realtime arrivals (the FINDINGS §"Separate the output clock from the input" pattern, already proven in flow_stabilizer/mxl_multiview). Monotonic by construction (mxlsink has no backward-index guard), paced to wall-clock.
• HW-verified on a real 1080p60 camera: clean 30fps grid, err bounded 5–18ms, 0 hard-relocks, 0 -5.
• Note the commit trail shows two superseded intermediate attempts (54d2e8a slew servo, e7da4e3 grain_ns) — the grid in 489b115 is the one that HW-verified; 7c362b0 adds probe-driving regression tests (HW caught a KeyError the launch-string tests couldn't).
• Review focus: the _restamp grid logic + the drop-ahead-of-realtime guard.

2. IN-005 ingress registry + Timing-Model soak tool

tools/contribution_core.py, tools/ingress-soak.sh, docs/CONFIG.md · commit fe317f3

• Per-ingest JSON record ($MXL_INGRESS_DIR/<flow8>.json, default /tmp/mxl-ingress, opt-out via empty env) — source caps + the restamp's offset/err/grain/relock/dropped. Makes the restamp observable (would've made the 60fps churn obvious at a glance).
• ingress-soak.sh renders PASS/FAIL in the Timing Model's terms (bounded err, no churn, frames advancing). Both paths tested.

3. IS-05 control plane — NMOS node made routable, then v1.2

tools/nmos_node.py, docs/ADR-001-...md, docs/TIMED-DATA-READINESS.md · commits 871c066, 83c83ab, 888417f

• IS-05 Connection API shim: PATCH a receiver's /staged + activate → a real selector cut. Then matured to v1.2 with MXL flow identity in transport_params ({mxl_flow_id, mxl_domain_id}) + disable/re-enable — the shape a real controller (garethsb/mxl-nmos-c-example) speaks.
• ADR-001 records the stance this converges to: NMOS owns connection control, we own the media engine — written after @rochonma closed our reader-wedge report (Readers wedge silently when a source flow is recreated (status APIs still report running) cbcrc/mxl-hands-on#34) with "embrace NMOS."
• 8 tests drive the real stdlib server end-to-end.
• Review focus / input wanted: does the ADR's layer split match how you'd position this? And is transport_params: [{mxl_flow_id, mxl_domain_id}] the shape you'd standardize on?

4. ANC / closed-caption data essence 🟡 WRITE-path HW-verified, DORMANT

tools/adapters.py (CaptionFileAdapter), tools/contribution_core.py (essence='data'), docker/guest-ingest.Dockerfile (WITH_CAPTIONS opt-in) · commit 3050d72

• Produces a real video/smpte291 MXL data flow from captions (CEA-608 → ST-2038 → mxlsink). Write path HW-verified (flow created, grains committed). Dormant: no entrypoint/quickstart instantiates it; Dockerfile flag defaults off. Safe to carry.
• Read path (mxlsrc data-flow consumer) recovers 0 grains — a tracked follow-up (sparse-ANC / reader semantics), not blocking.

5. Docs / pins

docs/VERSIONS.md (MXL SDK v1.1.0 pin + verify recipe), docs/ROADMAP-FROM-SPECS-2026-10.md (the tiered plan this all came from).

Status summary

Group	Status
60fps restamp fix	✅ HW-verified
IN-005 registry + soak	✅ unit-tested
IS-05 v1.2 shim	✅ unit + functional smoke-tested
Caption data essence	🟡 write-path HW-verified, dormant; read path = follow-up
v1.1.0 pin + docs	✅

Tests: 152 py + 36 node green. Follow-ups (not blocking, both want HW): caption read path, fan-out -4 on 60fps — both diagnosed, deferred to the local AVX host.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GwSKEZZz4yEc2CGq5hcTmT

[guycochran]

⚠️ HW test result: fix is INCOMPLETE — do not merge

Ran this branch on the Azure lab VM (mxl-lab, D32s_v5) against a real 1080p60 source (studio camera via the cam1 relay, confirmed r_frame_rate=60/1) and a paced synthetic 60fps. Isolated the pure restamp path (standalone guest_ingest.py srt-listen → _restamp → mxlsink, no A/V fan-out).

Finding: _grain_ns works as written, but it does not fix the 60fps bug.

The restamp probe sits after the conform stage's videorate ! framerate=30/1, so the pad's negotiated caps are 30fps — _grain_ns correctly returns 33.3ms (grain=33.3ms in the diag). The step was therefore never the root cause. The real failure, observed on a paced 60fps source:

diag n=600 fps=39.32 err=-4709ms grain=33.3ms
cadence re-synced by -4715ms   (repeating every batch, never settles)

• videorate decimates 60→30 unevenly (fps=39, not 30), and the offset servo never settles — a large **standing offset (-4.7s)** with perpetual re-sync churn.
• Under an unpaced flood (synthetic x264 no -re), the monotonic clamp advances the PTS one grain per buffer, so a burst marches mapped ahead of wall-clock without bound → err=-135s, and 9× -5. The clamp pins spacing but not rate vs wall-clock.

So the actual bug is the offset servo + videorate interaction on a faster-than-canonical source, not the grain-step constant. _grain_ns is a correct, harmless improvement (right thing for 50/59.94 flows) but insufficient.

Next (real fix, follow-up)

• Decimate/pace before the restamp probe so it sees a clean canonical cadence (move the probe, or rate-limit the source leg), OR
• Make the offset re-sync settle instead of churning (the ~-4.7s standing offset suggests the MARGIN/RESYNC interaction with a decimated cadence is the lever), OR
• Pace the source leg to grain cadence upstream of _restamp.

Keeping this branch open as needs-hw-verify → reclassify: fix incomplete. The _grain_ns helper + tests + the roadmap/VERSIONS docs are still worth keeping; the restamp behaviour change needs a second pass. VM deallocated; cam1 relay restored.

[guycochran]

Second pass: proportional slewing servo (supersedes the _grain_ns-only attempt)

The HW run showed the real failure wasn't the grain step — it was the offset servo. videorate decimates 60→~30 unevenly (fps=39), and the old step re-sync (offset += err after RESYNC_COUNT frames) overcorrected and re-synced every batch, leaving a standing offset ~−4.7s that never settled.

Fix (commit 54d2e8a): replace the step re-sync with the proven slewing servo from layout_pgm v4 / mxl_multiview / flow_stabilizer (FINDINGS §"Separate the output clock from the input"):

• proportional controller: offset += clamp(err * 0.02, ±80µs) per frame → converges to a stable standing offset instead of fighting
• gross error (>1s) still hard re-locks once, with an 8-relocks/10s escape hatch
• explicit buf.duration = grain_ns

Kept the Oct-6 monotonic clamp (mxlsink has no backward-index guard) and the rate-derived grain_ns. Removed the now-dead RESYNC_NS/RESYNC_COUNT/drift_n.

Tests: 137 py + 36 node green.

Still needs-hw-verify

Next lab session (60fps camera): expect err to converge to a small bounded value with no per-batch re-sync churn, 0 -5, cuttable. Not merging until observed. The separate fan-out -4 not-negotiated on 60fps is still open and independent.

[guycochran]

📋 Scope & merge status (this branch grew past its name)

This branch now carries four related pieces of work. Splitting was considered and rejected: the IN-005 registry records the servo's hard-relock count and grain step, so it can't be decoupled from the slew-servo change without gutting the fields that make it useful. Instead, here's an honest per-piece status.

Piece	Commits	Tests	HW-gated?	Merge readiness
Restamp slew servo (the 60fps fix)	e7da4e3 54d2e8a	✓ 139 py	YES — needs 60fps camera re-verify	hold for HW
IN-005 ingress registry (/tmp/mxl-ingress/<flow8>.json)	fe317f3	✓	no (rides on servo state)	ready once servo verified
Timing-Model soak verdict (tools/ingress-soak.sh)	fe317f3	both paths tested locally	no	ready
ANC/CC data flow (essence=data, CaptionFileAdapter)	3050d72	✓	YES — needs rsclosedcaption source build + round-trip on HW	hold for HW
v1.1.0 SDK pin (VERSIONS + Dockerfile) + specs ROADMAP	e7da4e3	n/a	no	ready

Verification plan (single next lab session):

1. Run tools/ingress-soak.sh guest1 against the 60fps camera through the slew servo → expect PASS (bounded err, 0 relock churn). This simultaneously validates the servo and the registry/soak.
2. Build the caption image (--build-arg WITH_CAPTIONS=1 + rsclosedcaption from source) and run the ST-2038 round-trip.
3. Chase the independent fan-out -4 not-negotiated on 60fps (logged earlier).

Reclassify from needs-hw-verify to fixed per-piece as each is observed. Nothing here has touched the live system; all changes are code/tests/docs.

[guycochran]

✅ HW-VERIFIED: the 60fps restamp fix works (Oct 8, real 1080p60 camera)

Ran on the Azure lab against the real 1080p60 studio camera (via the cam1 relay, confirmed r_frame_rate=60/1), pure restamp path (standalone guest_ingest.py srt-listen, domain + IN-005 registry mounted).

It took three in-session iterations, all HW-driven:

1. _grain_ns + slew servo → still churned / hard-relock death-looped (err=-1.03s growing −30ms/frame → 8-relock escape → restart loop). Every offset-follows-source design fails because the 1000ms SRT jitter buffer flushes a burst ~1s ahead of the just-started wall clock.
2. Rewrote to a local wall-clock grain grid + drop-ahead-of-realtime (FINDINGS §"Separate the output clock from the input") → HW caught a KeyError('last_mapped') (state dict never had the key; launch-string tests couldn't see it) → added probe-driving regression tests.
3. Fixed the err diagnostic (was a meaningless source-vs-offset gap reading −10s on a healthy grid) to report the grid residual (mapped − target).

Final HW result:

cadence offset locked: 9394ms
diag n=300 fps=29.81 err=5ms   grain=33.3ms dropped=300
diag n=600 fps=29.79 err=18ms  grain=33.3ms dropped=303

IN-005 record: err_ms: 17.7, hard_relocks: 0, dropped: 303, frames: 600.

• ✅ clean 30fps grid (correctly decimated from 60), ✅ err bounded ~5–18ms (was −41s / −10s / death-loop), ✅ 0 hard-relocks, ✅ 0 -5, ✅ burst-drop guard working (half the 60fps frames dropped to hit 30fps). Would PASS ingress-soak.sh.

Reclassify restamp slew servo → HW-verified FIXED. Commits: 489b115 (grid), 7c362b0 (KeyError + probe tests), + the err-diag fix. VM deallocated.

Still owed on this branch: the caption round-trip (needs rsclosedcaption build) and the fan-out -4 on 60fps (independent). A longer continuous ingress-soak.sh run would be the belt-and-suspenders confirmation (this run used a 25s clip).

[guycochran]

🟢 Caption/ANC data flow — HW-verified on the WRITE path (Oct 8)

Built libgstrsclosedcaption.so (gst-plugins-rs 0.14.5, the 4 ANC/CC elements) and ran the producer round-trip against real MXL.

Verified working on hardware:

• ✅ All 4 elements load: tttocea608, cctost2038anc, st2038anctocc, cea608tott
• ✅ Producer chain runs clean (subparse → CEA-608 → ccconverter → cctost2038anc → meta/x-st-2038 → mxlsink), reaches PLAYING, EOS, no errors
• ✅ mxlsink creates a real video/smpte291 MXL data flow — flow_def.json confirms format: urn:x-nmos:format:data, media_type: video/smpte291, grain_rate 30000/1001, and the ring buffer is populated (grains/data.0..4). This is Tier 3.1's core claim proven: our essence='data' path produces a spec-correct MXL data flow from live caption input.

Not yet closed (read path): the mxlsrc consumer connects + negotiates but recovered 0 grains in-test. Most likely sparse-ANC timing — pop-on CEA-608 emits ST-2038 only on caption change, so a late-joining reader lands on empty/duplicate ring slots — rather than an integration defect (the write side is proven and the flow is structurally complete). Closing the reader loop is CEA-608 timing nuance, not the data-flow plumbing.

Build logistics (for the record): rsclosedcaption was built on prodbots (VM apt is broken; prodbots apt needed mirrors.kernel.org + forced-IPv4 + the full pango/cairo/gtk dep set) and the .so carried to the VM — prodbots is a no-AVX QEMU CPU so it can build but can't run mxlsink; the AVX-512 VM runs it. The .so is saved at ~/ccbuild-out/libgstrsclosedcaption.so for reuse.

Reclassify Tier 3.1: producer/write path HW-VERIFIED; consumer/read path pending (sparse-ANC reader timing). VM deallocated.

[guycochran]

✅ Ready for review — what lands here, and why it's all safe

Un-drafted. This branch is cleanly on top of master (f2c45f6, no rebase needed). Everything here is HW-verified, unit-tested, or dormant/opt-in — nothing changes a running path by default.

Lands now

Piece	Status
60fps restamp fix (local wall-clock grain grid + drop-ahead-of-realtime)	✅ HW-verified on a real 1080p60 camera: clean 30fps, err bounded 5–18ms, 0 hard-relocks, 0 -5
IN-005 ingress registry (/tmp/mxl-ingress/<flow8>.json)	✅ unit-tested; opt-out via empty MXL_INGRESS_DIR
Timing-Model soak tool (tools/ingress-soak.sh)	✅ both PASS/FAIL paths tested
IS-05 connection shim (NMOS node now routable)	✅ 5 end-to-end tests
v1.1.0 SDK pin + specs ROADMAP + Timed-Data note	✅ docs
ANC/CC data essence (CaptionFileAdapter, essence='data')	🟡 write path HW-verified (mxlsink creates a real video/smpte291 flow); dormant — no entrypoint/quickstart instantiates it, Dockerfile WITH_CAPTIONS defaults off

Tests: 149 py + 36 node green. The caption code is inert unless explicitly invoked, so it's safe to carry in master.

Tracked follow-ups (NOT blocking this merge; both want hardware)

1. Caption read path — mxlsrc consumer recovered 0 grains in-test (likely sparse pop-on CEA-608 ANC timing, not an integration defect; write path is proven). Built .so saved for reuse.
2. Fan-out -4 not-negotiated on 60fps — independent guest A/V-fan-out bug, surfaced during HW testing.

Both are documented in docs/ROADMAP-FROM-SPECS-2026-10.md and best finished on the (incoming) local AVX host rather than the billable VM.

The note about the old "slew servo" in earlier commits is superseded — commit 489b115 replaced it with the grid design that HW-verified; 54d2e8a/e7da4e3 are intermediate steps in the same fix.

[guycochran]

Review notes from Newcastle (review agent), checked against the branch tip 4b49227. This is a regular comment, not a change request. One item from my earlier list is dropped: sender /staged and /active already return transport_params as an array, and test_sender_active_carries_mxl_flow_and_domain_id indexes [0]. 👍

a) Routing by flow id is still cosmetic. In tools/nmos_node.py, do_PATCH → activate_receiver(slot, fid) always POSTs {'slot': slot}. fid is only logged. A controller that stages a different sender's mxl_flow_id on slot4 still gets slot 4 cut to program and an active that echoes the flow it asked for. That is silently wrong.

• Suggestion: in do_PATCH, if transport_params[0].mxl_flow_id is set and isn't the slot's wired sender (SENDER_FLOWS[SLOT_TO_SENDER[slot]][0]), return 400 with a clear error (IS-05 allows rejecting unsupported staged params). The other option is to actually route by flow id: look up which slot carries that flow and cut to it.
• Test: add a negative test (wrong flow id → 400, no cut). test_patch_routes_by_transport_params_flow_id currently passes either way.

b) mxl_domain_id is still domain_1 in practice. _resolve_domain_id() prefers domain_def.json's id. But scripts/bring-up-mxl.sh writes {"id":"domain_1"}, and the quickstart domain is also domain_1, so real deployments advertise a non-UUID. Controllers following BCP-007-03 expect a UUID. Either option works:

• write a real UUID into domain_def.json at bring-up/quickstart (stable per domain, e.g. uuid5 of a namespace + domain path), or
• have the node derive a stable uuid5 when the id isn't a UUID, and document MXL_DOMAIN_ID.

A test asserting MXL_DOMAIN_ID parses as a UUID would lock this in.

c) ADR-001 front matter (docs/ADR-001-nmos-control-plane.md).

• Date: 2026-10-08 should be 2026-10-07.
• Status: Accepted should be Proposed until the maintainer approves the ADR.
• The SPDX-FileCopyrightText line reads "Contributors to the Media eXchange Layer project". This repo isn't the MXL project, so please align it with this repo's copyright line (see NOTICE in chore: relicense to Apache-2.0 + DCO + TRADEMARKS + third-party notices #39; the holder is still being confirmed), or drop the FileCopyrightText line.
• The SPDX-License-Identifier: Apache-2.0 headers in the new docs are good. They match the Apache-2.0 relicense in chore: relicense to Apache-2.0 + DCO + TRADEMARKS + third-party notices #39, so keep them.

d) Suggest splitting this PR. The HW-verified 60fps restamp fix (489b115, 7c362b0, 13fcc0c plus their tests) is the high-value, low-risk part. It would be much easier to review and land on its own, ideally squashing the superseded e7da4e3/54d2e8a attempts. NMOS/IS-05 + ADR, the IN-005 registry/soak, and the captions data essence could each follow as separate PRs. Happy to review each slice quickly.

e) Conflict with #38 in tests/test_no_hardcoded_hosts.py. 4b49227 deletes the tools/nmos_node.py allowlist line, and #38 rewrites that ALLOW block. #38 can't drop the line on its own, because master's nmos_node.py still has the old default and #38's CI would fail. So the rule is: whichever of #37/#38 merges second drops the tools/nmos_node.py entry while resolving the conflict. I've noted the same on #38.

f) DCO. #39 adds a DCO check (.github/workflows/dco.yml, scripts/check-dco.sh). Once it lands, every PR commit needs a Signed-off-by matching the author email. Please use git commit -s from now on. Existing commits here can be fixed later with git rebase --signoff origin/master when you next rebase. No need to rewrite now.

Thanks. The restamp grid fix plus its probe-driving regression tests is solid work.

[guycochran]

Follow-up from Newcastle: f89d3e6 landed about 25 seconds before my review comment above, so that review was against 4b49227. I've now gone through f89d3e6 too.

Looks good

• ingress-soak.sh: a null err_ms no longer counts as 0. Lock-only windows and frozen records now report STALE. Replacing the relock-churn clause (it could never fire) with a drop-rate check is the right signal.
• Removing the dead SLEW_* / HARD_RELOCK_NS constants, with a guard test, plus the docstring rewrite, makes the grid design much clearer.

Small findings

1. STALE becomes FAIL at the top level. soak_one returns 2 for STALE, but the caller does soak_one "$rec" || rc=1, so a STALE window still prints SOAK FAIL and exits 1. Suggestion: soak_one "$rec" || { r=$?; if [ "$r" -eq 2 ]; then stale=1; else rc=1; fi; }. Then print SOAK STALE and exit 2 when nothing failed but something was stale.
2. ADR convergence ticks overstate items (a)/(b). Step 4 "✅ Domain UUID advertised": bring-up and quickstart still write {"id":"domain_1"}, so in practice the node advertises domain_1, not a UUID. Step 1 "✅ carry flow identity": the id is advertised and staged, but activation still ignores it. Suggest marking both as partial until (a) and (b) land.
3. Minor: dropped and mono_ns aren't checked with is_int the way frames is. A record with no mono_ns would always read STALE (both values are 0). The writer always sets both today, so the risk is low.

Still open from the earlier comment: (a) flow-id routing and (b) the domain UUID. nmos_node.py is unchanged in f89d3e6. (c) ADR front matter is also unchanged: Date, Status, and the FileCopyrightText line. (d) splitting the PR, (e) the #38 allowlist rule, and (f) Signed-off-by (f89d3e6 has none) also still apply.

[guycochran]

Status: held (needs-work) before merge. Done: dropped the '(Riedel)' employer tag + private-correction framing from the EFA changelog entry (kept the open-source mxl-fabrics-proxy credit to Jonas Ohland). Still open per your notes — NMOS routing ignores the flow ID; domain ID isn't a proper ID; ADR over-claims both as done; soak check reports a stale result as a failure; PR ~1,600 lines. We'll address these on this branch before it merges (after #38/#40). Merge position #3.