Repository navigation
A hash written without a review reads as green: the backfill loophole #23
Description
Split out of #20, which documents it as a side-finding of the Intan port. It needs to outlive that issue: the loophole is general, it is how a whole MATLAB feature went missing here, and it very likely exists in NDI-python and DID-python too.
The gap
matlab_last_sync_hash is meant to record "I examined this version of this file." Nothing enforces that it means anything of the kind. Three failure modes, and after #22 only two are covered:
| An entry's hash is… | caught by | status |
|---|---|---|
| missing | test_every_entry_naming_a_matlab_path_carries_a_hash |
✅ #22 (f2efd66) |
| stale — MATLAB touched the file since | TestNoEntryHasDrifted |
✅ #22 (e2c3182) |
| written without anyone reading the diff | — | ❌ silent, and green |
The third is the worst of the three, because the other two are self-correcting: a red build gets fixed. This one asserts "reviewed and current" and nothing can contradict it.
It is not a "bump" — it is a first-time backfill
I assumed someone had bumped a stale hash to clear a red build. That is not what happened, and the real mechanism matters for choosing a fix.
Commit 6377973 "Make the bridge contract complete and change-detectable" (2026-08-28):
20 files changed, 1148 insertions(+), 0 deletions(-)
→ every file is a bridge YAML or PORT_STATUS.md
→ zero Python files touched
→ zero deletions: no hash was overwritten
It added matlab_last_sync_hash to entries that had none, recording what MATLAB's HEAD looked like for each file at that moment — not the commit whose content had actually been ported. Those are different claims, and the field only supports the first.
For read_Intan_RHD2000_header.m and Intan_RHD2000_blockinfo.m, MATLAB's then-current commit was a938988 — the commit that added multi-file recording support. So the backfill wrote "examined as of a938988" onto entries whose Python had never seen that change. Everything went green, and stayed green, and the feature is still absent today:
$ grep -rn "fileMode\|multiFile\|multifile" src/ndr/format/intan/ src/ndr/reader/intan_rhd.py
(no matches)
The near-miss makes it sharper. A later commit, a14513c, recorded the two new helpers from a938988 correctly, with a decision_log that even names the commit: "Landed in NDR-matlab a938988." So a938988 was read — and the two brand-new files got honest entries — while the three already-ported files it also changed kept their backfilled hashes and said nothing. Being already recorded is what hid them.
Why a backfill is the dangerous shape
A stale hash is one entry going wrong at a time. A backfill writes the same unverified claim across every entry at once — here, 1148 lines in a single commit — and it does it at exactly the moment the bridge looks most complete. #211's 147 hashless NDI entries are candidates for precisely this treatment, so whoever closes that gap could reproduce this at scale unless the rule lands first.
Proposed check
Preferred — diff-scoped, catches the real case. In CI, if a PR adds or changes matlab_last_sync_hash on an entry while touching no Python file for that entry, require the entry's decision_log to say why the MATLAB change is a no-op on the Python side. 6377973 touched zero Python files, so this would have flagged all 1148 lines of it.
Cheap to implement — it reads the PR diff and the YAML, needs no NDR-matlab checkout, and so can live in tests/test_matlab_bridge_conventions.py and run in every job (same placement argument as #211 decision 2). The awkward part is getting the diff in a test; a small git diff origin/main...HEAD helper is probably enough, with the check skipping when there is no merge-base.
Rejected — require a decision_log on every hashed entry. Would catch it, but demands 105 boilerplate justifications here and ~98 more in NDI, and directly contradicts #211 decision 4 (a regular port carries no decision_log obligation). Boilerplate would devalue the logs that carry real reasoning, which is the same argument that won decision 4.
Also worth considering, process not mechanism. Document in spec § 5 that a hash records an examination, not a snapshot — so a bulk backfill is only legitimate when someone actually read each diff. Cheap, and it names the mistake for the next person even if the mechanical check lands later.
Cross-repo
This is a candidate for NDI-python#211 rather than a purely local concern: the field, the drift rule and the decision_log convention are shared, so the guard should be too. Flagging here because NDR-python is where the evidence is. NDI-python and DID-python should check whether their own hashes arrived by backfill — the signature is a commit touching many bridge YAMLs and no source files.
Not blocking
#22 is green and merges independently; it narrows this gap without closing it. No urgency — the Intan divergence has been present since 2026-05-06 — but the rule is worth landing before anyone backfills NDI's 147.
Activity
stevevanhooser commentedon Sep 8, 2026
Design: three layers, and only one of them actually guards
@stevevanhooser asked the two questions that decide the shape of this: should CI report rather than fill the gap, and should the failure message tell the agent how to address it, so it doesn't just update the hash. Yes to both — but the second one cannot carry the weight alone, and it is worth writing down why before implementing.
The message currently green-lights the wrong fix
The drift failure says, verbatim, on
maintoday:We are telling the reader that bumping the hash is an acceptable remedy — and nothing checks that the note was ever written. To an agent optimising for a green build that is not a caution, it is permission, and it names the cheapest path to green. (The wording predates #22; it was carried over from the equals-latest message.)
Layer 1 — CI reports, never fixes
Already true, and it must stay true. An auto-fixer here would industrialise exactly the failure this issue describes: it would write "reviewed as of X" across every drifted entry without anyone reading a line.
6377973did that by hand 1148 times; a bot would do it on every push.Layer 2 — make the message a work order, not a diagnosis
Today it prints short hashes only.
cae4b00tells a reader nothing. It should print commit subjects, becausesettles in one glance whether the change is behavioural. Concretely:
<short hash> <subject>;git -C ../NDR-matlab diff <hash>..HEAD -- <matlab_path>to read it;Layer 3 — make the cheap path fail a different check
This is the only layer that guards, and the reason is worth being blunt about: prose does not survive contact with an agent that wants green. Layers 1 and 2 shape behaviour; layer 3 constrains it.
So: a commit that adds or changes
matlab_last_sync_hashon an entry while touching no Python file for that entry must also change that entry'sdecision_log, and the note must name the commit(s) it is accounting for. Bumping alone then trades one red check for another instead of reaching green.6377973touched zero Python files, so this catches the exact case in the issue body.The honest ceiling
No check can verify that anyone read a diff. What it can do is force the claim to be explicit, specific and attributable — written into the entry, naming the commit, visible in a diff a reviewer can see. That converts a silent omission into a signed statement someone can later be shown to have made.
6377973made that claim 1148 times without writing a word; under layer 3 it could not have.Worth stating rather than implying the check enforces comprehension. It does not, and a guard that oversells itself is its own hazard.
Implementation notes
All three land together — the mechanical check is what gives the message its teeth, and the message is what makes the check actionable rather than merely obstructive.
Layer 3 reads the PR diff plus the YAML and needs no NDR-matlab checkout, so it belongs in
tests/test_matlab_bridge_conventions.pyand runs in every job (same placement argument as NDI-python#211 decision 2). The awkward part is obtaining the diff from inside a test:git diff origin/main...HEADworks in CI, and the check should skip explicitly and loudly when there is no merge-base rather than passing silently — a check that cannot run must not report the same result as one that ran and passed, which is the rule this whole guard rests on.Starting on the PR now.
Generated by Claude Code
Say in each decision_log which commit was examined, per NDR-python#23
Read a multi-file Intan recording as one continuous stream (closes #20)