Publish the study the README's claim actually rests on - #489
Merged
Conversation
The generated block declared M4 while the paragraph above it stated an M5 result. That gap was filed rather than closed during the release, because rewriting the declaration between a CI failure and a tag is the careless publication the declaration exists to prevent. There is no tag pending now. Repointing it found a second thing. The block reported 1,240 runs where the registered analysis reports 1,160, because the generator counted rows on disk and M5 has seven shards of which three are re-runs. A study that re-runs a shard has more rows than measurements, and a line reporting the file count invites a reader to take the larger number for the study's size. The fix is in the generator rather than in the number. It now counts distinct task-arm-seed cells the way bench/m5-analysis.ts does, and says both figures with the difference named, so the published line and the registered analysis cannot disagree without one of them changing. The status note carries what the M4 note carried: why three shards are re-runs, where the supersession rule lives, the excluded error row, the unequal truncation and which direction it pushes, and that every record rendered `[claim]` so this is not the `[directive]` path 0.7.1 made reachable. M1, M1-b, M2 and M4 stay unpooled for the reason the M4 note already gave. Limit: the seven shards are declared individually, so a shard added later is invisible to this block until someone lists it -- which is the property the declaration was built for and the cost that comes with it Ruled-out: globbing bench/results for m5-*.jsonl | the file next to them is a withdrawn design log, and a glob is how the wrong dataset gets published without anyone deciding to Ruled-out: reporting only the 1,160 and dropping the row count | the rows are on disk and a reader who counts them would find the block understating; naming both and the reason is what makes either checkable Warn: the generator's supersession key is task, condition and seed, which is M5's cell shape; a study whose cell is keyed differently would silently collapse rows that are not duplicates Blast: local Undo: easy Certainty: firm Verified: 1,240 rows across the seven declared shards resolve to 1,160 distinct cells with 80 superseded, matching the registered analysis exactly; the gate passes with no --write and all four READMEs regenerate identically Provenance: authored Record-Id: r-m5sources
CommitLore — record lintTrailers: clean — 1 commit in Active constraints for the paths this PR touchesLimits (59)
Ruled out (137)
Warnings (32)
Trailer violations fail this check. Active constraints are informational — they are what the repository already decided, not a verdict on this PR. |
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #480. The gap I filed during the 0.7.0 release rather than fixing between a CI failure and a tag.
The block declared M4 while the paragraph above it stated M5
Repointing it found a second thing: the block reported 1,240 runs where the registered analysis reports 1,160.
The generator counted rows on disk. M5 has seven shards of which three are re-runs — 400 rows re-produced after a temp reaper, and one seed range re-run whole. A study that re-runs a shard has more rows than measurements, and a line reporting the file count invites a reader to take the larger number for the study's size.
The fix is in the generator, not the number
It now counts task-arm-seed cells the way
bench/m5-analysis.tsdoes, and says both figures with the difference named:The published line and the registered analysis cannot disagree now without one of them changing.
The status note carries what M4's carried
Why three shards are re-runs, where the supersession rule lives, the excluded
stopped_by: errorrow, the unequal truncation and which direction it pushes, and that every record rendered[claim]— so this is not the[directive]path 0.7.1 made reachable. M1, M1-b, M2 and M4 stay unpooled for the reason the M4 note already gave.Tests
An existing assertion caught the new field immediately and is updated with a comment saying why the field exists. A new case pins the behaviour: three rows, two cells,
recorded: 3anddistinct: 2.103 files, 2,278 cases pass. All four READMEs regenerate identically; the gate passes with no
--write.Stated limits
The seven shards are declared individually, so a shard added later is invisible until someone lists it — the property the declaration was built for, and its cost.
The supersession key is task, condition and seed, which is M5's cell shape. A study whose cell is keyed differently would silently collapse rows that are not duplicates.