fix(ORACLE-PIN-SURFACES): restate the a7c23ac96d pin, and re-anchor every claim the substitution made false - #3335
phantomic12 wants to merge 2 commits into
Conversation
c0e7e96 to
762a47c
Compare
localai-org-maint-bot
left a comment
There was a problem hiding this comment.
Source review at 762a47cfcd4a5785dca5d90f319ef78fd9635355: the oracle block still makes a machine-readable claim that the new prose explicitly withdraws.
.agents/oracles/vllm.md changes pin to a7c23ac96d... and evidence to the September 22 sync report while retaining gateable = yes. That report records this project's C++ library build and scheduler tests, not an upstream oracle build/model run. The added text agrees that no board has built this pin. AGENTS.md's gateability rule requires the oracle to demonstrably build and run a model; until then its record must say gateable = no and name the owing issue.
The prose caveat cannot enforce this: scripts/check-oracle-pins.py:314-327 accepts a yes record when its evidence path exists. Pointing it at the sync report therefore leaves the checker accepting an unmeasured oracle. Please change the structured disposition and reference the open issue that owes the run, or supply exact-pin build/model evidence before retaining yes. Keep the previous-pin measurements as historical evidence.
@mudler this should be resolved before merge. I read the sync report and checker path; I have not executed the Python checker or any oracle/device gate in this environment.
6957f29 to
fa49f61
Compare
|
Correct — and the point about the checker is the part that mattered most, because it means my prose caveat was not a caveat at all.
You are right that the checker would have kept accepting it. The owing issue is #3346, which I filed. It records the two gates the sync report actually contains — the Previous-pin measurements kept. Every One judgement call worth surfacing: the device-scoped gateability table has rows that were measured, and I did not demote them. A
One thing I did not do, since it is the same gate and a separate change: |
mudler-agent
left a comment
There was a problem hiding this comment.
Changes requested: this PR grows .agents/NOW.md to 104 lines and python3 scripts/check-now-current.py fails the enforced 100-line budget. The PR body also reports that gate as green. Move historical detail out of NOW and update the evidence.
fa49f61 to
9844006
Compare
…very claim the substitution made false The pin advanced in 4f11dfc (mudler#3320) from e126687a9a to a7c23ac96d and the parity-pin authority moved with it. check-oracle-pins went red on eleven sites and every one was the same defect: the oracle record copy and the five prose surfaces in PIN_SURFACES still carried the old revision. Moving the seven pin-bearing values turns that gate green and leaves the tree asserting what the advance did not do, so this change moves the surrounding claims too rather than only the spans: - NOW.md claimed a gate HAD run at the pin and PASSED. It ran at e126687a9a on 2026-09-04, and mudler#2817 is that advance's issue. It now records NO gate at the current pin, which is what FEATURES.md has been quoting it as saying. - oracles/vllm.md: the whole of "What this pin establishes" is measured at e126687a9a, so it is scoped to that pin explicitly; the device table's dgx row is relabelled the prior pin, the strix rows two pins back, the "exactly one row AT THE CURRENT PIN" paragraph is corrected to no rows (the 2026-09-22 advance added a pin and no measurement), and the residual's next row moves to the current pin. - The `oracle-pin` block's `gateable` field is now `no`, naming mudler#3346. It read `yes`, carried over from the pin that earned it, which made the STRUCTURED field assert a measurement that does not exist at this revision. The September 22 sync report it pointed at records this tree's own C++ -- the `vllm` library build and `test_scheduler` 48/48 -- not an oracle build or a model run, and the one PORT-NOW commit in the range was ported from source reading rather than from a capture. AGENTS.md requires `no` until the oracle demonstrably builds and runs a model, and every other ungateable oracle in `.agents/oracles/` already says so. The prose caveat this change previously relied on cannot enforce it: check-oracle-pins.py:314-327 accepts a `yes` whose `evidence` path merely EXISTS in the tree, so pointing that field at a sync report left the checker reading `yes` on an unmeasured pin. The device table's rows are unaffected and still outrank the file-level field for their own devices. - The three benchmark pages stop dating the advance 2026-09-03 and citing mudler#2817 for it, and stop claiming FlashInfer "moves to 0.6.18 at the new pin". The parity-pin block still reads 0.6.15.post1 and the 0.6.18 leg is refused by online_gate.py, so no published ratio has a 0.6.18 denominator. - how-we-measure.md's "no gate has run at the new pin" was stale in the other direction: mudler#2794 closed COMPLETED on 2026-09-06 with TOKENGATE_VERDICT PASS at the prior pin. Four of five goldens remain owed and are unanchored. - .agents/NOW.md's rewritten Current gate paragraph had grown to 104 lines, past the 100-line budget check-now-current.py enforces. It is condensed to the live position (96 lines): the job ID, the verdict fields and the per-item evidence it drops already live in oracles/vllm.md's NOT-established items, which the paragraph now points at instead of duplicating. check-oracle-pins, check-device-leakage, check-now-current and the rest of the doc gates are green. check-env-doc is red at this base, and was red at the base this branch was cut from: 891007a (2026-08-29) added VT_VK_DISABLE, VT_VK_DISABLE_PAGED_ATTN and VT_VK_FENCE_TIMEOUT_MS to src/vt/vulkan/vulkan_ops.cpp without documenting them, so the gate has been red on main since that commit landed -- upstream's own scheduled CI at fce3673 fails on the same three names -- and the repair belongs to the Vulkan row, not to this change. check-agent-record still reports the nine _intake archive-evidence errors, which are the CRLF checkout artefact and are fixed by row/ENG-EOL-BYTE-EXACT, not by this change. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:codebuff/buffy [freebuff]
9844006 to
31eb5e5
Compare
…9-26 advance date upstream/main landed its own pin restatement (gateable = yes with the sync report as evidence, advance dated 2026-09-26). This merge keeps the reviewed disposition -- gateable = no, evidence = mudler#3346, the issue that owes the exact-pin build -- and adopts upstream's advance date 2026-09-26 everywhere the branch said 2026-09-22. NOW.md keeps the shorter 'NO gate has run at it' wording and stays inside the 100-line budget. Resolved files: .agents/NOW.md, .agents/oracles/vllm.md, docs/FEATURES.md, docs/benchmarks/{how-we-measure,speculative-decoding, vllm-online-serving}.md. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:anthropic/devin [devin]
What
The vLLM parity pin advanced in
4f11dfc10(#3320) frome126687a9atoa7c23ac96d, and theparity-pinauthority in.agents/upstream-sync.mdmoved with it. Nothing else did.scripts/check-oracle-pins.pywent red with 11 errors, and every one of them was the same defect at a different site: the oracle-record copy and the five prose surfaces named inPIN_SURFACESstill carried the old revision.Moving the seven pin-bearing values turns that gate green — and leaves the tree asserting things the advance did not do. This PR moves the surrounding claims too, which is the actual defect.
Why the spans alone are the wrong fix
.agents/NOW.md:25a7c23ac96d... A gate HAS now run at it and it PASSED (2026-09-04)". No gate has run ata7c23ac96d; the capture was ate126687a9a, and#2817is that advance's issue, not this one's..agents/oracles/vllm.md"What this pin establishes"e126687a9a— thethorbuild block names that sha outright, item 2 dates the capture, item 3 owes step 6 there.dgx:gpu0row read "e126687a9a, the CURRENT pin", and a paragraph claimed the table holds "EXACTLY ONE ROW AT THE CURRENT PIN". Ata7c23ac96dit holds none.docs/benchmarks/*.mdx3#2817for it, and claimed FlashInfer "moves to0.6.18at the new pin" — while the authority block still reads0.6.15.post1andonline_gate.pyrefuses a0.6.18leg.What changed
oracles/vllm.md— an explicit lead scoping everythis pin/this revision/this onein the section toe126687a9a, naming the 2026-09-22 advance, its 1187/232-commit range, and the fact that its two recorded gates (vllmlibrary build,test_scheduler48/48) are this tree's own C++ rather than the oracle. The "prior pin" section is retitled "the two pins before it". The device table's rows are relabelled the prior pin / two pins back, the one-row paragraph is corrected to no rows, and the residual's "next row" moves toa7c23ac96d.NOW.md— records NO gate has run at it at the current pin, and re-anchors the passing OPT-125m capture toe126687a9a. This restores the literal sentencedocs/FEATURES.mdhas been quoting.docs/FEATURES.md— dates and issue references for both advances; drops "the prior pin555967922" (two advances back) and names#2817as the 2026-09-03 advance specifically.docs/benchmarks/how-we-measure.md— states that no oracle has been built at the current pin, that0.6.18/4.6.2are what thee126687a9aoracle was installed with, and that the0.6.18leg isREFUSED_AT_FLASHINFER. Its "no gate has run at the new pin" sentence was stale in the other direction —#2794closedCOMPLETEDon 2026-09-06 — and is corrected alongside what that capture did and did not close.docs/benchmarks/speculative-decoding.md,vllm-online-serving.md— same date/reference/FlashInfer corrections.Gates
Green on this branch:
check-oracle-pins(11 errors to 0),check-device-leakage,check-env-doc,check-now-current,check-symbol-anchors,check-benchmark-index,check-release-binary-contract,check-windows-release-state,check-commit-style,check-commit-trailers.check-agent-recordstill reports 9 errors, all of them_intake Frozen archive evidence must equal the declared line in the frozen archive source. Those are the CRLF-checkout artefact (core.autocrlf=truewith no line-ending policy in.gitattributes) and are fixed by #3333, not by this change. This PR is independent of it.No product code, no build input and no test is touched — every edit is a record or a public projection of a record.
Row:
ENG-RECORD-CLAIM-AGREEMENT. Filed asISSUE-LOCAL-01M3JM3HMK2F41A10P7T2WQXD1(mirror PENDING).🤖 Generated with Codebuff
FOLLOWING_AGENTS_PROTOCOL
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:codebuff/buffy [freebuff]