From ab1dfbb8e2b9562a407372b7dcbce3e63fe30fe1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ralph=20K=C3=BCpper?= Date: Wed, 12 Aug 2026 15:40:53 +0200 Subject: [PATCH 1/2] ci: put every Node the project chooses on 26, and assert it from one file .node-version (26.5.1) is the authoritative oracle, but the pin has leaked twice since #6367 made it single: CLAUDE.md's prose drifted off the file (#7599), and npm-launcher.yml was created by #6350 on the SAME DAY #6367 converted every existing workflow, keeping that day's ambient "22.23.1" literal by omission rather than by decision. - npm-launcher.yml (x2): "22.23.1" -> node-version-file: .node-version. It runs npm/perry/test/detect.test.cjs, which exercises the shipped launcher logic every installing user hits, so its Node is a behavioural input. Safe on the ubuntu-22.04 job: Node 26 needs glibc >= 2.28 and jammy has 2.35. - release-packages.yml: "20" -> "26". Node 20 reached EOL on 2026-04-30 and this is the repo's most privileged job (id-token: write, OIDC-publishes every platform package). - release-hono-server.yml: "24" -> "26". Both release workflows stay pinned to a bare MAJOR rather than node-version-file: they are publishing toolchains, and a gap-suite oracle bump must never be able to move the runtime that publishes releases. New scripts/check_node_version_consistency.py, wired as a lint step (a required context). It re-derives every restatement of a Node version from the file it quotes, and requires every literal node-version: in a workflow to be a registered exemption with a reason. Exemptions are asserted against the tree, so one that stops matching FAILS and must be updated or deleted. Reverting npm-launcher.yml to "22.23.1" reproduces the historical bug as two named failures. --self-test proves each rule can fail; both vacuity floors can fail too. Not changed, deliberately: test-compat/node-core/pinned-version.txt (v22.x runs Node's own corpus) and benchmarks/public-baseline-config.json (v22.23.1). The latter is in public_baseline.HARNESS_PATHS, so editing it alone reddens the required lint job -- measured, ci_public_baseline_check.py exits 2 with "benchmark harness changed". The pin and its ~2 h measurement are atomic by design (#7282/#7958) and the regeneration needs the quiet M1 mini, so it is registered as a self-clearing exemption carrying the runbook instead. Node 26 is faster than Node 22, so that regeneration is expected to reduce Perry's published advantage. Also: CLAUDE.md said "Two workflows are deliberately exempt" and then listed three, which is probably why the fourth pin read as accounted-for; and external-tools.json told readers to bump a NODE_PIN constant that does not exist in node_compat_matrix.mjs (it reads external-tools.json). Claude-Session: https://claude.ai/code/session_012B8z92S82sCfqCrVqrFgS2 --- .github/workflows/npm-launcher.yml | 14 +- .github/workflows/release-hono-server.yml | 10 +- .github/workflows/release-packages.yml | 19 +- .github/workflows/test.yml | 15 + CLAUDE.md | 4 +- external-tools.json | 2 +- gc-handoff/NODE26-NOTES.md | 190 +++++++++ scripts/check_node_version_consistency.py | 478 ++++++++++++++++++++++ 8 files changed, 722 insertions(+), 10 deletions(-) create mode 100644 gc-handoff/NODE26-NOTES.md create mode 100755 scripts/check_node_version_consistency.py diff --git a/.github/workflows/npm-launcher.yml b/.github/workflows/npm-launcher.yml index d35ab59c87..5d2769dee0 100644 --- a/.github/workflows/npm-launcher.yml +++ b/.github/workflows/npm-launcher.yml @@ -31,9 +31,15 @@ jobs: - uses: actions/checkout@v7 with: persist-credentials: false + # Single source of truth: .node-version at the repo root. detect.cjs is + # shipped JS that every installing user executes, so the Node it is + # exercised under is a behavioural input, not a toolchain detail. This + # file was created by #6350 on the same day #6367 converted every other + # workflow to the shared pin, so it kept that day's ambient "22.23.1" + # literal by omission rather than by decision. - uses: actions/setup-node@v7 with: - node-version: "22.23.1" + node-version-file: .node-version - run: node npm/perry/test/detect.test.cjs - name: install.sh is valid POSIX sh run: sh -n packaging/install.sh @@ -50,9 +56,13 @@ jobs: - uses: actions/checkout@v7 with: persist-credentials: false + # Single source of truth: .node-version. Safe on this image — Node 26 + # requires glibc >= 2.28 (nodejs/node v26.x BUILDING.md, Tier 1 linux + # x64/arm64) and jammy ships 2.35. The glibc under test here is the one + # Perry's own prebuilt binary needs, not Node's. - uses: actions/setup-node@v7 with: - node-version: "22.23.1" + node-version-file: .node-version - name: Host glibc run: | diff --git a/.github/workflows/release-hono-server.yml b/.github/workflows/release-hono-server.yml index f8e126ae20..e8276d89d4 100644 --- a/.github/workflows/release-hono-server.yml +++ b/.github/workflows/release-hono-server.yml @@ -29,14 +29,20 @@ jobs: # DELIBERATE EXEMPTION from the repo-wide .node-version pin: this Node is a # *publishing* toolchain, not a test oracle — it never runs a parity diff. # - # Node 24 ships npm 11.x. OIDC Trusted Publishing needs npm >= 11.5.1; + # Node 26 ships npm 11.x. OIDC Trusted Publishing needs npm >= 11.5.1; # Node 22's npm 10.x silently fails the OIDC handshake and the registry # returns a misleading 404 on PUT. Do NOT set `registry-url` here — it # writes a token-based .npmrc that contradicts OIDC (the publish targets # the default registry.npmjs.org regardless). + # + # Pinned to a MAJOR literal on purpose, not `node-version-file`: an oracle + # bump made for gap-suite reasons must never be able to move the runtime + # that publishes releases. Registered in + # scripts/check_node_version_consistency.py, which keeps this major in + # step with .node-version's major. Was "24" until the Node-26 sweep. - uses: actions/setup-node@v7 with: - node-version: "24" + node-version: "26" - name: Upgrade npm (OIDC Trusted Publisher requires >= 11.5.1) run: npm install -g npm@latest diff --git a/.github/workflows/release-packages.yml b/.github/workflows/release-packages.yml index 6715424e92..6bb9b5dcf2 100644 --- a/.github/workflows/release-packages.yml +++ b/.github/workflows/release-packages.yml @@ -2034,15 +2034,26 @@ jobs: # DELIBERATE EXEMPTION from the repo-wide .node-version pin: this Node is a # *publishing* toolchain (npm registry auth), not a test oracle — it never # runs a parity diff, so the gap suite's Node version is irrelevant here. + # Pinned to a MAJOR literal on purpose, not `node-version-file`: an oracle + # bump made for gap-suite reasons must never be able to move the runtime + # that publishes releases. Registered in + # scripts/check_node_version_consistency.py, which keeps this major in + # step with .node-version's major. + # + # Was "20" until the Node-26 sweep. Node 20 reached end-of-life on + # 2026-04-30, and this is the repo's most privileged job (`id-token: write`, + # OIDC-publishes every platform package) — an EOL runtime is a defect there. - uses: actions/setup-node@v7 with: - node-version: "20" + node-version: "26" registry-url: "https://registry.npmjs.org" - name: Upgrade npm (OIDC Trusted Publisher requires >= 11.5.1) - # Node 20 ships npm 10.x; versions below 11.5.1 silently fail the OIDC - # handshake and the registry returns a misleading 404 ("'pkg@ver' is - # not in this registry") on PUT. See npm/cli#9088. + # Node 26 already ships npm 11.x, which clears the floor; kept because + # it is free and pins the guarantee rather than inheriting it. Below + # 11.5.1 the OIDC handshake fails silently and the registry returns a + # misleading 404 ("'pkg@ver' is not in this registry") on PUT. See + # npm/cli#9088. run: npm install -g npm@latest - name: Download all build artifacts diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 598fb01893..49babe9233 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -258,6 +258,21 @@ jobs: python3 scripts/check_gc_doc_claims.py --self-test python3 scripts/check_gc_doc_claims.py + # Node is a correctness input (see CLAUDE.md "TypeScript Parity Status"): + # an oracle that cannot run a gap test drops it from the gate instead of + # failing it. #6367 made `.node-version` the single pin, and the pin has + # leaked twice since -- CLAUDE.md's prose drifted off the file (#7599), and + # npm-launcher.yml was created the same day as #6367 and kept that day's + # ambient "22.23.1" literal by omission. This re-derives every restatement + # from the file it quotes and requires every literal `node-version:` in a + # workflow to be a registered exemption with a reason. Build-free, so it + # belongs in `lint`, which IS a required context. + - name: Node version consistency + if: ${{ !cancelled() }} + run: | + python3 scripts/check_node_version_consistency.py --self-test + python3 scripts/check_node_version_consistency.py + # #7341 layer 3. A RuntimeHandleScope gives an object liveness; it does # nothing for a raw pointer already read out of the slot. Every rooting bug # in the quarantine sweep had rooting ALREADY -- what was missing was diff --git a/CLAUDE.md b/CLAUDE.md index 35c7c72c78..7a4972faea 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -17,7 +17,9 @@ Tracked via the gap test suite (`test-files/test_gap_*.ts`). Compared byte-for-b **The oracle is Node `26.5.1`, pinned in `.node-version` at the repo root** (the FILE is authoritative — this prose drifted once, #7599) — the single source of truth every CI workflow reads via `setup-node`'s `node-version-file`. **Run the gap suite against that exact version locally**, or your results won't match CI. The version is a *correctness input*, not an incidental toolchain detail: when node can't run a test (a feature newer than the pinned node), node exits non-zero, the harness classifies it `node_fail`, and the test is **silently dropped from the gate** rather than going red. CI sat on Node 22 while the suite grew Node 24/26 features, which hid 14 tests — all of Temporal, plus DisposableStack, Float16Array, and `Uint8Array` base64/hex (#6364). Node patch releases also change observable output (error-message text, `v8` heap fields), which is why the pin is exact. Raising it is a deliberate act: measure the failure delta under both oracles first, then triage what it exposes. -Two workflows are deliberately exempt and say so inline: `node-core-subset.yml` derives its Node from `test-compat/node-core/pinned-version.txt` (it runs Node's *own* test corpus, which must match its own Node line), and the two release workflows use Node purely as an npm *publishing* toolchain. +**Don't audit this by reading prose — run `python3 scripts/check_node_version_consistency.py --list`** (it is a `lint` step, so it is a required check). It re-derives every restatement of a Node version from the file that restatement quotes, and requires every literal `node-version:` in a workflow to be a *registered* exemption carrying a reason. That registry is the authoritative exemption list; the three below are what it currently holds, and an entry that stops matching the tree fails, so a bump must update or delete its own exemption. + +Three workflows are deliberately exempt and say so inline: `node-core-subset.yml` derives its Node from `test-compat/node-core/pinned-version.txt` (it runs Node's *own* test corpus, which must match its own Node line), and the two release workflows (`release-packages.yml`, `release-hono-server.yml`, both Node 26) use Node purely as an npm *publishing* toolchain — pinned to a bare major on purpose, so a gap-suite oracle bump can never move the runtime that publishes releases. A fourth file, `benchmarks/public-baseline-config.json`, still pins **Node v22.23.1** for the *published* performance comparison: it is in `public_baseline.HARNESS_PATHS`, so editing it invalidates the committed artifact and reddens `lint` until the baseline is regenerated (~2 h on the quiet mini). The pin and the measurement are atomic by design — see the exemption's own text for the runbook. **Last full sweep:** run `./run_parity_tests.sh` for the current snapshot. The umbrella tracker is #793 (Node.js + TypeScript compatibility roadmap); the previously-cited #447–#452 batch closed on 2026-05-04. Currently-open trackers worth knowing about: diff --git a/external-tools.json b/external-tools.json index b12d3a5f72..f512326849 100644 --- a/external-tools.json +++ b/external-tools.json @@ -7,7 +7,7 @@ "repository": "github:nodejs/node", "distBaseUrl": "https://nodejs.org/dist", "notes": [ - "Latest CURRENT stable at pin time. Bump via the NODE_PIN.version constant in scripts/node_compat_matrix.mjs, then run --update-baseline and review the diff.", + "Latest CURRENT stable at pin time. Bump by editing tools.node.version here (plus each platform asset name and its sha512 SRI), then run scripts/node_compat_matrix.mjs --update-baseline and review the diff. There is no NODE_PIN constant in that script — it reads this file; an earlier revision of this note said otherwise.", "Assets are the official nodejs.org dist tarballs. Integrity is sha512 recomputed from the downloaded bytes (matching this file's sha512 SRI convention); nodejs.org also publishes sha256 in SHASUMS256.txt, cross-checked at pin time against every asset here.", "Installed by scripts/node_compat_matrix.mjs via its own resolver (download + SRI-verify + cache under .cache/node-pin/), NOT the shared tool rack: nodejs.org is not a github release host, and the tarball is a full node tree rather than a single bin.", "Official dist ships no musl build (musl lives on unofficial-builds.nodejs.org); alpine/musl runners fall back to a system node.", diff --git a/gc-handoff/NODE26-NOTES.md b/gc-handoff/NODE26-NOTES.md new file mode 100644 index 0000000000..e0ddef6da3 --- /dev/null +++ b/gc-handoff/NODE26-NOTES.md @@ -0,0 +1,190 @@ +# Node 26 everywhere — inventory, verdicts, and what must be regenerated + +Status: IN PROGRESS (written incrementally; see "Open" at the bottom). + +Owner directive: "We want Node 26 everywhere." Project oracle is **26.5.1** +(`.node-version`, authoritative). + +## Verified inventory (all read from `git show origin/main:`, not a working tree) + +| location | value on origin/main | verdict | +|---|---|---| +| `.node-version` | `26.5.1` | correct, authoritative | +| `external-tools.json` `tools.node.version` | `26.5.1` | correct | +| `benchmarks/public-baseline-config.json` `toolchains.node` | `v22.23.1` | TBD (main target) | +| `test-compat/node-core/pinned-version.txt` | `v22.x` | LEAVE — Node's own corpus must match its own line | +| `.github/workflows/npm-launcher.yml` | `"22.23.1"` x2 | TBD | +| `.github/workflows/release-hono-server.yml` | `"24"` | TBD | +| `.github/workflows/release-packages.yml` | `"20"` | TBD | + +Correction to the handed-down inventory: `npm-launcher.yml` is **not** a bare +`22` — it is `node-version: "22.23.1"` in **two** jobs (lines 36 and 55). + +## Notes log + +### FINDING 1 (load-bearing) — the benchmark config cannot be changed on its own + +`benchmarks/public-baseline-config.json` is listed in `public_baseline.py`'s +`HARNESS_PATHS`, so its bytes feed `freshness.harness_fingerprint`. Two +independent hard checks in `validate_public()` fire on a node-pin edit: + +1. `harness_fingerprint` mismatch -> `"public artifact benchmark harness changed; + regenerate it with ./benchmarks/run_public_baseline.sh"`. +2. `for runtime, expected_version in measurement_config["toolchains"].items()` + compares the config against `artifact["runtimes"][runtime]["version"]`, which + is baked as `v22.23.1` in `benchmarks/results/public-node-bun-v1.json`. No + migration tuple can bypass this one. + +Measured, not assumed (edit applied, checker run, file restored in one step): + +``` +$ (node pin -> v26.5.1) python3 benchmarks/ci_public_baseline_check.py +EXIT: 2 +public baseline error: public artifact benchmark harness changed; +regenerate it with ./benchmarks/run_public_baseline.sh +``` +Unmodified tree: exit 0. + +`benchmarks/ci_public_baseline_check.py` runs as the "Public benchmark evidence +freshness" step of the **`lint`** job (`.github/workflows/test.yml:88` job, +step at :137). `lint` is a REQUIRED context (CLAUDE.md "Workflow Requirements"). + +=> Editing the node pin without regenerating turns a required gate red for main +and for every open PR. The config change and the regeneration are **atomic by +design** (#7282/#7958). Shipping the pin edit alone is not "stale but honest", +it is a broken required gate — the exact failure CLAUDE.md's GATE THEATRE note +warns about. + +The artifact also currently passes only via `_HARNESS_FINGERPRINT_MIGRATION` +(it records the pre-#7282 digest `28117b86...`), so there is no spare slack. + +### FINDING 2 — the unaccounted-for workflow is `npm-launcher.yml`, and it is drift + +`.github/workflows/npm-launcher.yml` pins `node-version: "22.23.1"` twice +(lines 36, 55) with **no exemption comment**. Provenance: + +* `4f50d797c` (**2026-07-13**) created the file with the literal `22.23.1` — + PR #6350, the glibc-2.35 launcher fix for #6298. +* `db4b068d8` (**2026-07-13**, the SAME DAY) is #6367, the standardization that + introduced `.node-version` (26.5.0) and converted every workflow to + `node-version-file:`. Its commit message enumerates the exemptions it granted: + `node-core-subset.yml`, `release-hono-server.yml` (24), `release-packages.yml` + (20). **`npm-launcher.yml` is not among them** — it did not exist when #6367 + was written and landed alongside it. + +So `22.23.1` is not a decision; it is the ambient Node of the day, frozen. It is +also exactly the value still sitting in `benchmarks/public-baseline-config.json`, +which is the same fossil. + +Behaviour-sensitivity: the `detect-self-test` job runs +`node npm/perry/test/detect.test.cjs`, which exercises `npm/perry/bin/detect.cjs` +— the platform-resolution logic every installing user hits. That is JS behaviour +under test, not a publishing toolchain. It should track the oracle. + +Blocker check for the `ubuntu-22.04` job (glibc 2.35): Node 26 requires +**glibc >= 2.28** (nodejs/node `v26.x` BUILDING.md support matrix, Tier 1 for +linux x64 and arm64), so 2.35 is comfortably above the floor. No blocker. + +VERDICT: both occurrences -> `node-version-file: .node-version`. + +### FINDING 3 — the two release workflows + +Node release schedule (nodejs/Release `schedule.json`, fetched): +`v20` EOL **2026-04-30** (today is 2026-08-12 -> EOL), `v22` EOL 2027-04-30, +`v24` EOL 2028-04-30, `v26` start 2026-05-05, LTS 2026-10-28, EOL 2029-04-30. + +* `release-packages.yml` = `"20"`: an **end-of-life** runtime in the repo's most + privileged job (`id-token: write`, OIDC-publishes every platform package). + That is a defect independent of the directive. -> **26**. +* `release-hono-server.yml` = `"24"`: supported, chosen for npm >= 11.5.1. + Bumping is consistency only, and is safe (Node 26 ships npm 11.x). -> **26**. + +Both keep their `npm install -g npm@latest` step (Node 26 already satisfies the +OIDC floor, but the step is free insurance) and both keep an exemption comment: +they are pinned to a **major literal**, deliberately NOT `node-version-file`, +because a gap-suite oracle bump must not be able to move a publishing toolchain. + +### FINDING 4 — stale instruction inside `external-tools.json` + +`tools.node.notes[0]` says "Bump via the NODE_PIN.version constant in +scripts/node_compat_matrix.mjs". There is **no `NODE_PIN` constant** in that +file (grep: zero hits); the script reads `external-tools.json` itself and its +own `--help` says so ("The pinned Node version lives in external-tools.json"). +CLAUDE.md gives the correct instruction. Fixed the note in place. + +### FINDING 5 — CLAUDE.md said "Two workflows are deliberately exempt", then listed three + +Verbatim on origin/main (line 20): *"**Two** workflows are deliberately exempt +and say so inline: `node-core-subset.yml` ... and **the two release +workflows**"* — 1 + 2 = 3. That undercount is the most likely reason the fourth +pin (`npm-launcher.yml`) read as accounted-for when it never was. Corrected, and +the prose now points at the checker instead of trying to be the registry. + +### What was changed, and what was not + +CHANGED +* `.github/workflows/npm-launcher.yml` (x2): `"22.23.1"` -> `node-version-file: .node-version`. +* `.github/workflows/release-packages.yml`: `"20"` -> `"26"` (Node 20 is EOL). +* `.github/workflows/release-hono-server.yml`: `"24"` -> `"26"`. +* `external-tools.json`: removed the stale "NODE_PIN constant" bump instruction. +* `CLAUDE.md`: "Two" -> "Three"; points at the checker; states the benchmark pin. +* NEW `scripts/check_node_version_consistency.py` + a `lint` step in `test.yml`. + +LEFT ALONE, with reasons +* `.node-version` (26.5.1), `external-tools.json` (26.5.1) — already correct. +* `test-compat/node-core/pinned-version.txt` (`v22.x`) — Node's own corpus must + be run by its own line. Registered exemption. +* `benchmarks/public-baseline-config.json` (`v22.23.1`) — see FINDING 1: not + editable without a 2 h regeneration, which this host cannot perform. + Registered as a self-clearing exemption. +* `benchmarks/results/public-node-bun-v1.json`, + `benchmarks/honest_bench/results/metadata.json` — measurement OUTPUTS, already + tied to the config by `public_baseline.validate_public`. A second checker over + them would be a place for the two policies to disagree. +* `bun: 1.3.14` in the same config — OUT OF SCOPE per the brief, and it is *not* + equally stale: 1.3.x is the current Bun line (Node 22 is four majors behind; + Bun 1.3 is not). It is frozen by the same atomic-regeneration rule, so if the + baseline is regenerated it should be re-pinned to current Bun in that same run. +* `package-lock.json` / vendored fixture lockfiles (`"node": ">=20"` etc.) — + dependency `engines` floors declared by third-party packages, not pins Perry + chooses. Changing them would be fiction. + +### Does the published comparison move against us? + +Direction: **yes, against us.** Node 26 ships a materially newer V8 than Node +22, so Perry's published "x times faster than Node" ratios should shrink. That +is the honest direction and the reason to do it. + +Magnitude: **not measurable on this host, and deliberately not guessed.** See +the regeneration section — the artifact's own policy (<=25% CPU active for 60 +consecutive seconds, re-checked before each of five components) is the project's +own statement of what a trustworthy number costs, and this box has run at load +30-200 all day with several concurrent agent builds. Any number produced here +would be junk wearing a decimal point. + +### Regeneration runbook (what the next person must do) + +Host: `perry@perry-macos.local` — the Apple M1 / 8-core / 8 GB mini recorded in +the artifact's `host` block. It must be that machine: the artifact pins host +identity, so regenerating elsewhere replaces the baseline rather than updating +it, and no before/after ratio would be comparable. + +1. Quiesce the mini (no agent builds, no other benchmarks). +2. Install Node 26.5.1 and confirm `node --version` matches `.node-version`. +3. Edit `benchmarks/public-baseline-config.json`: `"node": "v26.5.1"` + (and consider re-pinning `bun` in the same run — same atomicity applies). +4. `./benchmarks/run_public_baseline.sh` (~2 h; five components, each gated on + the quiet-host check). +5. `python3 benchmarks/ci_public_baseline_check.py` must print OK. +6. Delete the `benchmarks/public-baseline-config.json` entry from + `scripts/check_node_version_consistency.py` — leaving it stale FAILS the + check, which is what makes this exemption self-clearing rather than a fossil. +7. Expect the headline ratios to drop. Publish the new ones. + +### Incidental: do not run `scripts/check_gate_freshness.py` locally + +It is not a read-only checker — it calls the GitHub API and opens/updates a +sticky issue. Running it during this task opened +https://github.com/PerryTS/perry/issues/7966 ("CI gate freshness alert: 11 +gate(s)..."). The content is accurate and the issue is self-closing and updated +in place by `gate-freshness.yml`, so it was left open rather than suppressed. diff --git a/scripts/check_node_version_consistency.py b/scripts/check_node_version_consistency.py new file mode 100755 index 0000000000..c79df1cda0 --- /dev/null +++ b/scripts/check_node_version_consistency.py @@ -0,0 +1,478 @@ +#!/usr/bin/env python3 +"""Hold every Node version in the tree to a source a machine can re-derive. + +Node is a *correctness input* here, not a toolchain detail (CLAUDE.md, +"TypeScript Parity Status"): when the oracle cannot run a gap test, node exits +non-zero, the harness classifies it `node_fail`, and the test is silently +dropped from the gate instead of going red. CI sat on Node 22 while the suite +grew Node 24/26 features and hid 14 tests that way (#6364). + +The defence #6367 chose was a single `.node-version` pin every workflow reads +through `setup-node`'s `node-version-file`. That defence has leaked twice, in +the two shapes this checker owns one rule for each of: + +1. **Prose drifting off the file.** CLAUDE.md's stated oracle version drifted + from `.node-version` (#7599) and had to be corrected by hand. A restatement + of a version is now a marker compared against the file it claims to quote, + so the next drift is a red build rather than a reader's problem. + +2. **A new workflow born outside the sweep.** `npm-launcher.yml` was created + (#6350) on the same day #6367 converted every *existing* workflow, and kept + that day's ambient `"22.23.1"` literal for a month — not a decision, an + omission. Every `node-version:` literal in `.github/workflows/` must now be + registered here with a reason, so the next one cannot arrive silently. + +WHAT THIS DOES NOT CATCH (per CLAUDE.md's gate rules, said plainly) +------------------------------------------------------------------ +Whether the pinned version is the *right* one. Nothing here re-runs the gap +suite under two oracles or measures a benchmark; raising a pin stays the +deliberate act CLAUDE.md describes. This checker only guarantees that every +place naming a Node version either derives it from the authoritative file or +carries a written, still-accurate exemption. + +It also does not police measurement *outputs* +(`benchmarks/results/public-node-bun-v1.json`, +`benchmarks/honest_bench/results/metadata.json`). Those are already tied to +`benchmarks/public-baseline-config.json` by `public_baseline.validate_public`, +and a second implementation of that policy would be a place for the two to +disagree. + +Usage: + python3 scripts/check_node_version_consistency.py # check + python3 scripts/check_node_version_consistency.py --self-test # check me + python3 scripts/check_node_version_consistency.py --list # describe +""" + +from __future__ import annotations + +import argparse +import json +import re +import sys +from pathlib import Path +from typing import Callable + +REPO = Path(__file__).resolve().parent.parent + +# The two independent Node pins this repo maintains, and what each one is for. +# They are equal today and are NOT required to be: CLAUDE.md documents the +# compat-matrix oracle as "independent of the .node-version gap-suite oracle", +# so asserting equality would encode a coupling the project deliberately does +# not have. +GAP_ORACLE = ".node-version" +MATRIX_ORACLE = "external-tools.json" + +SEMVER = re.compile(r"^\d+\.\d+\.\d+$") + +# --------------------------------------------------------------------------- +# Rule 1 -- restatements of a pin must equal the file they claim to quote. +# +# Each entry is (path, regex with exactly one capture group, source key). The +# regex MUST match at least once: a rewording that loses the marker fails here +# rather than quietly ceasing to be checked, which is the whole point. +# --------------------------------------------------------------------------- +MIRRORS: tuple[tuple[str, str, str], ...] = ( + ( + "CLAUDE.md", + r"The oracle is Node `(\d+\.\d+\.\d+)`", + "gap", + ), + ( + "CLAUDE.md", + r"`tools\.node\.version`\s*[—-]\s*currently \*\*(\d+\.\d+\.\d+)\*\*", + "matrix", + ), + ( + "llms.txt", + r"pinned Node oracle \((\d+\.\d+\.\d+)\)", + "matrix", + ), + ( + "test-parity/node-compat-matrix.baseline.json", + r'"nodeVersion":\s*"(\d+\.\d+\.\d+)"', + "matrix", + ), +) + +# Vacuity floors. A scan that silently matched nothing is the failure mode this +# repo has paid for most often (CLAUDE.md, "Four ways a gate can be unable to +# fail", item 4), so both populations have a floor that must itself be able to +# fail -- see self_test(). +MIN_MIRRORS = 4 +MIN_WORKFLOW_PINS = 20 + + +# --------------------------------------------------------------------------- +# Rule 2/3 -- every literal Node version outside the authoritative files is a +# registered exemption with a reason, an expected value, and a statement about +# whether its major tracks the oracle. +# +# `value` is asserted against the tree. An entry that no longer matches FAILS, +# so a future bump or baseline regeneration must update or delete its own +# exemption instead of leaving a fossil behind (CLAUDE.md's allowlist rule: +# "an entry that matches nothing FAILS, so a fix must delete its entry"). +# --------------------------------------------------------------------------- +class Exemption: + def __init__( + self, + path: str, + value: str, + major_tracks_oracle: bool, + reason: str, + locator: Callable[[str], list[str]] | None = None, + ) -> None: + self.path = path + self.value = value + self.major_tracks_oracle = major_tracks_oracle + self.reason = reason + self.locator = locator + + +def _workflow_pins(text: str) -> list[str]: + return [value for _, key, value in _node_version_keys(text) if key == "node-version"] + + +EXEMPTIONS: tuple[Exemption, ...] = ( + Exemption( + path=".github/workflows/node-core-subset.yml", + value="${{ steps.node_core_version.outputs.version }}", + major_tracks_oracle=False, + reason=( + "Runs Node's OWN test corpus, which must be executed by the Node line " + "it was taken from. Derived from test-compat/node-core/pinned-version.txt " + "so the coupling is explicit instead of looking like drift (#6367)." + ), + locator=_workflow_pins, + ), + Exemption( + path="test-compat/node-core/pinned-version.txt", + value="v22.x", + major_tracks_oracle=False, + reason=( + "The ref of the vendored node-core test corpus. Bumping it means " + "re-vendoring that corpus, not chasing the gap-suite oracle." + ), + ), + Exemption( + path=".github/workflows/release-packages.yml", + value="26", + major_tracks_oracle=True, + reason=( + "npm *publishing* toolchain (OIDC registry auth), never a test oracle. " + "Pinned to a major literal rather than node-version-file so a gap-suite " + "oracle bump cannot move the runtime that publishes releases." + ), + locator=_workflow_pins, + ), + Exemption( + path=".github/workflows/release-hono-server.yml", + value="26", + major_tracks_oracle=True, + reason=( + "npm *publishing* toolchain for @perryts/hono-server, never a test " + "oracle. Major literal for the same reason as release-packages.yml." + ), + locator=_workflow_pins, + ), + Exemption( + path="benchmarks/public-baseline-config.json", + value="v22.23.1", + major_tracks_oracle=False, + reason=( + "PUBLISHED PERFORMANCE BASELINE -- KNOWN STALE, AND NOT EDITABLE ALONE. " + "This file is in public_baseline.HARNESS_PATHS, so its bytes feed the " + "artifact's harness_fingerprint; and validate_public() separately " + "compares this value against benchmarks/results/public-node-bun-v1.json's " + "recorded runtime version. Changing it without regenerating turns the " + "REQUIRED lint job red (measured: ci_public_baseline_check.py exits 2). " + "The pin and the measurement are therefore atomic by design (#7282/#7958). " + "To clear this exemption: on the quiet M1 mini (perry@perry-macos.local, " + "the host recorded in the artifact), set toolchains.node to the oracle and " + "run ./benchmarks/run_public_baseline.sh -- ~2h, and it enforces " + "<=25% CPU active for 60 consecutive seconds before each of five " + "components. Then delete this entry; leaving it stale fails this check." + ), + locator=lambda text: [json.loads(text)["toolchains"]["node"]], + ), +) + + +# --------------------------------------------------------------------------- +# Scanning +# --------------------------------------------------------------------------- +_NODE_KEY = re.compile(r"^\s*(node-version|node-version-file):\s*(\S.*?)\s*$") + + +def _node_version_keys(text: str) -> list[tuple[int, str, str]]: + """Every setup-node version input in a workflow, as (line, key, value). + + Comment lines are skipped so prose like `# Single source of truth: + .node-version` cannot be read as a pin. + """ + out: list[tuple[int, str, str]] = [] + for lineno, raw in enumerate(text.splitlines(), 1): + if raw.lstrip().startswith("#"): + continue + m = _NODE_KEY.match(raw) + if m: + out.append((lineno, m.group(1), m.group(2).strip().strip('"').strip("'"))) + return out + + +def _read(path: str) -> str: + return (REPO / path).read_text(encoding="utf-8") + + +def _sources() -> dict[str, str]: + gap = _read(GAP_ORACLE).strip() + matrix = json.loads(_read(MATRIX_ORACLE))["tools"]["node"]["version"] + return {"gap": gap, "matrix": matrix} + + +def check( + sources: dict[str, str] | None = None, + mirrors: tuple[tuple[str, str, str], ...] = MIRRORS, + exemptions: tuple[Exemption, ...] = EXEMPTIONS, + reader: Callable[[str], str] = _read, + min_mirrors: int = MIN_MIRRORS, + min_workflow_pins: int = MIN_WORKFLOW_PINS, + workflows: list[Path] | None = None, +) -> list[str]: + failures: list[str] = [] + if sources is None: + sources = _sources() + + for key, value in sources.items(): + if not SEMVER.match(value): + failures.append(f"{key} oracle is not a bare X.Y.Z version: {value!r}") + + # Rule 1: restatements. + mirror_hits = 0 + for path, pattern, source in mirrors: + found = re.findall(pattern, reader(path)) + if not found: + failures.append( + f"{path}: marker {pattern!r} matched nothing -- the prose was " + f"reworded out from under this check; restore the marker or update it" + ) + continue + for value in found: + mirror_hits += 1 + if value != sources[source]: + failures.append( + f"{path}: states Node {value}, but the {source} pin is " + f"{sources[source]}" + ) + if mirror_hits < min_mirrors: + failures.append( + f"only {mirror_hits} version restatements found, floor is {min_mirrors} " + f"-- the scan is matching less than it used to" + ) + + # Rule 2: every workflow pin is derived or registered. + registered = {e.path for e in exemptions} + if workflows is None: + workflows = sorted((REPO / ".github/workflows").glob("*.yml")) + workflow_pins = 0 + for wf in workflows: + rel = wf.relative_to(REPO).as_posix() + for lineno, key, value in _node_version_keys(wf.read_text(encoding="utf-8")): + workflow_pins += 1 + if key == "node-version-file": + if value != GAP_ORACLE: + failures.append( + f"{rel}:{lineno}: node-version-file is {value!r}, " + f"must be {GAP_ORACLE!r}" + ) + continue + if rel not in registered: + failures.append( + f"{rel}:{lineno}: literal node-version {value!r} is not registered. " + f"Use `node-version-file: {GAP_ORACLE}`, or add an exemption with a " + f"reason to {Path(__file__).name}" + ) + if workflow_pins < min_workflow_pins: + failures.append( + f"only {workflow_pins} setup-node inputs found across workflows, floor is " + f"{min_workflow_pins} -- the workflow scan is matching less than it used to" + ) + + # Rule 3: every exemption still describes the tree, and declares its major. + for e in exemptions: + try: + text = reader(e.path) + except (OSError, FileNotFoundError): + failures.append(f"{e.path}: exemption names a file that does not exist") + continue + values = e.locator(text) if e.locator else [text.strip()] + if e.value not in values: + failures.append( + f"{e.path}: exemption expects {e.value!r} but the file has " + f"{values!r} -- update this exemption or delete it" + ) + continue + if e.major_tracks_oracle: + want = sources["gap"].split(".")[0] + got = e.value.lstrip("v").split(".")[0] + if got != want: + failures.append( + f"{e.path}: pinned to Node {e.value}, but it declares " + f"major_tracks_oracle and {GAP_ORACLE} is major {want}" + ) + elif not e.reason.strip(): + failures.append(f"{e.path}: exemption off the oracle major needs a reason") + + return failures + + +# --------------------------------------------------------------------------- +# Self-test: every rule above must be able to fail. +# --------------------------------------------------------------------------- +def self_test() -> int: + failures: list[str] = [] + real = _sources() + + def fake_reader(overrides: dict[str, str]) -> Callable[[str], str]: + return lambda path: overrides.get(path, _read(path)) + + # Rule 1a: a drifted restatement is caught. + drifted = _read("CLAUDE.md").replace( + f"The oracle is Node `{real['gap']}`", "The oracle is Node `22.23.1`", 1 + ) + if not any( + "states Node 22.23.1" in f + for f in check(reader=fake_reader({"CLAUDE.md": drifted})) + ): + failures.append("rule 1 does not catch a drifted version restatement") + + # Rule 1b: a marker deleted by rewording is caught, not silently skipped. + reworded = _read("llms.txt").replace( + f"pinned Node oracle ({real['matrix']})", "pinned Node oracle", 1 + ) + if not any( + "matched nothing" in f for f in check(reader=fake_reader({"llms.txt": reworded})) + ): + failures.append("rule 1 does not catch a marker removed by rewording") + + # Rule 1c: the mirror vacuity floor must itself be able to fail. + if not any("floor is" in f for f in check(min_mirrors=99)): + failures.append("MIN_MIRRORS floor cannot fail") + + # Rule 2a: an unregistered literal pin in a workflow is caught. Written to a + # temp file inside .github/workflows so the real glob picks it up, then + # removed -- exercising the discovery path, not just the parser. + rogue = REPO / ".github/workflows" / "zz__self_test_rogue.yml" + rogue.write_text( + "jobs:\n x:\n steps:\n - uses: actions/setup-node@v7\n" + ' with:\n node-version: "18"\n', + encoding="utf-8", + ) + try: + if not any("is not registered" in f for f in check()): + failures.append("rule 2 does not catch an unregistered literal pin") + finally: + rogue.unlink() + + # Rule 2b: a node-version-file pointing somewhere else is caught. + astray = REPO / ".github/workflows" / "zz__self_test_astray.yml" + astray.write_text( + "jobs:\n x:\n steps:\n - uses: actions/setup-node@v7\n" + " with:\n node-version-file: .nvmrc\n", + encoding="utf-8", + ) + try: + if not any("must be '.node-version'" in f for f in check()): + failures.append("rule 2 does not catch a node-version-file pointing elsewhere") + finally: + astray.unlink() + + # Rule 2c: a comment mentioning .node-version must not read as a pin. + if _node_version_keys(" # node-version: 18 (historical)\n"): + failures.append("workflow scanner reads commented-out pins as real") + + # Rule 2d: the workflow vacuity floor must itself be able to fail. + if not any("workflow scan is matching less" in f for f in check(min_workflow_pins=9999)): + failures.append("MIN_WORKFLOW_PINS floor cannot fail") + + # Rule 3a: a stale exemption (tree moved on) is caught. + stale = Exemption( + path="test-compat/node-core/pinned-version.txt", + value="v20.x", + major_tracks_oracle=False, + reason="self-test", + ) + if not any( + "update this exemption or delete it" in f for f in check(exemptions=(stale,)) + ): + failures.append("rule 3 does not catch an exemption that stopped matching") + + # Rule 3b: an exemption naming a deleted file is caught. + ghost = Exemption( + path="does/not/exist.txt", value="x", major_tracks_oracle=False, reason="s" + ) + if not any("does not exist" in f for f in check(exemptions=(ghost,))): + failures.append("rule 3 does not catch an exemption naming a deleted file") + + # Rule 3c: a "tracks the oracle" exemption left behind on an old major fails. + behind = Exemption( + path=".github/workflows/release-packages.yml", + value="26", + major_tracks_oracle=True, + reason="self-test", + locator=_workflow_pins, + ) + bumped = dict(real, gap="28.0.0") + if not any( + "declares major_tracks_oracle" in f + for f in check(sources=bumped, exemptions=(behind,)) + ): + failures.append("rule 3 does not catch a tracking exemption stuck on an old major") + + for failure in failures: + print(f"check_node_version_consistency self-test: {failure}", file=sys.stderr) + if failures: + return 1 + print("check_node_version_consistency self-test: OK") + return 0 + + +def describe() -> int: + sources = _sources() + print(f"gap-suite oracle {GAP_ORACLE}: {sources['gap']}") + print(f"compat-matrix pin {MATRIX_ORACLE} tools.node.version: {sources['matrix']}") + print("\nrestatements checked against a pin:") + for path, pattern, source in MIRRORS: + print(f" {path} [{source}] {pattern}") + print("\nregistered exemptions (each must still match the tree):") + for e in EXEMPTIONS: + tracks = "major tracks oracle" if e.major_tracks_oracle else "off-oracle" + print(f" {e.path} = {e.value} ({tracks})") + print(f" {e.reason}") + return 0 + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--self-test", action="store_true") + parser.add_argument("--list", action="store_true") + args = parser.parse_args() + if args.self_test: + return self_test() + if args.list: + return describe() + failures = check() + for failure in failures: + print(f"node version consistency: {failure}", file=sys.stderr) + if failures: + print( + f"\n{len(failures)} problem(s). The authoritative pin is {GAP_ORACLE}; " + f"run --list to see every checked restatement and exemption.", + file=sys.stderr, + ) + return 1 + print("node version consistency: OK") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) From 9c43c8dea17f9c25f1220bbb4dc2cbd0c34d906b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ralph=20K=C3=BCpper?= Date: Wed, 12 Aug 2026 15:42:22 +0200 Subject: [PATCH 2/2] docs: changelog fragment for #7967 Claude-Session: https://claude.ai/code/session_012B8z92S82sCfqCrVqrFgS2 --- changelog.d/7967-node26-everywhere.md | 19 +++++++++++++++++++ gc-handoff/NODE26-NOTES.md | 2 +- 2 files changed, 20 insertions(+), 1 deletion(-) create mode 100644 changelog.d/7967-node26-everywhere.md diff --git a/changelog.d/7967-node26-everywhere.md b/changelog.d/7967-node26-everywhere.md new file mode 100644 index 0000000000..37c6f5f196 --- /dev/null +++ b/changelog.d/7967-node26-everywhere.md @@ -0,0 +1,19 @@ +**Every Node the project chooses is now 26, and one file is asserted to be the source (#7967).** `.node-version` (26.5.1) has been the authoritative oracle since #6367, but the pin leaked twice: CLAUDE.md's prose drifted off the file (#7599), and `.github/workflows/npm-launcher.yml` was created by #6350 on the *same day* #6367 converted every workflow that existed, keeping that day's ambient `"22.23.1"` literal by omission rather than by decision. CLAUDE.md compounded it by saying "**Two** workflows are deliberately exempt" and then listing three, which is why the fourth pin read as accounted-for. + +Changed: + +- **`npm-launcher.yml`** (both jobs): `"22.23.1"` → `node-version-file: .node-version`. This one is behaviour-sensitive, not a toolchain: the `detect-self-test` job runs `node npm/perry/test/detect.test.cjs`, exercising `npm/perry/bin/detect.cjs` — shipped JS that every installing user executes. Safe on the `ubuntu-22.04` glibc-2.35 job: Node 26 requires glibc ≥ 2.28 (nodejs/node `v26.x` BUILDING.md, Tier 1 for linux x64/arm64). +- **`release-packages.yml`**: `"20"` → `"26"`. Node 20 reached end-of-life on 2026-04-30, and this is the repo's most privileged job (`id-token: write`, OIDC-publishes every platform package) — an EOL runtime there is a defect independent of the sweep. +- **`release-hono-server.yml`**: `"24"` → `"26"`. + +Both release workflows stay pinned to a bare **major literal** rather than `node-version-file`, deliberately: they are npm *publishing* toolchains, and a gap-suite oracle bump made for parity reasons must never be able to move the runtime that publishes releases. Their `npm install -g npm@latest` step is kept — Node 26 already clears the OIDC ≥ 11.5.1 floor, but pinning the guarantee beats inheriting it. + +New **`scripts/check_node_version_consistency.py`**, wired as a step in `lint` (a required context), following the `check_gc_doc_claims.py` pattern: it re-derives every restatement of a Node version from the file that restatement quotes (CLAUDE.md ×2, `llms.txt`, `test-parity/node-compat-matrix.baseline.json`), and requires every literal `node-version:` under `.github/workflows/` to be a *registered exemption carrying a reason*. A rewording that removes a marker fails rather than silently ceasing to be checked; exemptions are asserted against the tree, so one that stops matching fails and must be updated or deleted. It deliberately does **not** assert that `.node-version` and `external-tools.json`'s pin are equal — CLAUDE.md documents the compat-matrix oracle as independent, so encoding that coupling would be wrong. `--self-test` proves each rule can fail (including both vacuity floors); reverting `npm-launcher.yml` to `"22.23.1"` reproduces the historical bug as two named failures. + +**Not changed, and this is the substantive finding: `benchmarks/public-baseline-config.json` still pins `v22.23.1`.** Perry's *published* performance comparison is measured against a runtime four majors old (pre-existing — #6376, regenerated #7285; #7958 only made it declarative, which is how it became visible). It cannot be edited on its own: the file is in `public_baseline.HARNESS_PATHS`, so its bytes feed the committed artifact's `harness_fingerprint`, and `validate_public()` separately compares the config against the version recorded in `benchmarks/results/public-node-bun-v1.json`. Measured rather than assumed — flipping the pin to `v26.5.1` and running the gate exits **2** with `public artifact benchmark harness changed`, and that gate is the "Public benchmark evidence freshness" step of **`lint`**. Shipping the pin edit alone would therefore not be "stale but honest"; it would be a required gate red for `main` and every open PR. The pin and its ~2 h measurement are atomic by design (#7282/#7958). + +It is instead registered as a **self-clearing exemption** carrying the runbook: regenerate on `perry@perry-macos.local` (the Apple M1 / 8 GB mini recorded in the artifact's `host` block — the artifact pins host identity, so regenerating elsewhere replaces the baseline rather than updating it), ~2 h across five components, each gated on ≤ 25% CPU active for 60 consecutive seconds. When that happens the config value moves, the exemption stops matching, and the check fails until the entry is deleted — so the stale pin cannot quietly become a fossil. **Expect the regeneration to reduce Perry's published advantage**: Node 26 ships a materially newer V8 than Node 22. The magnitude was not measured, because the box available for this work has run at load 30–200 all day and cannot satisfy the artifact's own quiet-host policy; a number produced there would be junk wearing a decimal point. + +Also corrected: `external-tools.json` instructed readers to bump "the `NODE_PIN.version` constant in `scripts/node_compat_matrix.mjs`", which does not exist — that script reads `external-tools.json` itself. + +Out of scope: `bun: 1.3.14` in the same config is **not** equally stale (1.3.x is the current Bun line), but it is frozen by the same atomicity rule and should be re-pinned in the same regeneration run. diff --git a/gc-handoff/NODE26-NOTES.md b/gc-handoff/NODE26-NOTES.md index e0ddef6da3..e45aa6eac3 100644 --- a/gc-handoff/NODE26-NOTES.md +++ b/gc-handoff/NODE26-NOTES.md @@ -1,6 +1,6 @@ # Node 26 everywhere — inventory, verdicts, and what must be regenerated -Status: IN PROGRESS (written incrementally; see "Open" at the bottom). +Status: COMPLETE. PR https://github.com/PerryTS/perry/pull/7967 Owner directive: "We want Node 26 everywhere." Project oracle is **26.5.1** (`.node-version`, authoritative).