From 44a912dd3c901adee1a7721aaf362b347254dac7 Mon Sep 17 00:00:00 2001 From: modusensus Date: Tue, 20 Oct 2026 08:53:41 +0800 Subject: [PATCH 1/2] docs: the coverage gaps that closed, the draft status that outlived its run AGENTS.md still stated Go and Rust coverage as live confirmed gaps in the same file whose producer row already listed the formats that closed them, and the fact contract still described itself as a design draft not yet validated by a real run. Each is amended with the date and record that disproved it, not deleted. The contract's `effect` field is now stated as what it is: required of the engine, emitted by no producer here, and unchecked by `euthyna gate`, which reads only the per-gate status. --- AGENTS.md | 6 +++--- docs/fact-contract-zh.md | 8 +++++++- docs/fact-contract.md | 11 ++++++++++- 3 files changed, 20 insertions(+), 5 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index ae512b8..37c9455 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -687,7 +687,7 @@ is faithful — that is a reading job, and a mechanical stand-in would only manu | Path | What it is | |---|---| -| `bin/` + `src/` | **Fact producers** (zero-dependency Node CLI). `src/contract.js` is the contract in code; `src/facts/history.js`, `src/facts/coverage.js` and `src/facts/deps.js` are the three measurements (`history` has `--origins` to chase the first introducer and classifies by message + deleted-line + diff, **on by default since `euthyna audit`**; `coverage.js` reads c8/V8 JSON, coverage.py JSON (format 3), Go's text `-coverprofile` **and lcov tracefiles** (the Go profile needs `--source `: the profile carries no function names, so the producer parses top-level `func` ranges from the gofmt'd source; a file absent from the profile is notEvaluated, never "never executed"; the lcov path needs no `--source` because `FN:` carries the names — they are mangled, and the producer matches a `module::function` query against the `` tail segments of Rust v0 mangling (what `-C instrument-coverage` implies), since no demangler exists on this machine — each segment is anchored at a boundary and never inside a longer `` pair, so a query for `get` does not answer for `cfg_3get`; see `docs/fact-contract.md` §6.2 for the measured record, including that inlining was measured **not** to flatten a called function to 0); `deps.js` reads package-lock.json / Cargo.lock / go.mod, **located from the repository root only** — a monorepo sub-package lock is measured only by `deps --lockfile `, and `audit` does not notice its absence, which `docs/fact-contract.md` §6.3 records with the numbers from the probe). `audit` compares the manifest it measured with the one at `--head` and records `manifest-not-head` when the content differs or the file is absent there. `euthyna audit` (0.4.0, `src/cli.js`) runs history + the full dependency surface in one command — the single entry point the CI recipe and the skill both describe. `src/gate.js` is the six-gate report validator behind `euthyna gate <报告> [--verify] [--allow-exec]` — it parses the skill's 裁定格式, downgrades findings that lack evidence/reproduce or contradict their verdict, and executes nothing but `git` unless the caller consents to interpreters | +| `bin/` + `src/` | **Fact producers** (zero-dependency Node CLI). `src/contract.js` is the contract in code; `src/facts/history.js`, `src/facts/coverage.js` and `src/facts/deps.js` are the three measurements (`history` has `--origins` to chase the first introducer and classifies by message + deleted-line + diff, **on by default since `euthyna audit`**; `coverage.js` reads **five** formats — c8/V8 JSON, classic istanbul JSON (jest's default provider, nyc: the same `fnMap`/`f` shape, so the symbol locator shares the c8 code path), coverage.py JSON (format 3), Go's text `-coverprofile` **and lcov tracefiles** (the Go profile needs `--source `: the profile carries no function names, so the producer parses top-level `func` ranges from the gofmt'd source; a file absent from the profile is notEvaluated, never "never executed"; the lcov path needs no `--source` because `FN:` carries the names — they are mangled, and the producer matches a `module::function` query against the `` tail segments of Rust v0 mangling (what `-C instrument-coverage` implies), since no demangler exists on this machine — each segment is anchored at a boundary and never inside a longer `` pair, so a query for `get` does not answer for `cfg_3get`; see `docs/fact-contract.md` §6.2 for the measured record, including that inlining was measured **not** to flatten a called function to 0); `deps.js` reads package-lock.json / Cargo.lock / go.mod, **located from the repository root only** — a monorepo sub-package lock is measured only by `deps --lockfile `, and `audit` does not notice its absence, which `docs/fact-contract.md` §6.3 records with the numbers from the probe). `audit` compares the manifest it measured with the one at `--head` and records `manifest-not-head` when the content differs or the file is absent there. `euthyna audit` (0.4.0, `src/cli.js`) runs history + the full dependency surface in one command — the single entry point the CI recipe and the skill both describe. `src/gate.js` is the six-gate report validator behind `euthyna gate <报告> [--verify] [--allow-exec]` — it parses the skill's 裁定格式, downgrades findings that lack evidence/reproduce or contradict their verdict, and executes nothing but `git` unless the caller consents to interpreters | | `test/` | 296 tests via `node --test`, no third-party framework. The pass/skip split is keyed on **tar flavour, not the OS**: `test/bench.test.js` skips `every exploit expectation holds` unless `tar --version` reports bsdtar. Measured on `cdece0f`: ubuntu 245 pass + 1 skip (its log line: *the exploit ground truth is defined against bsdtar, not "tar (GNU tar) 1.35"*), windows 246 pass + 0 skip — and this machine's Git Bash PATH puts `D:\Git\usr\bin\tar.exe` (GNU tar 1.35) ahead of System32's `tar.exe`, so a local run **on `cdece0f`** reported 245 + 1 even though it is Windows (measured again at `4d620cd` for 0.8.0: pwsh 296 pass / 0 skip, Git Bash 295 pass / 1 skip, the difference being that one case — the 245/246 pair above is the `cdece0f` record and 296/295 is the current count, so read that scope with the number). **The history tests build real git repositories rather than mocking**; the coverage tests carry fixtures shaped like c8, coverage.py, Go profile and lcov tracefile output (`coverage.test.js` + `coverage-go.test.js` + `coverage-lcov.test.js`, the lcov fixtures using mangled names copied from a real `llvm-cov export --format=lcov` run); the deps tests carry fixture lockfiles (npm v1/v2/v3, Cargo.lock, go.mod); the contract tests carry the shell-quoting and render-boundary regressions from PR #1; `git.test.js` carries the error-path quoting and stderr-sanitization regressions from issue #2; `gate.test.js` pins the six-gate validator (including that `--verify` does not run an interpreter command without `--allow-exec`); `cli.test.js` pins the `audit` command, its exit codes, and the manifest-version declaration — content against `--head`, so a PR merge commit whose lockfile is byte-identical to `--head`'s declares nothing while a gitignored lockfile declares itself; the decision itself is an exported pure function so the fail-closed branch (`manifest-unverified`) is reachable from a test; `skill.test.js` pins the written discipline (the six gates, the three verdicts, the `euthyna gate` reference) so weakening the skill turns CI red; `plugin.test.js`, `claude-plugin.test.js` and `hermes-plugin.test.js` pin the three manifest surfaces; `manifest-version.test.js` pins `package.json` against the three plugin manifests (0.5.0 shipped them a release behind and nothing compared them); `i18n.test.js` pins language rendering and report parsing (zh/en); `skill-mirror.test.js` compares the two skill editions against each other — file set, heading sequence (ATX and Setext), fenced blocks, table shapes, and the `euthyna ` / `--flag` identifier sets — so an unmirrored discipline change goes red instead of relying on memory, `docs-pair.test.js` holds `docs/` to the same rule (every document paired, and each pair equal on structure and identifiers — not on prose fidelity), and `markdown-structure.test.js` pins the shared scanner itself with seven crafted-document cases, because a legal Markdown form it cannot read makes any comparison pass on that form (neither guard compares word counts, which a translation legitimately needs room for). **The suite count is written down once, in this row**: README's quickstart carried its own number (207) and was 39 tests stale, so it now says only what `npm test` does, and `package.json` defines that as `node --test` | | `.agents/skills/euthyna/` | **The skill.** Doubles as source and as a project skill root (rank 200), so it is live in this workspace without a restart | | `bench/` | **The recall benchmark.** `exploits.js` establishes ground truth by execution (18 cases, 8 near-neighbour pairs); `prepare-blind.js` produces answer-free copies with per-round shuffled ids; `adjudicate.js` closes the loop end to end (blind tree → one adjudicator process per case → `collect-verdicts.js` machine validation → `score.js`), with a deterministic `golden` mode (ground truth written into reports — plumbing self-check only, never a real round) for CI; `collect-verdicts.js` machine-validates reports and extracts verdicts without the orchestrator reading report bodies; `score.js` computes the confusion matrix and the pair view; `perf.js` is the `history` scaling baseline. The verified-platform matrix (bsdtar vs GNU tar, p6 skip) lives in `bench/README.md`. Results: `bench/RESULTS.md` (round 1), `bench/RESULTS-round2.md` (round 2: three independent runs per case, zero flips), `bench/RESULTS-round3.md` (round 3: 18 cases, 54 adjudications, two fixture defects caught by adjudicators), `bench/RESULTS-round4.md` (round 4: 54/54, p6b v3 confirmed by first blind adjudication, cross-model run executed — zero flips across two model families); protocols in `bench/README.md`, design pre-registrations in `bench/DESIGN-round3.md` and `bench/DESIGN-round4.md` | @@ -697,8 +697,8 @@ is faithful — that is a reading job, and a mechanical stand-in would only manu | `docs/ci-integration.md` + `-zh` | A paste-ready GitHub Actions recipe that gates a pull request on `euthyna audit`, keyed to the CLI's published exit-code contract (needs `fetch-depth: 0` for the history producer) | | `docs/case-study-axe-core.md` + `-zh` | Validation run #1: JavaScript (axe-core), 5/5 coarse-screen candidates refuted at the gates | | `docs/case-study-crewai.md` + `-zh` | Validation run #2: Python (crewAI). **Proved the Python claim**: `history` works on Python repos, and the coverage producer was extended to read coverage.py JSON. 3 candidates → 2 FP + 1 INCONCLUSIVE (pickle, supply-chain-dependent) | -| `docs/case-study-act.md` + `-zh` | Validation run #3: Go (nektos/act, v0.2.89). **Proved the Go claim** for `history` (GHSL-2023-004 fix's deleted lines traced to the introducing commit) and `deps` (go.mod, honestly labelled "declared, not resolved"). **Coverage on Go targets is a confirmed gap** (`go test -coverprofile` text format refused). 4 candidates → 4 FP + 2 observations; the traversal fix re-verified by an executable PoC that also confirmed the artifacts server is unauthenticated. The act clone and PoC live in `.scratch/` (gitignored) | -| `docs/case-study-rust.md` + `-zh` | Validation run #4: Rust (rustls, `v/0.23.44`→`v/0.23.45` = RUSTSEC-2026-0285). **Proved the Rust claim** for `history` (43 facts over the fix's deleted lines) and `deps` (366 Cargo.lock facts). **The project's first TRUE POSITIVE**: a standalone PoC drives `ServerConnection` with hand-built ClientHello wire bytes and shows the pre-fix server accepting a second hello that withdraws the HelloRetryRequest's named suite (ServerHello selects `0x1303` after the retry said `0x1301`), while the fixed tag refuses with `CipherSuiteDifferedOnRetry`. 6 verdicts → 1 TP / 4 FP / 1 INC, validated by `euthyna gate` (exit 0). Rust coverage remains a **confirmed gap**; the target's own test suite cannot run here at all (`aws-lc-sys` build script requires NASM), which is why the PoC is a separate crate on the `ring` provider. Clone, PoC and reports live in `.scratch/` (gitignored) | +| `docs/case-study-act.md` + `-zh` | Validation run #3: Go (nektos/act, v0.2.89). **Proved the Go claim** for `history` (GHSL-2023-004 fix's deleted lines traced to the introducing commit) and `deps` (go.mod, honestly labelled "declared, not resolved"). **Coverage on Go targets was a confirmed gap in that round** (`go test -coverprofile` text format refused), **closed 2026-10-08** — the text profile is now read by the producer, and the run's own record is amended by the revision note in that document rather than rewritten. 4 candidates → 4 FP + 2 observations; the traversal fix re-verified by an executable PoC that also confirmed the artifacts server is unauthenticated. The act clone and PoC live in `.scratch/` (gitignored) | +| `docs/case-study-rust.md` + `-zh` | Validation run #4: Rust (rustls, `v/0.23.44`→`v/0.23.45` = RUSTSEC-2026-0285). **Proved the Rust claim** for `history` (43 facts over the fix's deleted lines) and `deps` (366 Cargo.lock facts). **The project's first TRUE POSITIVE**: a standalone PoC drives `ServerConnection` with hand-built ClientHello wire bytes and shows the pre-fix server accepting a second hello that withdraws the HelloRetryRequest's named suite (ServerHello selects `0x1303` after the retry said `0x1301`), while the fixed tag refuses with `CipherSuiteDifferedOnRetry`. 6 verdicts → 1 TP / 4 FP / 1 INC, validated by `euthyna gate` (exit 0). Rust coverage was a **confirmed gap in that round**, and **closed 2026-10-12** with lcov tracefiles read from a probe crate built under `-C instrument-coverage`; the follow-up note in that document keeps the two records apart rather than rewriting the first. What is still true today: the target's own test suite cannot run here at all (`aws-lc-sys` build script requires NASM), which is why the PoC is a separate crate on the `ring` provider. Clone, PoC and reports live in `.scratch/` (gitignored) | | `audits/` | **Audit reports, one per run, local-only** (never written into the target repository, and deliberately **not committed** here — the public form is `docs/case-study-*.md`). See `audits/CREWAI_EUTHYNA_AUDIT_2026-09-20.md` and `audits/RUSTLS_EUTHYNA_AUDIT_2026-10-09.md` (gitignored) | | `tools/fetch-references.js` | Fetches upstream sources on demand into `.refs/` (gitignored). **The repo distributes no third-party files** | | `tools/check-license-text.mjs` | Verifies `LICENSE` against the canonical Apache-2.0 text, fetched live. **The licence claim is checkable rather than asserted** | diff --git a/docs/fact-contract-zh.md b/docs/fact-contract-zh.md index 36658e4..14de1d7 100644 --- a/docs/fact-contract-zh.md +++ b/docs/fact-contract-zh.md @@ -5,7 +5,9 @@ > 这是 euthyna 的**核心资产**：定义「测量层」交给「判定层」的到底是什么。 > 定位分析的结论是 euthyna 不重造测量，那它就必须定义**接口**——这份文档就是那个接口。 > -> 状态：**设计稿，未经实跑验证**。字段会在第一次 quick 档实跑后修订。 +> 状态：**v0.1，已按四次真实验证运行修订**（axe-core、crewAI、act、rustls）。这里原本那句「设计稿，未经实跑 +> 验证」写在第一次实跑之前，作为它写下时的状态留在那个提交里。仍未闭合的两件：§6 记着还没有关掉的产出器 +> 边界，以及一个契约要求、却没有任何工具产出或校验的字段（§3.2）。 --- @@ -142,6 +144,10 @@ > 这是 methodology M6 的核心，也是最容易被 LLM 混为一谈的地方—— > 契约必须在数据结构上强制区分，而不是靠提示词提醒。 > 已有工具没人产出这个字段（八个测量工具全部只看「有没有」，不看「防到什么程度」）。 +> **2026-10-20 实测，本仓库也算在内**：这里没有任何产出器产生它，`euthyna gate` 也不校验它——`src/gate.js` +> 只校验每个门禁的状态（`PASS` / `FAIL` / `NOT EVALUATED`），所以一份报告可以完全不区分两者就过第 5、第 6 +> 道。这条要求约束的是引擎，不是工具链。把这道缺口补上（给它一个产出器，或者让校验器拒绝没有带 `effect` +> 的 Gate 5/6 `PASS`）是一条未闭合项。 ### 3.3 ⛔ 契约里**不许出现**的东西 diff --git a/docs/fact-contract.md b/docs/fact-contract.md index cec13f0..cbe8d62 100644 --- a/docs/fact-contract.md +++ b/docs/fact-contract.md @@ -5,7 +5,11 @@ English · [中文](fact-contract-zh.md) > This is euthyna's **core asset**: it defines what exactly the "measurement layer" hands over to the "adjudication layer". > The positioning analysis concluded that euthyna does not rebuild measurement; it must therefore define the **interface** — and this document is that interface. > -> Status: **design draft, not yet validated by a real run**. Fields will be revised after the first real run in quick mode. +> Status: **v0.1, revised against four real validation runs** (axe-core, crewAI, act, rustls). The line +> that stood here — "design draft, not yet validated by a real run" — was written before the first of +> them and is kept in that commit as the record of the state it was written in. Two things are still +> open: §6 records the producer boundaries that have not been closed, and one field the contract +> mandates is produced and checked by nothing (§3.2). --- @@ -141,6 +145,11 @@ Organized by which step of the adjudication layer consumes each one: > This is the core of methodology M6, and the thing an LLM most easily conflates — > the contract must enforce the distinction in the data structure, not rely on a prompt to remember it. > No existing tool produces this field (all eight measurement tools look only at "whether one exists", never at "how much it actually prevents"). +> **Measured 2026-10-20, and this repository included:** no producer here emits it either, and `euthyna gate` +> does not check it — `src/gate.js` validates only the per-gate status (`PASS` / `FAIL` / `NOT EVALUATED`), so a +> report can pass Gate 5 or Gate 6 without ever drawing the distinction. The requirement therefore binds the +> engine, not the tooling. Closing that gap (a producer, or a validator that refuses a Gate 5/6 `PASS` with no +> `effect`) is an open item. ### 3.3 ⛔ What must **not** appear in the contract From 1bdc7266781be7882a2b8b89adbfd1f13123ee12 Mon Sep 17 00:00:00 2001 From: modusensus Date: Tue, 20 Oct 2026 18:43:49 +0800 Subject: [PATCH 2/2] docs(contract): state what the gate validator checks, instead of "status only" MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The review caught an inference dressed as a measurement. `src/gate.js` does not validate "only the per-gate status": it requires the gate number to fall in 1..6, a reason on every FAIL or NOT EVALUATED gate, and — for a TRUE POSITIVE — evidence in path:L123 form, a reproduce command and an impact statement. The accurate claim is narrower and just as load-bearing: not one of those checks reads an `effect` value, so Gate 5 and Gate 6 pass without the distinction. Both twins amended. The pairing is guarded rather than assumed: deleting `euthyna gate` from the Chinese half turns docs-pair red with "subcommands only in zh". --- docs/fact-contract-zh.md | 9 +++++---- docs/fact-contract.md | 10 ++++++---- 2 files changed, 11 insertions(+), 8 deletions(-) diff --git a/docs/fact-contract-zh.md b/docs/fact-contract-zh.md index 14de1d7..f8bcdd2 100644 --- a/docs/fact-contract-zh.md +++ b/docs/fact-contract-zh.md @@ -144,10 +144,11 @@ > 这是 methodology M6 的核心，也是最容易被 LLM 混为一谈的地方—— > 契约必须在数据结构上强制区分，而不是靠提示词提醒。 > 已有工具没人产出这个字段（八个测量工具全部只看「有没有」，不看「防到什么程度」）。 -> **2026-10-20 实测，本仓库也算在内**：这里没有任何产出器产生它，`euthyna gate` 也不校验它——`src/gate.js` -> 只校验每个门禁的状态（`PASS` / `FAIL` / `NOT EVALUATED`），所以一份报告可以完全不区分两者就过第 5、第 6 -> 道。这条要求约束的是引擎，不是工具链。把这道缺口补上（给它一个产出器，或者让校验器拒绝没有带 `effect` -> 的 Gate 5/6 `PASS`）是一条未闭合项。 +> **2026-10-20 实测，本仓库也算在内**：这里没有任何产出器产生它，`euthyna gate` 的第 5、第 6 道也没有任何一处读它。 +> `src/gate.js` 确实强制的是结构——门禁编号必须落在 1..6，`FAIL` 或 `NOT EVALUATED` 的门禁必须带理由，一条 +> TRUE POSITIVE 必须带 `path:L123` 形式的证据、复现命令与影响说明——但那份清单里没有一项读 `effect` 值， +> 所以一份报告可以完全不区分两者就过第 5、第 6 道。这条要求约束的是引擎，不是工具链。把这道缺口补上（给它一个 +> 产出器，或者让校验器拒绝没带 `effect` 的 Gate 5/6 通过）是一条未闭合项。 ### 3.3 ⛔ 契约里**不许出现**的东西 diff --git a/docs/fact-contract.md b/docs/fact-contract.md index cbe8d62..d7fe75e 100644 --- a/docs/fact-contract.md +++ b/docs/fact-contract.md @@ -146,10 +146,12 @@ Organized by which step of the adjudication layer consumes each one: > the contract must enforce the distinction in the data structure, not rely on a prompt to remember it. > No existing tool produces this field (all eight measurement tools look only at "whether one exists", never at "how much it actually prevents"). > **Measured 2026-10-20, and this repository included:** no producer here emits it either, and `euthyna gate` -> does not check it — `src/gate.js` validates only the per-gate status (`PASS` / `FAIL` / `NOT EVALUATED`), so a -> report can pass Gate 5 or Gate 6 without ever drawing the distinction. The requirement therefore binds the -> engine, not the tooling. Closing that gap (a producer, or a validator that refuses a Gate 5/6 `PASS` with no -> `effect`) is an open item. +> reads it in no Gate 5 or Gate 6 check. `src/gate.js` does enforce structure — the gate number must fall in +> 1..6, a `FAIL` or `NOT EVALUATED` gate must carry a reason, and a TRUE POSITIVE must carry evidence in +> `path:L123` form, a reproduce command and an impact statement — but not one item on that list reads an +> `effect` value, so a report can pass Gate 5 or Gate 6 without ever drawing the distinction. The requirement +> therefore binds the engine, not the tooling. Closing that gap (a producer, or a check that refuses a +> Gate 5/6 pass carrying no `effect`) is an open item. ### 3.3 ⛔ What must **not** appear in the contract