diff --git a/AGENTS.md b/AGENTS.md index 02748b9..1b47317 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -687,7 +687,7 @@ is faithful — that is a reading job, and a mechanical stand-in would only manu | Path | What it is | |---|---| -| `bin/` + `src/` | **Fact producers** (zero-dependency Node CLI). `src/contract.js` is the contract in code; `src/facts/history.js`, `src/facts/coverage.js` and `src/facts/deps.js` are the three measurements (`history` has `--origins` to chase the first introducer and classifies by message + deleted-line + diff, **on by default since `euthyna audit`**; `coverage.js` reads c8/V8 JSON, coverage.py JSON (format 3), Go's text `-coverprofile` **and lcov tracefiles** (the Go profile needs `--source `: the profile carries no function names, so the producer parses top-level `func` ranges from the gofmt'd source; a file absent from the profile is notEvaluated, never "never executed"; the lcov path needs no `--source` because `FN:` carries the names — they are mangled, and the producer matches a `module::function` query against the `` tail segments of Rust v0 mangling (what `-C instrument-coverage` implies), since no demangler exists on this machine — each segment is anchored at a boundary and never inside a longer `` pair, so a query for `get` does not answer for `cfg_3get`; see `docs/fact-contract.md` §6.2 for the measured record, including that inlining was measured **not** to flatten a called function to 0); `deps.js` reads package-lock.json / Cargo.lock / go.mod, **located from the repository root only** — a monorepo sub-package lock is measured only by `deps --lockfile `, and `audit` does not notice its absence, which `docs/fact-contract.md` §6.3 records with the numbers from the probe). `audit` compares the manifest it measured with the one at `--head` and records `manifest-not-head` when the content differs or the file is absent there. `euthyna audit` (0.4.0, `src/cli.js`) runs history + the full dependency surface in one command — the single entry point the CI recipe and the skill both describe. `src/gate.js` is the six-gate report validator behind `euthyna gate <报告> [--verify] [--allow-exec]` — it parses the skill's 裁定格式, downgrades findings that lack evidence/reproduce or contradict their verdict, and executes nothing but `git` unless the caller consents to interpreters. `src/graph.js` is the Mermaid view behind `--graph` on `euthyna history` and `euthyna audit`: it draws the attribution `history` already emits (the range → the commit that last modified each deleted line → the commit that first introduced its content, plus the `notEvaluated` surface as dotted edges), adds no measurement, and carries states only — colouring a node by risk would make §3.3 of the contract false | +| `bin/` + `src/` | **Fact producers** (zero-dependency Node CLI). `src/contract.js` is the contract in code; `src/facts/history.js`, `src/facts/coverage.js` and `src/facts/deps.js` are the three measurements (`history` has `--origins` to chase the first introducer and classifies by message + deleted-line + diff, **on by default since `euthyna audit`**; `coverage.js` reads **five** formats — c8/V8 JSON, classic istanbul JSON (jest's default provider, nyc: the same `fnMap`/`f` shape, so the symbol locator shares the c8 code path), coverage.py JSON (format 3), Go's text `-coverprofile` **and lcov tracefiles** (the Go profile needs `--source `: the profile carries no function names, so the producer parses top-level `func` ranges from the gofmt'd source; a file absent from the profile is notEvaluated, never "never executed"; the lcov path needs no `--source` because `FN:` carries the names — they are mangled, and the producer matches a `module::function` query against the `` tail segments of Rust v0 mangling (what `-C instrument-coverage` implies), since no demangler exists on this machine — each segment is anchored at a boundary and never inside a longer `` pair, so a query for `get` does not answer for `cfg_3get`; see `docs/fact-contract.md` §6.2 for the measured record, including that inlining was measured **not** to flatten a called function to 0); `deps.js` reads package-lock.json / Cargo.lock / go.mod, **located from the repository root only** — a monorepo sub-package lock is measured only by `deps --lockfile `, and `audit` does not notice its absence, which `docs/fact-contract.md` §6.3 records with the numbers from the probe). `audit` compares the manifest it measured with the one at `--head` and records `manifest-not-head` when the content differs or the file is absent there. `euthyna audit` (0.4.0, `src/cli.js`) runs history + the full dependency surface in one command — the single entry point the CI recipe and the skill both describe. `src/gate.js` is the six-gate report validator behind `euthyna gate <报告> [--verify] [--allow-exec]` — it parses the skill's 裁定格式, downgrades findings that lack evidence/reproduce or contradict their verdict, and executes nothing but `git` unless the caller consents to interpreters. `src/graph.js` is the Mermaid view behind `--graph` on `euthyna history` and `euthyna audit`: it draws the attribution `history` already emits (the range → the commit that last modified each deleted line → the commit that first introduced its content, plus the `notEvaluated` surface as dotted edges), adds no measurement, and carries states only — colouring a node by risk would make §3.3 of the contract false | | `test/` | 316 tests via `node --test` (measured 2026-10-20: +20 over the 0.8.0 count — `test/graph.test.js` is 15, `--graph` adds 5 to `test/cli.test.js` including one that drives a real three-commit repository and one that runs `audit --graph` over a merged report), no third-party framework. The pass/skip split is keyed on **tar flavour, not the OS**: `test/bench.test.js` skips `every exploit expectation holds` unless `tar --version` reports bsdtar. Measured on `cdece0f`: ubuntu 245 pass + 1 skip (its log line: *the exploit ground truth is defined against bsdtar, not "tar (GNU tar) 1.35"*), windows 246 pass + 0 skip — and this machine's Git Bash PATH puts `D:\Git\usr\bin\tar.exe` (GNU tar 1.35) ahead of System32's `tar.exe`, so a local run **on `cdece0f`** reported 245 + 1 even though it is Windows (measured again at `4d620cd` for 0.8.0: pwsh 296 pass / 0 skip, Git Bash 295 pass / 1 skip, the difference being that one case — the 245/246 pair above is the `cdece0f` record and 296/295 was the count at `4d620cd`, so read that scope with the number). **The history tests build real git repositories rather than mocking**; the coverage tests carry fixtures shaped like c8, coverage.py, Go profile and lcov tracefile output (`coverage.test.js` + `coverage-go.test.js` + `coverage-lcov.test.js`, the lcov fixtures using mangled names copied from a real `llvm-cov export --format=lcov` run); the deps tests carry fixture lockfiles (npm v1/v2/v3, Cargo.lock, go.mod); the contract tests carry the shell-quoting and render-boundary regressions from PR #1; `git.test.js` carries the error-path quoting and stderr-sanitization regressions from issue #2; `gate.test.js` pins the six-gate validator (including that `--verify` does not run an interpreter command without `--allow-exec`); `cli.test.js` pins the `audit` command, its exit codes, and the manifest-version declaration — content against `--head`, so a PR merge commit whose lockfile is byte-identical to `--head`'s declares nothing while a gitignored lockfile declares itself; the decision itself is an exported pure function so the fail-closed branch (`manifest-unverified`) is reachable from a test; `skill.test.js` pins the written discipline (the six gates, the three verdicts, the `euthyna gate` reference) so weakening the skill turns CI red; `plugin.test.js`, `claude-plugin.test.js` and `hermes-plugin.test.js` pin the three manifest surfaces; `manifest-version.test.js` pins `package.json` against the three plugin manifests (0.5.0 shipped them a release behind and nothing compared them); `i18n.test.js` pins language rendering and report parsing (zh/en); `skill-mirror.test.js` compares the two skill editions against each other — file set, heading sequence (ATX and Setext), fenced blocks, table shapes, and the `euthyna ` / `--flag` identifier sets — so an unmirrored discipline change goes red instead of relying on memory, `docs-pair.test.js` holds `docs/` to the same rule (every document paired, and each pair equal on structure and identifiers — not on prose fidelity), and `markdown-structure.test.js` pins the shared scanner itself with seven crafted-document cases, because a legal Markdown form it cannot read makes any comparison pass on that form (neither guard compares word counts, which a translation legitimately needs room for). **The suite count is written down once, in this row**: README's quickstart carried its own number (207) and was 39 tests stale, so it now says only what `npm test` does, and `package.json` defines that as `node --test` | | `.agents/skills/euthyna/` | **The skill.** Doubles as source and as a project skill root (rank 200), so it is live in this workspace without a restart | | `bench/` | **The recall benchmark.** `exploits.js` establishes ground truth by execution (18 cases, 8 near-neighbour pairs); `prepare-blind.js` produces answer-free copies with per-round shuffled ids; `adjudicate.js` closes the loop end to end (blind tree → one adjudicator process per case → `collect-verdicts.js` machine validation → `score.js`), with a deterministic `golden` mode (ground truth written into reports — plumbing self-check only, never a real round) for CI; `collect-verdicts.js` machine-validates reports and extracts verdicts without the orchestrator reading report bodies; `score.js` computes the confusion matrix and the pair view; `perf.js` is the `history` scaling baseline. The verified-platform matrix (bsdtar vs GNU tar, p6 skip) lives in `bench/README.md`. Results: `bench/RESULTS.md` (round 1), `bench/RESULTS-round2.md` (round 2: three independent runs per case, zero flips), `bench/RESULTS-round3.md` (round 3: 18 cases, 54 adjudications, two fixture defects caught by adjudicators), `bench/RESULTS-round4.md` (round 4: 54/54, p6b v3 confirmed by first blind adjudication, cross-model run executed — zero flips across two model families); protocols in `bench/README.md`, design pre-registrations in `bench/DESIGN-round3.md` and `bench/DESIGN-round4.md` | @@ -697,8 +697,8 @@ is faithful — that is a reading job, and a mechanical stand-in would only manu | `docs/ci-integration.md` + `-zh` | A paste-ready GitHub Actions recipe that gates a pull request on `euthyna audit`, keyed to the CLI's published exit-code contract (needs `fetch-depth: 0` for the history producer) | | `docs/case-study-axe-core.md` + `-zh` | Validation run #1: JavaScript (axe-core), 5/5 coarse-screen candidates refuted at the gates | | `docs/case-study-crewai.md` + `-zh` | Validation run #2: Python (crewAI). **Proved the Python claim**: `history` works on Python repos, and the coverage producer was extended to read coverage.py JSON. 3 candidates → 2 FP + 1 INCONCLUSIVE (pickle, supply-chain-dependent) | -| `docs/case-study-act.md` + `-zh` | Validation run #3: Go (nektos/act, v0.2.89). **Proved the Go claim** for `history` (GHSL-2023-004 fix's deleted lines traced to the introducing commit) and `deps` (go.mod, honestly labelled "declared, not resolved"). **Coverage on Go targets is a confirmed gap** (`go test -coverprofile` text format refused). 4 candidates → 4 FP + 2 observations; the traversal fix re-verified by an executable PoC that also confirmed the artifacts server is unauthenticated. The act clone and PoC live in `.scratch/` (gitignored) | -| `docs/case-study-rust.md` + `-zh` | Validation run #4: Rust (rustls, `v/0.23.44`→`v/0.23.45` = RUSTSEC-2026-0285). **Proved the Rust claim** for `history` (43 facts over the fix's deleted lines) and `deps` (366 Cargo.lock facts). **The project's first TRUE POSITIVE**: a standalone PoC drives `ServerConnection` with hand-built ClientHello wire bytes and shows the pre-fix server accepting a second hello that withdraws the HelloRetryRequest's named suite (ServerHello selects `0x1303` after the retry said `0x1301`), while the fixed tag refuses with `CipherSuiteDifferedOnRetry`. 6 verdicts → 1 TP / 4 FP / 1 INC, validated by `euthyna gate` (exit 0). Rust coverage remains a **confirmed gap**; the target's own test suite cannot run here at all (`aws-lc-sys` build script requires NASM), which is why the PoC is a separate crate on the `ring` provider. Clone, PoC and reports live in `.scratch/` (gitignored) | +| `docs/case-study-act.md` + `-zh` | Validation run #3: Go (nektos/act, v0.2.89). **Proved the Go claim** for `history` (GHSL-2023-004 fix's deleted lines traced to the introducing commit) and `deps` (go.mod, honestly labelled "declared, not resolved"). **Coverage on Go targets was a confirmed gap in that round** (`go test -coverprofile` text format refused), **closed 2026-10-08** — the text profile is now read by the producer, and the run's own record is amended by the revision note in that document rather than rewritten. 4 candidates → 4 FP + 2 observations; the traversal fix re-verified by an executable PoC that also confirmed the artifacts server is unauthenticated. The act clone and PoC live in `.scratch/` (gitignored) | +| `docs/case-study-rust.md` + `-zh` | Validation run #4: Rust (rustls, `v/0.23.44`→`v/0.23.45` = RUSTSEC-2026-0285). **Proved the Rust claim** for `history` (43 facts over the fix's deleted lines) and `deps` (366 Cargo.lock facts). **The project's first TRUE POSITIVE**: a standalone PoC drives `ServerConnection` with hand-built ClientHello wire bytes and shows the pre-fix server accepting a second hello that withdraws the HelloRetryRequest's named suite (ServerHello selects `0x1303` after the retry said `0x1301`), while the fixed tag refuses with `CipherSuiteDifferedOnRetry`. 6 verdicts → 1 TP / 4 FP / 1 INC, validated by `euthyna gate` (exit 0). Rust coverage was a **confirmed gap in that round**, and **closed 2026-10-12** with lcov tracefiles read from a probe crate built under `-C instrument-coverage`; the follow-up note in that document keeps the two records apart rather than rewriting the first. What is still true today: the target's own test suite cannot run here at all (`aws-lc-sys` build script requires NASM), which is why the PoC is a separate crate on the `ring` provider. Clone, PoC and reports live in `.scratch/` (gitignored) | | `audits/` | **Audit reports, one per run, local-only** (never written into the target repository, and deliberately **not committed** here — the public form is `docs/case-study-*.md`). See `audits/CREWAI_EUTHYNA_AUDIT_2026-09-20.md` and `audits/RUSTLS_EUTHYNA_AUDIT_2026-10-09.md` (gitignored) | | `tools/fetch-references.js` | Fetches upstream sources on demand into `.refs/` (gitignored). **The repo distributes no third-party files** | | `tools/check-license-text.mjs` | Verifies `LICENSE` against the canonical Apache-2.0 text, fetched live. **The licence claim is checkable rather than asserted** | diff --git a/docs/fact-contract-zh.md b/docs/fact-contract-zh.md index 36658e4..f8bcdd2 100644 --- a/docs/fact-contract-zh.md +++ b/docs/fact-contract-zh.md @@ -5,7 +5,9 @@ > 这是 euthyna 的**核心资产**：定义「测量层」交给「判定层」的到底是什么。 > 定位分析的结论是 euthyna 不重造测量，那它就必须定义**接口**——这份文档就是那个接口。 > -> 状态：**设计稿，未经实跑验证**。字段会在第一次 quick 档实跑后修订。 +> 状态：**v0.1，已按四次真实验证运行修订**（axe-core、crewAI、act、rustls）。这里原本那句「设计稿，未经实跑 +> 验证」写在第一次实跑之前，作为它写下时的状态留在那个提交里。仍未闭合的两件：§6 记着还没有关掉的产出器 +> 边界，以及一个契约要求、却没有任何工具产出或校验的字段（§3.2）。 --- @@ -142,6 +144,11 @@ > 这是 methodology M6 的核心，也是最容易被 LLM 混为一谈的地方—— > 契约必须在数据结构上强制区分，而不是靠提示词提醒。 > 已有工具没人产出这个字段（八个测量工具全部只看「有没有」，不看「防到什么程度」）。 +> **2026-10-20 实测，本仓库也算在内**：这里没有任何产出器产生它，`euthyna gate` 的第 5、第 6 道也没有任何一处读它。 +> `src/gate.js` 确实强制的是结构——门禁编号必须落在 1..6，`FAIL` 或 `NOT EVALUATED` 的门禁必须带理由，一条 +> TRUE POSITIVE 必须带 `path:L123` 形式的证据、复现命令与影响说明——但那份清单里没有一项读 `effect` 值， +> 所以一份报告可以完全不区分两者就过第 5、第 6 道。这条要求约束的是引擎，不是工具链。把这道缺口补上（给它一个 +> 产出器，或者让校验器拒绝没带 `effect` 的 Gate 5/6 通过）是一条未闭合项。 ### 3.3 ⛔ 契约里**不许出现**的东西 diff --git a/docs/fact-contract.md b/docs/fact-contract.md index cec13f0..d7fe75e 100644 --- a/docs/fact-contract.md +++ b/docs/fact-contract.md @@ -5,7 +5,11 @@ English · [中文](fact-contract-zh.md) > This is euthyna's **core asset**: it defines what exactly the "measurement layer" hands over to the "adjudication layer". > The positioning analysis concluded that euthyna does not rebuild measurement; it must therefore define the **interface** — and this document is that interface. > -> Status: **design draft, not yet validated by a real run**. Fields will be revised after the first real run in quick mode. +> Status: **v0.1, revised against four real validation runs** (axe-core, crewAI, act, rustls). The line +> that stood here — "design draft, not yet validated by a real run" — was written before the first of +> them and is kept in that commit as the record of the state it was written in. Two things are still +> open: §6 records the producer boundaries that have not been closed, and one field the contract +> mandates is produced and checked by nothing (§3.2). --- @@ -141,6 +145,13 @@ Organized by which step of the adjudication layer consumes each one: > This is the core of methodology M6, and the thing an LLM most easily conflates — > the contract must enforce the distinction in the data structure, not rely on a prompt to remember it. > No existing tool produces this field (all eight measurement tools look only at "whether one exists", never at "how much it actually prevents"). +> **Measured 2026-10-20, and this repository included:** no producer here emits it either, and `euthyna gate` +> reads it in no Gate 5 or Gate 6 check. `src/gate.js` does enforce structure — the gate number must fall in +> 1..6, a `FAIL` or `NOT EVALUATED` gate must carry a reason, and a TRUE POSITIVE must carry evidence in +> `path:L123` form, a reproduce command and an impact statement — but not one item on that list reads an +> `effect` value, so a report can pass Gate 5 or Gate 6 without ever drawing the distinction. The requirement +> therefore binds the engine, not the tooling. Closing that gap (a producer, or a check that refuses a +> Gate 5/6 pass carrying no `effect`) is an open item. ### 3.3 ⛔ What must **not** appear in the contract