Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -687,7 +687,7 @@ is faithful — that is a reading job, and a mechanical stand-in would only manu

| Path | What it is |
|---|---|
| `bin/` + `src/` | **Fact producers** (zero-dependency Node CLI). `src/contract.js` is the contract in code; `src/facts/history.js`, `src/facts/coverage.js` and `src/facts/deps.js` are the three measurements (`history` has `--origins` to chase the first introducer and classifies by message + deleted-line + diff, **on by default since `euthyna audit`**; `coverage.js` reads c8/V8 JSON, coverage.py JSON (format 3), Go's text `-coverprofile` **and lcov tracefiles** (the Go profile needs `--source <module root>`: the profile carries no function names, so the producer parses top-level `func` ranges from the gofmt'd source; a file absent from the profile is notEvaluated, never "never executed"; the lcov path needs no `--source` because `FN:` carries the names — they are mangled, and the producer matches a `module::function` query against the `<byte length><name>` tail segments of Rust v0 mangling (what `-C instrument-coverage` implies), since no demangler exists on this machine — each segment is anchored at a boundary and never inside a longer `<length><name>` pair, so a query for `get` does not answer for `cfg_3get`; see `docs/fact-contract.md` §6.2 for the measured record, including that inlining was measured **not** to flatten a called function to 0); `deps.js` reads package-lock.json / Cargo.lock / go.mod, **located from the repository root only** — a monorepo sub-package lock is measured only by `deps --lockfile <path>`, and `audit` does not notice its absence, which `docs/fact-contract.md` §6.3 records with the numbers from the probe). `audit` compares the manifest it measured with the one at `--head` and records `manifest-not-head` when the content differs or the file is absent there. `euthyna audit` (0.4.0, `src/cli.js`) runs history + the full dependency surface in one command — the single entry point the CI recipe and the skill both describe. `src/gate.js` is the six-gate report validator behind `euthyna gate <报告> [--verify] [--allow-exec]` — it parses the skill's 裁定格式, downgrades findings that lack evidence/reproduce or contradict their verdict, and executes nothing but `git` unless the caller consents to interpreters. `src/graph.js` is the Mermaid view behind `--graph` on `euthyna history` and `euthyna audit`: it draws the attribution `history` already emits (the range → the commit that last modified each deleted line → the commit that first introduced its content, plus the `notEvaluated` surface as dotted edges), adds no measurement, and carries states only — colouring a node by risk would make §3.3 of the contract false |
| `bin/` + `src/` | **Fact producers** (zero-dependency Node CLI). `src/contract.js` is the contract in code; `src/facts/history.js`, `src/facts/coverage.js` and `src/facts/deps.js` are the three measurements (`history` has `--origins` to chase the first introducer and classifies by message + deleted-line + diff, **on by default since `euthyna audit`**; `coverage.js` reads **five** formats — c8/V8 JSON, classic istanbul JSON (jest's default provider, nyc: the same `fnMap`/`f` shape, so the symbol locator shares the c8 code path), coverage.py JSON (format 3), Go's text `-coverprofile` **and lcov tracefiles** (the Go profile needs `--source <module root>`: the profile carries no function names, so the producer parses top-level `func` ranges from the gofmt'd source; a file absent from the profile is notEvaluated, never "never executed"; the lcov path needs no `--source` because `FN:` carries the names — they are mangled, and the producer matches a `module::function` query against the `<byte length><name>` tail segments of Rust v0 mangling (what `-C instrument-coverage` implies), since no demangler exists on this machine — each segment is anchored at a boundary and never inside a longer `<length><name>` pair, so a query for `get` does not answer for `cfg_3get`; see `docs/fact-contract.md` §6.2 for the measured record, including that inlining was measured **not** to flatten a called function to 0); `deps.js` reads package-lock.json / Cargo.lock / go.mod, **located from the repository root only** — a monorepo sub-package lock is measured only by `deps --lockfile <path>`, and `audit` does not notice its absence, which `docs/fact-contract.md` §6.3 records with the numbers from the probe). `audit` compares the manifest it measured with the one at `--head` and records `manifest-not-head` when the content differs or the file is absent there. `euthyna audit` (0.4.0, `src/cli.js`) runs history + the full dependency surface in one command — the single entry point the CI recipe and the skill both describe. `src/gate.js` is the six-gate report validator behind `euthyna gate <报告> [--verify] [--allow-exec]` — it parses the skill's 裁定格式, downgrades findings that lack evidence/reproduce or contradict their verdict, and executes nothing but `git` unless the caller consents to interpreters. `src/graph.js` is the Mermaid view behind `--graph` on `euthyna history` and `euthyna audit`: it draws the attribution `history` already emits (the range → the commit that last modified each deleted line → the commit that first introduced its content, plus the `notEvaluated` surface as dotted edges), adds no measurement, and carries states only — colouring a node by risk would make §3.3 of the contract false |
| `test/` | 316 tests via `node --test` (measured 2026-10-20: +20 over the 0.8.0 count — `test/graph.test.js` is 15, `--graph` adds 5 to `test/cli.test.js` including one that drives a real three-commit repository and one that runs `audit --graph` over a merged report), no third-party framework. The pass/skip split is keyed on **tar flavour, not the OS**: `test/bench.test.js` skips `every exploit expectation holds` unless `tar --version` reports bsdtar. Measured on `cdece0f`: ubuntu 245 pass + 1 skip (its log line: *the exploit ground truth is defined against bsdtar, not "tar (GNU tar) 1.35"*), windows 246 pass + 0 skip — and this machine's Git Bash PATH puts `D:\Git\usr\bin\tar.exe` (GNU tar 1.35) ahead of System32's `tar.exe`, so a local run **on `cdece0f`** reported 245 + 1 even though it is Windows (measured again at `4d620cd` for 0.8.0: pwsh 296 pass / 0 skip, Git Bash 295 pass / 1 skip, the difference being that one case — the 245/246 pair above is the `cdece0f` record and 296/295 was the count at `4d620cd`, so read that scope with the number). **The history tests build real git repositories rather than mocking**; the coverage tests carry fixtures shaped like c8, coverage.py, Go profile and lcov tracefile output (`coverage.test.js` + `coverage-go.test.js` + `coverage-lcov.test.js`, the lcov fixtures using mangled names copied from a real `llvm-cov export --format=lcov` run); the deps tests carry fixture lockfiles (npm v1/v2/v3, Cargo.lock, go.mod); the contract tests carry the shell-quoting and render-boundary regressions from PR #1; `git.test.js` carries the error-path quoting and stderr-sanitization regressions from issue #2; `gate.test.js` pins the six-gate validator (including that `--verify` does not run an interpreter command without `--allow-exec`); `cli.test.js` pins the `audit` command, its exit codes, and the manifest-version declaration — content against `--head`, so a PR merge commit whose lockfile is byte-identical to `--head`'s declares nothing while a gitignored lockfile declares itself; the decision itself is an exported pure function so the fail-closed branch (`manifest-unverified`) is reachable from a test; `skill.test.js` pins the written discipline (the six gates, the three verdicts, the `euthyna gate` reference) so weakening the skill turns CI red; `plugin.test.js`, `claude-plugin.test.js` and `hermes-plugin.test.js` pin the three manifest surfaces; `manifest-version.test.js` pins `package.json` against the three plugin manifests (0.5.0 shipped them a release behind and nothing compared them); `i18n.test.js` pins language rendering and report parsing (zh/en); `skill-mirror.test.js` compares the two skill editions against each other — file set, heading sequence (ATX and Setext), fenced blocks, table shapes, and the `euthyna <subcommand>` / `--flag` identifier sets — so an unmirrored discipline change goes red instead of relying on memory, `docs-pair.test.js` holds `docs/` to the same rule (every document paired, and each pair equal on structure and identifiers — not on prose fidelity), and `markdown-structure.test.js` pins the shared scanner itself with seven crafted-document cases, because a legal Markdown form it cannot read makes any comparison pass on that form (neither guard compares word counts, which a translation legitimately needs room for). **The suite count is written down once, in this row**: README's quickstart carried its own number (207) and was 39 tests stale, so it now says only what `npm test` does, and `package.json` defines that as `node --test` |
| `.agents/skills/euthyna/` | **The skill.** Doubles as source and as a project skill root (rank 200), so it is live in this workspace without a restart |
| `bench/` | **The recall benchmark.** `exploits.js` establishes ground truth by execution (18 cases, 8 near-neighbour pairs); `prepare-blind.js` produces answer-free copies with per-round shuffled ids; `adjudicate.js` closes the loop end to end (blind tree → one adjudicator process per case → `collect-verdicts.js` machine validation → `score.js`), with a deterministic `golden` mode (ground truth written into reports — plumbing self-check only, never a real round) for CI; `collect-verdicts.js` machine-validates reports and extracts verdicts without the orchestrator reading report bodies; `score.js` computes the confusion matrix and the pair view; `perf.js` is the `history` scaling baseline. The verified-platform matrix (bsdtar vs GNU tar, p6 skip) lives in `bench/README.md`. Results: `bench/RESULTS.md` (round 1), `bench/RESULTS-round2.md` (round 2: three independent runs per case, zero flips), `bench/RESULTS-round3.md` (round 3: 18 cases, 54 adjudications, two fixture defects caught by adjudicators), `bench/RESULTS-round4.md` (round 4: 54/54, p6b v3 confirmed by first blind adjudication, cross-model run executed — zero flips across two model families); protocols in `bench/README.md`, design pre-registrations in `bench/DESIGN-round3.md` and `bench/DESIGN-round4.md` |
Expand All @@ -697,8 +697,8 @@ is faithful — that is a reading job, and a mechanical stand-in would only manu
| `docs/ci-integration.md` + `-zh` | A paste-ready GitHub Actions recipe that gates a pull request on `euthyna audit`, keyed to the CLI's published exit-code contract (needs `fetch-depth: 0` for the history producer) |
| `docs/case-study-axe-core.md` + `-zh` | Validation run #1: JavaScript (axe-core), 5/5 coarse-screen candidates refuted at the gates |
| `docs/case-study-crewai.md` + `-zh` | Validation run #2: Python (crewAI). **Proved the Python claim**: `history` works on Python repos, and the coverage producer was extended to read coverage.py JSON. 3 candidates → 2 FP + 1 INCONCLUSIVE (pickle, supply-chain-dependent) |
| `docs/case-study-act.md` + `-zh` | Validation run #3: Go (nektos/act, v0.2.89). **Proved the Go claim** for `history` (GHSL-2023-004 fix's deleted lines traced to the introducing commit) and `deps` (go.mod, honestly labelled "declared, not resolved"). **Coverage on Go targets is a confirmed gap** (`go test -coverprofile` text format refused). 4 candidates → 4 FP + 2 observations; the traversal fix re-verified by an executable PoC that also confirmed the artifacts server is unauthenticated. The act clone and PoC live in `.scratch/` (gitignored) |
| `docs/case-study-rust.md` + `-zh` | Validation run #4: Rust (rustls, `v/0.23.44`→`v/0.23.45` = RUSTSEC-2026-0285). **Proved the Rust claim** for `history` (43 facts over the fix's deleted lines) and `deps` (366 Cargo.lock facts). **The project's first TRUE POSITIVE**: a standalone PoC drives `ServerConnection` with hand-built ClientHello wire bytes and shows the pre-fix server accepting a second hello that withdraws the HelloRetryRequest's named suite (ServerHello selects `0x1303` after the retry said `0x1301`), while the fixed tag refuses with `CipherSuiteDifferedOnRetry`. 6 verdicts → 1 TP / 4 FP / 1 INC, validated by `euthyna gate` (exit 0). Rust coverage remains a **confirmed gap**; the target's own test suite cannot run here at all (`aws-lc-sys` build script requires NASM), which is why the PoC is a separate crate on the `ring` provider. Clone, PoC and reports live in `.scratch/` (gitignored) |
| `docs/case-study-act.md` + `-zh` | Validation run #3: Go (nektos/act, v0.2.89). **Proved the Go claim** for `history` (GHSL-2023-004 fix's deleted lines traced to the introducing commit) and `deps` (go.mod, honestly labelled "declared, not resolved"). **Coverage on Go targets was a confirmed gap in that round** (`go test -coverprofile` text format refused), **closed 2026-10-08** — the text profile is now read by the producer, and the run's own record is amended by the revision note in that document rather than rewritten. 4 candidates → 4 FP + 2 observations; the traversal fix re-verified by an executable PoC that also confirmed the artifacts server is unauthenticated. The act clone and PoC live in `.scratch/` (gitignored) |
| `docs/case-study-rust.md` + `-zh` | Validation run #4: Rust (rustls, `v/0.23.44`→`v/0.23.45` = RUSTSEC-2026-0285). **Proved the Rust claim** for `history` (43 facts over the fix's deleted lines) and `deps` (366 Cargo.lock facts). **The project's first TRUE POSITIVE**: a standalone PoC drives `ServerConnection` with hand-built ClientHello wire bytes and shows the pre-fix server accepting a second hello that withdraws the HelloRetryRequest's named suite (ServerHello selects `0x1303` after the retry said `0x1301`), while the fixed tag refuses with `CipherSuiteDifferedOnRetry`. 6 verdicts → 1 TP / 4 FP / 1 INC, validated by `euthyna gate` (exit 0). Rust coverage was a **confirmed gap in that round**, and **closed 2026-10-12** with lcov tracefiles read from a probe crate built under `-C instrument-coverage`; the follow-up note in that document keeps the two records apart rather than rewriting the first. What is still true today: the target's own test suite cannot run here at all (`aws-lc-sys` build script requires NASM), which is why the PoC is a separate crate on the `ring` provider. Clone, PoC and reports live in `.scratch/` (gitignored) |
| `audits/` | **Audit reports, one per run, local-only** (never written into the target repository, and deliberately **not committed** here — the public form is `docs/case-study-*.md`). See `audits/CREWAI_EUTHYNA_AUDIT_2026-09-20.md` and `audits/RUSTLS_EUTHYNA_AUDIT_2026-10-09.md` (gitignored) |
| `tools/fetch-references.js` | Fetches upstream sources on demand into `.refs/` (gitignored). **The repo distributes no third-party files** |
| `tools/check-license-text.mjs` | Verifies `LICENSE` against the canonical Apache-2.0 text, fetched live. **The licence claim is checkable rather than asserted** |
Expand Down
9 changes: 8 additions & 1 deletion docs/fact-contract-zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,9 @@
> 这是 euthyna 的**核心资产**:定义「测量层」交给「判定层」的到底是什么。
> 定位分析的结论是 euthyna 不重造测量,那它就必须定义**接口**——这份文档就是那个接口。
>
> 状态:**设计稿,未经实跑验证**。字段会在第一次 quick 档实跑后修订。
> 状态:**v0.1,已按四次真实验证运行修订**(axe-core、crewAI、act、rustls)。这里原本那句「设计稿,未经实跑
> 验证」写在第一次实跑之前,作为它写下时的状态留在那个提交里。仍未闭合的两件:§6 记着还没有关掉的产出器
> 边界,以及一个契约要求、却没有任何工具产出或校验的字段(§3.2)。
Comment thread
coderabbitai[bot] marked this conversation as resolved.

---

Expand Down Expand Up @@ -142,6 +144,11 @@
> 这是 methodology M6 的核心,也是最容易被 LLM 混为一谈的地方——
> 契约必须在数据结构上强制区分,而不是靠提示词提醒。
> 已有工具没人产出这个字段(八个测量工具全部只看「有没有」,不看「防到什么程度」)。
> **2026-10-20 实测,本仓库也算在内**:这里没有任何产出器产生它,`euthyna gate` 的第 5、第 6 道也没有任何一处读它。
> `src/gate.js` 确实强制的是结构——门禁编号必须落在 1..6,`FAIL` 或 `NOT EVALUATED` 的门禁必须带理由,一条
> TRUE POSITIVE 必须带 `path:L123` 形式的证据、复现命令与影响说明——但那份清单里没有一项读 `effect` 值,
> 所以一份报告可以完全不区分两者就过第 5、第 6 道。这条要求约束的是引擎,不是工具链。把这道缺口补上(给它一个
> 产出器,或者让校验器拒绝没带 `effect` 的 Gate 5/6 通过)是一条未闭合项。

### 3.3 ⛔ 契约里**不许出现**的东西

Expand Down
13 changes: 12 additions & 1 deletion docs/fact-contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,11 @@ English · [中文](fact-contract-zh.md)
> This is euthyna's **core asset**: it defines what exactly the "measurement layer" hands over to the "adjudication layer".
> The positioning analysis concluded that euthyna does not rebuild measurement; it must therefore define the **interface** — and this document is that interface.
>
> Status: **design draft, not yet validated by a real run**. Fields will be revised after the first real run in quick mode.
> Status: **v0.1, revised against four real validation runs** (axe-core, crewAI, act, rustls). The line
> that stood here — "design draft, not yet validated by a real run" — was written before the first of
> them and is kept in that commit as the record of the state it was written in. Two things are still
> open: §6 records the producer boundaries that have not been closed, and one field the contract
> mandates is produced and checked by nothing (§3.2).

---

Expand Down Expand Up @@ -141,6 +145,13 @@ Organized by which step of the adjudication layer consumes each one:
> This is the core of methodology M6, and the thing an LLM most easily conflates —
> the contract must enforce the distinction in the data structure, not rely on a prompt to remember it.
> No existing tool produces this field (all eight measurement tools look only at "whether one exists", never at "how much it actually prevents").
> **Measured 2026-10-20, and this repository included:** no producer here emits it either, and `euthyna gate`
> reads it in no Gate 5 or Gate 6 check. `src/gate.js` does enforce structure — the gate number must fall in
> 1..6, a `FAIL` or `NOT EVALUATED` gate must carry a reason, and a TRUE POSITIVE must carry evidence in
> `path:L123` form, a reproduce command and an impact statement — but not one item on that list reads an
> `effect` value, so a report can pass Gate 5 or Gate 6 without ever drawing the distinction. The requirement
> therefore binds the engine, not the tooling. Closing that gap (a producer, or a check that refuses a
> Gate 5/6 pass carrying no `effect`) is an open item.

### 3.3 ⛔ What must **not** appear in the contract

Expand Down
Loading