diff --git a/.github/workflows/fuzz.yml b/.github/workflows/fuzz.yml index 048c63d..1fd05ef 100644 --- a/.github/workflows/fuzz.yml +++ b/.github/workflows/fuzz.yml @@ -38,6 +38,16 @@ jobs: run: | rustup toolchain install nightly --profile minimal cargo +nightly install cargo-fuzz --locked + # Seeds are the inputs that already found bugs (committed under + # fuzz/seeds, which .gitignore does not exclude). Copying them into the + # generated corpus keeps every regression shape in the rotation without + # committing cargo-fuzz's own (huge) corpus directory. + - name: seed the corpus with known regressions + run: | + mkdir -p "fuzz/corpus/${{ matrix.target }}" + if [ -d "fuzz/seeds/${{ matrix.target }}" ]; then + cp -n "fuzz/seeds/${{ matrix.target }}/"* "fuzz/corpus/${{ matrix.target }}/" || true + fi - name: run ${{ matrix.target }} (60s) run: cargo +nightly fuzz run ${{ matrix.target }} -- -max_total_time=60 -timeout=30 diff --git a/TESTING.md b/TESTING.md index 6f64e65..bca1e60 100644 --- a/TESTING.md +++ b/TESTING.md @@ -122,6 +122,7 @@ traceable test, every test has a traceable requirement. | **A spent free-tier allowance failed silently** (2026-10-25: three consecutive prompts on the local service died with the raw 429 `FreeUsageLimitError` body; the client showed no actionable text and the user retried blindly) | source: `ocserve_llm::free_tier_limit` classifies the body; the prompt path and `surface_run_failure` emit upstream's fixed upsell as `APIError{statusCode:429,isRetryable:false}`, with `ocserve_zen_freetier_total{result="usage_limit"}` | `ocserve-llm::overflow_classifier_tests::free_tier_limit_is_named_and_scoped` (the real body; plain 429/500 are not misclassified), `prompt::free_tier_tests` (terminal + frozen APIError shape), event validated against `EventSessionError` in the vendored contract | | **Free tier rejected every request from our TLS stack AND from a synthetic fixture** (2026-10-25: every free-model prompt 429'd while the same model worked from opencode; five-stack replay with byte-identical body+headers passed only from BoringSSL; separately, the nightly fixture sent a short system prompt + 8 fabricated tools, which the tightened request gate rejects, so the fixture itself was reproducing the failure and hiding it) | opencode.ai is served through hyper-boring/BoringSSL; other providers unchanged (`D-ZEN-TLS`); the live trio builds its system prompt and tool array from the production functions (`ocserve_core::system_prompt`, `ocserve_tools::schemas`), so fixture drift is impossible | `zen_live` trio (nightly, live; 3/3 green 2026-10-25), end-to-end prompts through a real server on three free models returning `TIER_OK`, `D-ZEN-TLS` row | | **A guard control was vacuous because a tool was absent** (2026-10-25, CI on the macOS runner): rule 58 extracted names with `rg`, which is not installed there; the pipeline returned empty, which is exactly what "no uncontracted reads" looks like, so the guard reported its own planted control as *passing*. | rule 58 uses portable `grep -rE` for both extraction and the runtime-read check, and carries a second control (an allowlisted test-only name used at runtime must be flagged) | both controls run on every guard invocation; validated with `rg` removed from PATH | +| **Splice emits unparseable JSON while reporting success** (2026-10-26): the nightly fuzz job failed on `splice_parser`. Triage found **seven** distinct silent-corruption shapes, none of which any green gate could see because the fuzz corpus was untracked and the fuzzer resolved a *different* serde_json feature set than production: raw C0 bytes in keys/values, VT/FF accepted as whitespace (Rust's set, not JSON's), illegal escapes copied through, containers closing in an incomplete phase, nested trailing commas, an empty object closing from Colon phase, and serde's reserved `$serde_json::private::Number` key (only reserved when `arbitrary_precision`, which rolldown enables transitively). | Property test `accepted_output_always_parses` + 11 refusal cases in `splice_parity`; corpus differential re-run on the live DB (244,691 rows, 5,793 blob rows, 0 refused / 0 mismatched); 15-minute fuzz clean, then all 8 targets 60 s each; committed seeds under `fuzz/seeds/splice_parser` copied by the workflow; **guard 59** (`check-fuzz-parity.py`, planted feature-drop and de-listed-target controls) keeps the fuzzer on production's feature set | | **Web UI cannot be published intact** (2026-10-25): `ocserve-http` embedded both packs (35.8 MB) while crates.io caps a `.crate` at 10 MiB, so `cargo install ocserve` stayed on 0.3.1. Strategy: no yank (0.3.0 is already yanked, so yanking 0.3.1 would downgrade installs to 0.2.0) and no dropped assets; the packs are split at zstd frame boundaries into six `ocserve-ui-*` crates with the index kept in `ocserve-http`. | `scripts/shard-pack.py` + `scripts/check-embedded-ui.py` (rule 17: reconstructs each pack, hashes every entry against its manifest, caps each part; planted swap/truncation/oversize); loader cross-part test; payload guard 69.6% of cap; `cargo install ocserve@0.6.9` smoke | | **Final-chunk content was dropped when it carried `finish_reason`** (2026-10-25: the free tier answered in one chunk with finish+content; the reply rendered empty) | `parse_sse_payload` returns content and terminal events separately; live parser, fixture replayer and parser all share it | `finish_with_content_tests::content_on_the_finish_chunk_is_not_dropped` (+ tool-call variant, + split-chunk order) | | **Free-tier error was typed live but `UnknownError` durable** (2026-10-25T17:26:58, live): the prompt path emitted a typed `APIError` live-only, then bailed with the clean upsell text, which dropped the `FreeUsageLimitError` marker; `surface_run_failure` re-classifies from the error chain, so the durable event degraded to `UnknownError` and the client got the variant it does not branch on | the bail keeps the marker in the chain so exactly one durable typed event is emitted | `free_tier_error::spent_free_allowance_is_one_typed_non_retryable_error` asserts exactly ONE durable `session.error` with `name=APIError`, `statusCode=429`, `isRetryable=false` and upstream's upsell text; proven red (`UnknownError`) when the marker is dropped from the bail | diff --git a/TRACEABILITY.md b/TRACEABILITY.md index 42deaee..928ce0b 100644 --- a/TRACEABILITY.md +++ b/TRACEABILITY.md @@ -197,6 +197,7 @@ no-inflation correct), PASS cost (−58.9% weighted) + score (median 1.0 both) | K-PLUGIN-BATTERY | real-plugin battery + options tuple (Phase 2) | All v1 authoring forms loaded through ONE sidecar (`PluginModule.server`, `default` object with `.server`, legacy `default` function, named function export) each asserted to register its hook and fire on trigger; the `[spec, options]` tuple's options **actually reach the server factory** (was silently dropped — `plugin_specs` filtered non-strings). `ocserve-plugin::load`/`load_raw` thread `options` (Null for the plain-string form) and replay it on respawn. Planted control: passing Null where options expected → battery RED, restored GREEN | green | | K-V2CLI | live v2 CLIENT acceptance (Phase 4, GAPS-PLAN Tier 1.2) | The v2 surface was proven only by `pair-v2-check` driving the 2.0.28 CLI **as a server**; no live v2 CLIENT had booted against ocserve (Phase A's own exit criterion). `scripts/v2cli-check.sh` runs the REAL `@opencode/cli@2.0.28` as a client (`--server`) against an `OCSERVE_V2=1` fixture server, asserts it lists a seeded session through `/api/session`, and asserts every route it touched is JSON (not the SPA HTML crash class). Result: **PASS** — client listed the session; 5 routes (info/location/session/model), 0 HTML, `/api/info` JSON. Wired into nightly 6g. | `scripts/v2cli-check.sh` | green | | K-SECREV | adversarial security + reliability review remediation (2026-10-18) | Two subagent reviews (security-review-ninja, sre-reliability-engineer) of the gap-closure work; every finding dispositioned. **Fixed:** F1 subagent configured rules installed in the child (privilege escalation); C1/F2 `task_id` descendant+agent validation (session hijack + self-recursion); H1 child-turn failure surfaced (orphan busy session); F3 `subagent_rules` dedicated column (`session_grant_always` no longer wipes derived rules); F4 data dir 0700 + unit `UMask=0077` (WAL/SHM credential exposure); F5 `reject_open_nonloopback` (open network bind with auth off); F6 `resolve_within` ancestor canonicalization (symlink write escape); F7 unbound `/api/*` → JSON 404 (SPA HTML crash class); M4 guard rule 36 non-vacuous (7 hooks it passed vacuously now anchored); M6 counters for auth denials, task spawns, and rpc dispatch (the new subsystems were silent). **Cleared with evidence:** plugin mutex not held across `task_exec` (no deadlock), no child token double-count, depth cycle guard, v13→v14 migration idempotent+crash-safe, default auth `off` preserved, `check-guards.sh` exit 0. F6 and F7 fixes each carry a proven negative control (fix removed → test red). | `tests/v2_phase_a.rs` (F6 symlink), `tests/auth_metrics.rs` (F7), `prompt::task_perm_tests` (F1), `server_config` tests (F5), `check-v2-hooks.py` selftest (M4) | green | +| K-FUZZ | the nightly fuzz job exercises the shipped parser configuration, not a look-alike, and every accepted splice is valid JSON | The 2026-10-26 fuzz failure was the first time this job had run against a real finding. Seven silent-corruption shapes were fixed; the root cause of *detection* was that `fuzz/Cargo.toml` declared `serde_json = "1"` while production enables `preserve_order` and (transitively, via rolldown) `arbitrary_precision`, so the fuzzer accepted output the server could not parse. Seeds are now committed under `fuzz/seeds/` (the generated corpus is gitignored) and copied in by the workflow. | `check-fuzz-parity.py` (planted feature-drop and de-listed-target controls); `accepted_output_always_parses`; `splice_refuses_malformed_so_dom_path_runs` (11 shapes); corpus differential 244,691 rows 0/0; 15 min fuzz clean; 8/8 targets 60 s | green | | K-DEPS | dependency bumps land with the source change each major requires, and action pins are resolved from upstream tags rather than trusted from a PR title | Dependabot #21 bumped sha2 0.10→0.11 and parking_lot 0.12→0.13, both of which break call sites: sha2's digest output stopped implementing `LowerHex` (5 `format!("{:x}", ..)` sites) and parking_lot removed `const_mutex` because `new` is const. Fixing only the manifest (what the bot does) leaves the tree uncompilable, which is exactly how that PR sat red. Action bumps are verified by resolving the version tag to a commit SHA and checking the release notes for breaking inputs; CodeQL's init/analyze/upload-sarif are bumped as a set because a partial bump is what broke that job on 2026-10-12. | `cargo test --workspace` (741), clippy -D, guards, matrix, replay 26/0, pair 0 divergences; sha2 hex encoding asserted by the existing hash tests | green | | K-CRATES | `cargo install ocserve` ships the same product as the release tarball: both embedded web-UI packs, no dropped assets, on crates.io | `crates.io` caps one `.crate` at 10 MiB and `ocserve-http` carried 35.8 MB, so post-0.3.1 versions could not publish. The packs are split at zstd frame boundaries into six `ocserve-ui-*` crates; the index stays in `ocserve-http` and the loader maps absolute offsets across parts. Guard rule 17 reconstructs every pack, hashes every entry against its manifest and caps each part (planted swap/truncation/oversize); payload guard reports the largest at 69.6% of the cap; live check served a file from every part on both packs; publish order is derived from `Cargo.toml` and `ocserve` publishes last | green | | K-WEBUI-FULL | complete web-UI packs + OCAP2 lazy loader (GAPS-PLAN Tier 1.3) | The Playwright browser acceptance (`scripts/webui-v2-check.py`) proved the pack was incomplete: only absolute index refs were crawled, so ~52 lazily code-split chunks (rolldown dynamic imports referenced by BARE name) and root assets (sw.js/favicon.ico/apple-touch-icon.png) were missing → 404/SPA-HTML. `extract-app.sh` now seeds root assets, captures bare chunk names + slash-less `_assets/` refs, skips SPA-fallback responses, and covers the full closure (v1 944 files/36 MB raw; v2 1678 files/87 MB raw). Pack format is OCAP2 (per-file zstd frames), sharded at frame boundaries across six `ocserve-ui-*` crates because crates.io caps a crate at 10 MiB (`scripts/shard-pack.py`); the loader concatenates the parts and decompresses one file at a time into a **16 MiB bounded LRU** (`bytes::Bytes`) so resident memory is O(16 MiB), not the 87 MB closure (MEMORY.md). Binary unchanged: nothing is dropped to fit the registry. Browser acceptance green (mounted, all API JSON, zero JS errors). | `scripts/webui-v2-check.py`; `embedded_ui::tests` (lazy decompress + entrypoints); nightly 6f browser step; `scripts/check-embedded-ui.py` (rule 17b — pack/manifest entry-count + format consistency, catches the 2026-10-18 mismatched-pack class the browser caught; selftest proves non-vacuous) | green | diff --git a/crates/ocserve-store/src/splice.rs b/crates/ocserve-store/src/splice.rs index 3b05c7e..9d2bc78 100644 --- a/crates/ocserve-store/src/splice.rs +++ b/crates/ocserve-store/src/splice.rs @@ -112,6 +112,15 @@ pub fn push_json_string(out: &mut Vec, s: &str) { out.push(b'"'); } +/// JSON's own whitespace set: space, tab, LF, CR. `char::is_ascii_whitespace` +/// also accepts VT (0x0b) and FF (0x0c), which are ILLEGAL outside strings, so +/// skipping them let such input through and the splice then emitted JSON serde +/// cannot parse (fuzz-found 2026-10-26, `{"iiii"\x0c:{}}`). +#[inline] +fn is_json_ws(b: u8) -> bool { + matches!(b, b' ' | b'\t' | b'\n' | b'\r') +} + /// Scan a JSON string starting at `b[start] == b'"'`; returns the index one /// past the closing quote. fn scan_string(b: &[u8], start: usize) -> Option { @@ -176,10 +185,26 @@ fn fnv1a(bytes: &[u8]) -> u64 { fn needs_string_normalize(raw: &[u8]) -> bool { let mut i = 0usize; while i < raw.len() { + // A raw C0 byte is illegal in a JSON string and cannot be copied + // through: the splice would report success while emitting output + // serde rejects (fuzz-found 2026-10-26: `{"\x04":1}`). Returning + // true delegates to serde, whose parse fails, so the caller falls + // back to the DOM path like any other unprovable input. + if raw[i] < 0x20 { + return true; + } if raw[i] == b'\\' { match raw.get(i + 1) { + // Non-canonical: serde re-encodes these, so delegate. Some(b'/') | Some(b'u') => return true, - _ => i += 2, + // Canonical short escapes serde emits verbatim. + Some(b'"') | Some(b'\\') | Some(b'b') | Some(b'f') | Some(b'n') | Some(b'r') + | Some(b't') => i += 2, + // Anything else is an ILLEGAL escape (`\{`, `\'`). Copying + // it produced output serde rejects while claiming success + // (fuzz-found 2026-10-26). Delegate to serde, whose parse + // fails, so the caller falls back to the DOM path. + _ => return true, } } else { i += 1; @@ -233,7 +258,7 @@ fn is_canonical_number(tok: &[u8]) -> bool { /// in `{"a":1,"b":2}`. fn emit_value(b: &[u8], pos: usize, out: &mut Vec) -> Option { let mut i = pos; - while i < b.len() && (b[i] as char).is_ascii_whitespace() { + while i < b.len() && is_json_ws(b[i]) { i += 1; } match *b.get(i)? { @@ -256,13 +281,31 @@ fn emit_value(b: &[u8], pos: usize, out: &mut Vec) -> Option { // in the fixture corpus (the `input: {}` shape). let mut saw_member = false; loop { - while i < b.len() && (b[i] as char).is_ascii_whitespace() { + while i < b.len() && is_json_ws(b[i]) { i += 1; } match *b.get(i)? { c if c == close => { - if is_obj && phase == Phase::Key && saw_member { - return None; // `{"a":1,}` — trailing comma + // A close is valid only where a value is genuinely + // complete: right after a value (Comma) or when the + // container is still empty. In Key/Value phase WITH a + // member already seen, the previous token was a comma, + // so closing would emit a trailing comma (`{"a":1,}`, + // `[1,]`) that does not parse (fuzz-found 2026-10-26). + // Empty container: no member yet AND we are where the + // first token belongs (Key for `{}`, Value for `[]`). + // `phase` alone is not enough: `{":2"}` sits in Colon + // phase with no member and would wrongly close + // (fuzz-found 2026-10-26). + let empty = !saw_member + && if is_obj { + phase == Phase::Key + } else { + phase == Phase::Value + }; + let can_close = phase == Phase::Comma || empty; + if !can_close { + return None; } out.push(close); return Some(i + 1); @@ -297,6 +340,7 @@ fn emit_value(b: &[u8], pos: usize, out: &mut Vec) -> Option { return None; } i = emit_value(b, i, out)?; + saw_member = true; phase = Phase::Comma; } } @@ -353,7 +397,7 @@ fn emit_value(b: &[u8], pos: usize, out: &mut Vec) -> Option { /// column key's value, which we re-emit ourselves). fn skip_value(b: &[u8], pos: usize) -> Option { let mut i = pos; - while i < b.len() && (b[i] as char).is_ascii_whitespace() { + while i < b.len() && is_json_ws(b[i]) { i += 1; } match b.get(i)? { @@ -417,7 +461,7 @@ pub fn compact_splice( ) -> bool { let b = src.as_bytes(); let mut i = 0usize; - while i < b.len() && (b[i] as char).is_ascii_whitespace() { + while i < b.len() && is_json_ws(b[i]) { i += 1; } if b.get(i) != Some(&b'{') { @@ -438,7 +482,7 @@ pub fn compact_splice( let mut ok = true; loop { - while i < b.len() && (b[i] as char).is_ascii_whitespace() { + while i < b.len() && is_json_ws(b[i]) { i += 1; } match b.get(i) { @@ -469,7 +513,7 @@ pub fn compact_splice( } }; i += raw_key.len(); - while i < b.len() && (b[i] as char).is_ascii_whitespace() { + while i < b.len() && is_json_ws(b[i]) { i += 1; } if b.get(i) != Some(&b':') { @@ -504,21 +548,43 @@ pub fn compact_splice( } scratch.put_key(k) } - Err(_) => None, + // Not a valid JSON string (raw C0 byte, bad escape, lone + // surrogate). Copying it verbatim would emit output serde + // cannot parse, so refuse and let the DOM path decide. + Err(_) => { + ok = false; + break; + } } } else { - match std::str::from_utf8(&raw_key[1..raw_key.len() - 1]) { - Ok(k) => { + let inner = &raw_key[1..raw_key.len() - 1]; + // A raw C0 byte in a key is illegal JSON and the key bytes are + // copied verbatim below, so validate before accepting (the same + // class the value path refuses in `needs_string_normalize`). + match std::str::from_utf8(inner) { + Ok(k) if k.as_bytes().iter().all(|b| *b >= 0x20) => { if !scratch.seen(fnv1a(k.as_bytes())) { ok = false; // duplicate key — DOM path decides break; } Some(k) } - // a non-UTF8 key cannot appear in valid JSON - Err(_) => None, + // non-UTF8 or a raw control byte: not valid JSON + _ => { + ok = false; + break; + } } }; + // serde_json reserves `$serde_json::private::*` keys for its own + // number representation when `arbitrary_precision` is enabled (it IS + // enabled in production, pulled in by rolldown). A user object that + // carries such a key makes the DOM path fail, so copying it through + // would emit output that does not parse (fuzz-found 2026-10-26). + if key_ref.is_some_and(|k| k.starts_with("$serde_json::private")) { + ok = false; + break; + } let key: Option = key_ref.and_then(Col::of); if !first { @@ -588,7 +654,7 @@ pub fn compact_splice( work.extend_from_slice(b"\":"); push_json_string(&mut work, v); } - while i < b.len() && (b[i] as char).is_ascii_whitespace() { + while i < b.len() && is_json_ws(b[i]) { i += 1; } if i != b.len() { diff --git a/crates/ocserve-store/tests/splice_parity.rs b/crates/ocserve-store/tests/splice_parity.rs index 0308a83..e71bbac 100644 --- a/crates/ocserve-store/tests/splice_parity.rs +++ b/crates/ocserve-store/tests/splice_parity.rs @@ -103,9 +103,10 @@ fn splice_refuses_duplicate_keys_which_serde_collapses() { #[test] fn splice_tolerates_trailing_comma_that_serde_rejects() { // serde rejects `{"a":1,}` so no DOM oracle exists for it. The splice - // still emits valid JSON — strictly safer than the old behavior (which - // errored the whole request), so the contract is "valid output", not - // "byte-identical". + // emits valid JSON for a TOP-LEVEL trailing comma by not copying it, + // which is safer than erroring the whole request; the contract here is + // "valid output", not "byte-identical". NESTED trailing commas are + // refused so the splice can never emit `[1,]`. let mut out = Vec::new(); let mut scratch = SpliceScratch::default(); assert!(compact_splice( @@ -132,6 +133,31 @@ fn splice_refuses_malformed_so_dom_path_runs() { r#"{,}"#, // leading comma "", "{", + // Raw C0 control bytes inside strings are illegal JSON. The splice + // used to copy them through and report success (fuzz-found + // 2026-10-26), so the "accepted output is valid JSON" invariant broke. + "{\"\u{4}\":44444}", + "{\"a\":\"x\u{1}y\"}", + // VT/FF are Rust whitespace but illegal JSON outside strings, so + // skipping them produced output serde rejects (fuzz-found 2026-10-26). + "{\"iiii\"\u{c}:{}}", + "{\u{c}\"a\":1}", + "{\"a\":\u{b}1}", + // Illegal escape sequences (`\{`, `\'`, trailing `\`) are copied + // through by a naive scanner; the output then does not parse. + r#"{"a":"\{/"}"#, + r#"{"a":"x\'y"}"#, + // serde_json reserves this key when arbitrary_precision is on, so the + // DOM path fails and the splice must not claim success. + r#"{"$serde_json::private::Number":0}"#, + // Object in Colon phase with no member yet must not close. + r#"{"" :{": 2"}}"#, + // A container that closes before its value is complete. + r#"{"a":}"#, + r#"{"a"}"#, + r#"{"a":{"b":}}"#, + r#"{"a":[1,]}"#, + "{\"itr]ed\"\r :{\"\" :\t\n}}", ]; for b in bad { let mut out = Vec::new(); @@ -188,6 +214,37 @@ fn splice_output_is_valid_json() { assert_eq!(v["text"], "a,b}c"); } +/// The invariant the fuzzer enforces, encoded for `cargo test`: whenever the +/// splice accepts, serde must parse its output. Seven distinct silent-corruption +/// shapes were found by fuzzing on 2026-10-26 (raw control bytes, VT/FF +/// whitespace, illegal escapes, incomplete containers, nested trailing commas, +/// empty-container mis-close, and serde's reserved `$serde_json::private` key); +/// all are now seeded into `fuzz/seeds/splice_parser`. +#[test] +fn accepted_output_always_parses() { + let cases = [ + r#"{"a":1}"#, + r#"{"a":[1,{"b":null},true]}"#, + r#"{"a":"\u00e9","b":"\n"}"#, + "{\"a\":\"\u{4}\",\"b\":1}", + "{\"a\":1}\u{c}", + r#"{"a":"\{/"}"#, + r#"{"a":}"#, + r#"{"a":[1,]}"#, + r#"{"" :{": 2"}}"#, + r#"{"$serde_json::private::Number":0}"#, + r#"{"a":1,}"#, + ]; + for src in cases { + let mut out = Vec::new(); + let mut scratch = SpliceScratch::default(); + if compact_splice(src, "i", "s", Some("m"), &mut out, &mut scratch) { + serde_json::from_slice::(&out) + .unwrap_or_else(|e| panic!("accepted {src:?} but emitted invalid JSON: {e}")); + } + } +} + /// **Corpus differential** — the actual ship gate. Runs only when /// `OCSERVE_SPLICE_DB` is set; CI runs it manually against a real db (and the /// pre-commit gate records the run in PERF-10X). Every stored `info` and diff --git a/fuzz/Cargo.toml b/fuzz/Cargo.toml index 65c3394..66a5bde 100644 --- a/fuzz/Cargo.toml +++ b/fuzz/Cargo.toml @@ -9,7 +9,11 @@ cargo-fuzz = true [dependencies] libfuzzer-sys = "0.4" -serde_json = "1" +# Match the real workspace: `preserve_order` (key order on the wire) and +# `arbitrary_precision` (enabled transitively by rolldown in production). The +# fuzzer was previously exercising a DIFFERENT serde configuration than the +# one we ship, which hid the reserved-key behaviour. +serde_json = { version = "1", features = ["preserve_order", "arbitrary_precision"] } [dependencies.ocserve-llm] path = "../crates/ocserve-llm" diff --git a/fuzz/seeds/splice_parser/ctl-byte-key b/fuzz/seeds/splice_parser/ctl-byte-key new file mode 100644 index 0000000..252d088 --- /dev/null +++ b/fuzz/seeds/splice_parser/ctl-byte-key @@ -0,0 +1 @@ +{"":44444} \ No newline at end of file diff --git a/fuzz/seeds/splice_parser/ctl-byte-value b/fuzz/seeds/splice_parser/ctl-byte-value new file mode 100644 index 0000000..e243ad6 --- /dev/null +++ b/fuzz/seeds/splice_parser/ctl-byte-value @@ -0,0 +1 @@ +{"a":"xy"} \ No newline at end of file diff --git a/fuzz/seeds/splice_parser/empty-colon b/fuzz/seeds/splice_parser/empty-colon new file mode 100644 index 0000000..2030495 --- /dev/null +++ b/fuzz/seeds/splice_parser/empty-colon @@ -0,0 +1 @@ +{"" :{": 2"}} \ No newline at end of file diff --git a/fuzz/seeds/splice_parser/illegal-escape b/fuzz/seeds/splice_parser/illegal-escape new file mode 100644 index 0000000..d7bf102 --- /dev/null +++ b/fuzz/seeds/splice_parser/illegal-escape @@ -0,0 +1 @@ +{"a":"\{/"} \ No newline at end of file diff --git a/fuzz/seeds/splice_parser/incomplete-value b/fuzz/seeds/splice_parser/incomplete-value new file mode 100644 index 0000000..a81d4b6 --- /dev/null +++ b/fuzz/seeds/splice_parser/incomplete-value @@ -0,0 +1 @@ +{"a":} \ No newline at end of file diff --git a/fuzz/seeds/splice_parser/nested-trailing-comma b/fuzz/seeds/splice_parser/nested-trailing-comma new file mode 100644 index 0000000..867076f --- /dev/null +++ b/fuzz/seeds/splice_parser/nested-trailing-comma @@ -0,0 +1 @@ +{"a":[1,]} \ No newline at end of file diff --git a/fuzz/seeds/splice_parser/reserved-serde-key b/fuzz/seeds/splice_parser/reserved-serde-key new file mode 100644 index 0000000..512329a --- /dev/null +++ b/fuzz/seeds/splice_parser/reserved-serde-key @@ -0,0 +1 @@ +{"$serde_json::private::Number":0} \ No newline at end of file diff --git a/fuzz/seeds/splice_parser/vt-ff-ws b/fuzz/seeds/splice_parser/vt-ff-ws new file mode 100644 index 0000000..56335d2 --- /dev/null +++ b/fuzz/seeds/splice_parser/vt-ff-ws @@ -0,0 +1 @@ +{"iiii" :{}} \ No newline at end of file diff --git a/scripts/check-fuzz-parity.py b/scripts/check-fuzz-parity.py new file mode 100755 index 0000000..d09df80 --- /dev/null +++ b/scripts/check-fuzz-parity.py @@ -0,0 +1,118 @@ +#!/usr/bin/env python3 +"""Fuzz-parity guard (rule 59). + +Two silent-failure modes in the fuzz workspace: + +1. The fuzzer resolves `serde_json` with a different feature set than the + shipped workspace. On 2026-10-26 the fuzzer had neither `preserve_order` nor + `arbitrary_precision`; splicing an object carrying + `$serde_json::private::Number` "succeeded" under the fuzzer and failed in + production, so the fuzzer reported green on code that emitted unparseable + JSON. +2. A target with committed regression seeds is missing from the CI matrix, so + those seeds never run. + +`fuzz/corpus/` is gitignored (cargo-fuzz's own corpus is huge), so seeds that +must survive live under `fuzz/seeds//` and are copied in by the +workflow. This check keeps both facts true. + +Usage: check-fuzz-parity.py [--selftest] +""" +import pathlib +import re +import sys + +ROOT = pathlib.Path(__file__).resolve().parent.parent + + +def features(text: str, crate: str) -> set[str]: + """Feature names declared for `crate` in a manifest (workspace or plain).""" + # Match the dependency line through its closing brace on the same or the + # following lines; `features = [..]` is the only list we care about. + m = re.search(rf'^{crate}\s*=\s*(\{{.*?\}}|"[^"]*")', text, re.S | re.M) + if not m: + return set() + block = m.group(1) + fm = re.search(r'features\s*=\s*\[([^\]]*)\]', block, re.S) + return set(re.findall(r'"([a-z_]+)"', fm.group(1))) if fm else set() + + +def check(root: pathlib.Path = ROOT) -> list[str]: + problems = [] + main = (root / "Cargo.toml").read_text() + fuzz_path = root / "fuzz/Cargo.toml" + if not fuzz_path.exists(): + return [f"missing {fuzz_path}"] + fuzz = fuzz_path.read_text() + + # Production feature set = the workspace declaration PLUS features that + # arrive transitively. `arbitrary_precision` is enabled by + # `rolldown_common` (verify: cargo tree -e features -p ocserve | grep + # arbitrary_precision), and cargo unifies features across the graph, so the + # shipped server always parses with it on. The fuzzer must match. + TRANSITIVE = {"arbitrary_precision"} + main_feats = features(main, "serde_json") | TRANSITIVE + fuzz_feats = features(fuzz, "serde_json") + missing = main_feats - fuzz_feats + if missing: + problems.append( + f"fuzz serde_json lacks workspace features {sorted(missing)}: the " + "fuzzer would exercise a different parser than production" + ) + + wf_path = root / ".github/workflows/fuzz.yml" + if not wf_path.exists(): + return problems + ["missing .github/workflows/fuzz.yml"] + wf = wf_path.read_text() + targets = re.findall(r'^name = "([a-z_]+)"$', fuzz, re.M) + if not targets: + problems.append("no fuzz targets parsed from fuzz/Cargo.toml") + for t in targets: + seeds = root / "fuzz/seeds" / t + if seeds.is_dir() and not re.search(rf"^\s*-\s*{t}\s*$", wf, re.M): + problems.append(f"{t} has committed seeds but is absent from the CI matrix") + return problems + + +def main() -> int: + if "--selftest" in sys.argv: + import shutil + import tempfile + + with tempfile.TemporaryDirectory() as td: + tmp = pathlib.Path(td) + shutil.copytree(ROOT / "fuzz", tmp / "fuzz", ignore=shutil.ignore_patterns("target", "corpus", "artifacts")) + shutil.copytree(ROOT / ".github", tmp / ".github") + shutil.copy(ROOT / "Cargo.toml", tmp / "Cargo.toml") + if check(tmp): + print("FAIL(selftest): clean tree flagged") + return 1 + # plant 1: drop a feature + f = tmp / "fuzz/Cargo.toml" + f.write_text(re.sub(r',\s*"arbitrary_precision"', "", f.read_text())) + if not check(tmp): + print("FAIL(selftest): feature mismatch not detected") + return 1 + shutil.copy(ROOT / "fuzz/Cargo.toml", f) + # plant 2: de-list a seeded target + w = tmp / ".github/workflows/fuzz.yml" + w.write_text(w.read_text().replace(" - splice_parser\n", "")) + if not check(tmp): + print("FAIL(selftest): de-listed seeded target not detected") + return 1 + print("selftest: PASS (feature mismatch and de-listed seeded target detected)") + return 0 + + problems = check() + if problems: + for p in problems: + print(f"FAIL: {p}") + return 1 + fuzz = (ROOT / "fuzz/Cargo.toml").read_text() + n = len(re.findall(r'^name = "([a-z_]+)"$', fuzz, re.M)) + print(f"ok ({n} fuzz targets; serde_json features match the workspace)") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/check-guards.sh b/scripts/check-guards.sh index 5135bd1..68062d1 100755 --- a/scripts/check-guards.sh +++ b/scripts/check-guards.sh @@ -128,6 +128,11 @@ # 58. every runtime `OCSERVE_*` read is in the env contract, and the committed # env reference matches the contract exactly (an undocumented knob is # undiscoverable; a stale reference lies) [plant: bogus read + stale doc] +# 59. the fuzz workspace resolves serde_json with production's features +# (arbitrary_precision arrives transitively from rolldown), and every +# target with committed seeds is in the CI matrix. The fuzzer previously +# ran a different parser and hid seven silent-corruption shapes found on +# 2026-10-26 [suite: check-fuzz-parity.py --selftest] # 51. hook/RPC dispatch takes a Handle, never the owner lock (holding it # across the await re-serialized every session; 2026-10-23 review H3) # [plant: owner-lock trigger call] @@ -914,6 +919,17 @@ else echo "ok (no interpreter spawns; planted node detected)" fi +echo "== guard: fuzz workspace matches production and seeds are wired (rule 59) ==" +if python3 scripts/check-fuzz-parity.py --selftest >/dev/null 2>&1; then + if out=$(python3 scripts/check-fuzz-parity.py 2>&1); then + echo "$out" + else + echo "$out"; echo "FAIL: fuzz parity (rule 59)"; fail=1 + fi +else + echo "FAIL: fuzz-parity planted controls not detected (rule 59 vacuous)"; fail=1 +fi + echo "== guard: docs are not stale (rule 31) ==" # Docs are the spec (AGENTS §1); nothing checked them, so dead references # accumulated (the 2026-10-13 audit: three files referenced but gone, a stale