Skip to content

Make #237's build and OCaml full-API checks pass honestly: ABI status ledger, property rewrite, defect fixes - #239

Merged
hyperpolymath merged 10 commits into
codex/real-header-refc-20261116from
claude/trusting-fermat-2msgw7
Nov 18, 2026
Merged

hyperpolymath merged 10 commits into
codex/real-header-refc-20261116from
claude/trusting-fermat-2msgw7

Conversation

@hyperpolymath

@hyperpolymath hyperpolymath commented Nov 17, 2026 •

Copy link
Copy Markdown
Owner

This is stacked on #237 (codex/real-header-refc-20261116). It targets the two failing code checks there: build (0.8.0) and ocaml-full-api. Neither check was weakened, and no exports were stubbed.

Reproduced first

Both failures were reproduced locally at bb98fbc with the pinned toolchains (Idris2 0.8.0, Idris2 b6cbc25d, Zig 0.15.2, OCaml 5.1.1):

1. C ABI status ledger (bindings/c/include/proven-abi.tsv)

proven.h declares 162 functions. The qualified RefC library:

  • defines 9 of them with matching signatures;
  • exports 2 under conflicting signatures: proven_init (header int32_t(void), Zig ?*Handle) and proven_is_initialized;
  • exports 7 symbols that are not in the header at all. One is proven_header_codegen_main, the RefC main, which leaks out through -Dmain=. The new checker caught it on its first run.

That leaves 151 owed. Each row is classified as exported, conflict, owed or undeclared, and names its backend: idris-refc or zig-legacy.

scripts/check-abi-ledger.sh checks the ledger in both directions against the header (after cc -E) and against nm of the real library:

  • An exported row that is missing from the library fails.
  • An owed row that appears in the library fails.
  • A library export with no ledger row fails.

test-header-bridge.sh (the "Real Idris header ABI" job) runs the checker together with planted controls: 1 accept, 6 reject, 1 NOT RUN.

The OCaml full binding now binds only the 7 exported symbols it previously declared. Before this change it could not load at all. full-api-audit.sh now checks that the binding ⊆ ledger-exported ⊆ nm, and that owed ∩ nm = ∅. Planted controls for this audit were run locally:

  • binding proven_deinit → FAIL
  • a library that exports proven_deinit → FAIL (owed-present)

The recommendation is in docs/adr/2026-11-17-c-abi-status-ledger.adoc. In short: narrow first, as done here. Then implement owed rows only through the Idris core, in consumer-weighted slices, starting with SafeMath's checked arithmetic, which is declared by 84–87 of the 117 binding directories. After that, move the declaration into an Idris2 module that generates proven.h, the ledger and each binding's raw declarations.

2. Property suite rewritten against the real APIs

All 28 failing modules were rewritten. Where a property holds definitionally it is now a proof. Closed cases that don't reduce are labelled finite execution checks. Properties that are true but unproved become named …Obligation : Type propositions, following the convention in #237's SafeMath work.

Every bodyless 0 prop_… declaration is gone; each one was acting as an unchecked postulate. Tautologies, and properties the API cannot express, are retired with a comment explaining why rather than deleted silently. Main no longer claims that compiling the suite proves every property.

3. Packaging

idris2 --install installed neither SafeJWT.Types nor SafeRegex.{Parser,Matcher}, SafeSQL.Builder, SafeXML.Parser or SafeYAML.Parser. So for every package user, import Proven.SafeJWT/SafeRegex/SafeSQL/SafeXML/SafeYAML failed. All six are now listed in proven.ipkg. No file content or header changed.

4. Library defects the checks exposed

Fixed, each with its check now passing and a regression case added:

  • SafeHeader: a header name containing CR/LF was accepted under the default options, so header injection worked through the name. NUL in a value and : in a name were also accepted.
  • SafePassword: every accepted password crashed at runtime on a bodyless trustCheckPolicy postulate. MeetsPolicy now carries erased evidence that checkPolicy returned [].
  • SafeJWT: two-segment tokens were accepted.
  • SafeJson: trailing commas were accepted. The full parser also rejected "1" and "[1]" because its fuel ran out.
  • SafePhone: +44… was split as country code 4; letters were silently stripped; country code 0 was accepted.
  • SafeBase64: "UTF-8" encoding actually truncated each Char to its low byte.
  • SafeUUID: hex version nibbles were decoded wrongly.
  • SafeEmail: every dotted local part was rejected.
  • SafeFile: the empty path was accepted.
  • SafeArgs: required positionals could never be satisfied.

Strict known-defect ledger (tests/properties/KnownDefects.idr). Each entry still reproduces. If one stops reproducing, the run fails until the entry is reclassified.

  • toml-parser-stub: the parser is a stub that returns Ok [] for everything.
  • yaml-block-collections-rejected
  • xml-doctype-unparsed: billion-laughs and external-DTD input is rejected only by accident, and allowDTD has no effect.
  • cookie-negative-max-age: an owner call between rejecting ≤ 0 and rejecting < 0.

5. CI step fix

The "Type check all modules" step ran from the repository root. From there, src/Proven/Core.idr is read as module src.Proven.Core, so the step could not pass for any module. It had never been reached before. It now runs from src/, the same way proof-check does. All 12 modules check.

Remote evidence at head 9e4e542

idris2-ci.yml and ocaml-ci.yml only trigger for PRs targeting main, so they were dispatched on this branch with workflow_dispatch.

  • Idris 2 CI run 44188825412: build (0.8.0) success, proof-check (0.8.0) success.
  • OCaml CI run 44188830214: ocaml-full-api success. ocaml-build, ocaml-symbol-audit, ocaml-smoke, ocaml-detachable-build and detachability-guard also succeeded.
  • pull_request runs at the same head:
    • Real Idris header ABI, including the ledger check: success.
    • Agda/Cubical/Lean/Rocq replay: success.
    • Idris → RefC → Zig: success.
    • verify-idris-build, enforce-idris-only, pr-fuzzing, semgrep, Sonar and the manifest checks: success.

secret-scanner and spark-theatre-gate ended with zero runtime. That is the known startup_failure pattern from the narrowed Actions allow-list, and the diff does not touch either workflow.

Local evidence

All runs are at the head of this branch:

  • build (0.8.0): every step passes.
    • totality regressions: exit 0
    • build: 0 errors
    • install
    • diagnose: 29/29 modules, 0 errors
    • tests.ipkg build
    • execute: 315 PASS, 20 ledgered expectations reproduce, exit 0
    • type-check: 12/12
  • proof-check (0.8.0): 29/29 *.Proofs modules check.
  • OCaml:
    • ocaml-build passes.
    • ocaml-symbol-audit: 5/5.
    • ocaml-smoke passes.
    • ocaml-full-api: the probe loads, and the audit reports Required=7 Present=7 Missing=0 LedgerOwed=151.
    • Alcotest: 3/3.
    • Detachability guard: OK.

Not addressed: owner decisions

Both are reported on #230.

  • Licence headers: five src/ files carry SPDX-License-Identifier: Palimpsest-MPL-1.0, and Enforce Trustfile Policies fails on them. They are untouched.
  • Retired format and stray files: 0-AI-MANIFEST.a2ml and Trustfile.a2ml use the retired A2ML format, and there are two empty root files named Off and On. These are reported, not converted.

🤖 Generated with Claude Code

https://claude.ai/code/session_01J4QvDi2XA8N9NEXSvCspMr

proven.h declares 162 functions; the qualified RefC library defines 9 of
them with matching signatures, exports 2 more under conflicting signatures
(proven_init, proven_is_initialized) and 7 symbols the header omits,
including proven_header_codegen_main leaking through -Dmain. The eager
OCaml binding declared 146 and could not load (undefined proven_deinit).

bindings/c/include/proven-abi.tsv records each symbol as exported,
conflict, owed or undeclared, with an idris-refc or zig-legacy backend.
scripts/check-abi-ledger.sh checks it against the preprocessed header and
nm of the real library, in both directions, and test-header-bridge.sh runs
it with planted controls (1 accept, 6 reject, 1 not-run). The OCaml binding
now binds the 7 ledger-exported symbols it declared; full-api-audit.sh
enforces binding <= ledger-exported <= nm and owed disjoint from nm.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J4QvDi2XA8N9NEXSvCspMr
All 28 failing modules (241 diagnostics) are restated against the real
exported functions. Properties that hold definitionally are proofs;
closed cases that do not reduce are labelled finite execution checks;
true but unproved properties are named Obligation types with OWED notes.
Former bodyless 0-quantity declarations, which acted as unchecked
postulates, are gone. Tautologies and properties the API cannot express
are retired with an explanatory comment rather than silently deleted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J4QvDi2XA8N9NEXSvCspMr
idris2 --build compiled these transitively, but --install only installs
listed modules, so `import Proven.SafeJWT` (and SafeRegex, SafeSQL,
SafeXML, SafeYAML) failed for every package user with "module not found".
List SafeJWT.Types, SafeRegex.Parser, SafeRegex.Matcher, SafeSQL.Builder,
SafeXML.Parser and SafeYAML.Parser. No file content or header changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J4QvDi2XA8N9NEXSvCspMr
…erties

KnownDefects names four library defects the suite reproduces: the TOML
parser stub, YAML block collections rejected, negative cookie Max-Age, and
XML DOCTYPE never parsed. knownDefect takes the correct expectation; it
reports while the defect reproduces and fails the run once it stops, so a
fix cannot go unrecorded. Removing a constructor breaks every remaining use.

With the parsers now installed, the XML and YAML obligations gain execution
instances. YAML security properties use flow syntax so that rejections are
not an artefact of the block-collection defect. Main no longer claims that
compilation proves every property.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J4QvDi2XA8N9NEXSvCspMr
Defects reproduced by the rewritten property checks, each now passing
with a regression case added:
- SafeUUID: hex-letter version nibbles mapped through ord - ord '0'
  (a -> 49); use hexToNibble (RFC 9562 s4.2).
- SafeEmail: parseAtom rejected every dotted local part because '.' is
  not atext; accept dot-atom-text (RFC 5322 s3.2.3), still rejecting
  leading, trailing and consecutive dots.
- SafeFile: validatePath accepted the empty path (POSIX.1-2017 XBD s4.13).
- SafeArgs: required positionals were looked up under "<positional>"
  instead of "positional-N", so they could never be satisfied.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J4QvDi2XA8N9NEXSvCspMr
…JWTs

- SafeHeader: under default options a name was never checked for
  characters, so "X-A\r\nSet-Cookie" rendered a second header line. Names
  now reject CR/LF as HeaderInjection and any other CTL, SP, DEL or ':'
  in every mode (RFC 9110 s5.1, RFC 9112 s5.1); strictTokens still demands
  a pure token. Values reject NUL (RFC 9110 s5.5). The bodyless, and false,
  renderedHeaderSafe and headerValueNoCRLF become named Obligation types.
- SafePassword: every accepted password aborted on the bodyless
  trustCheckPolicy. MeetsPolicy now carries erased evidence that
  checkPolicy returned [], built by the new decideMeetsPolicy.
- SafeJWT: splitToken accepted "h.p"; a JWS compact serialisation has
  exactly three segments (RFC 7515 s7.1; unsecured is "h.p.").

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J4QvDi2XA8N9NEXSvCspMr
… UTF-8

- SafeJson: the full parser accepted "[1,]" and "{\"a\":1,}" (RFC 8259
  s4, s5). It also rejected "1" and "[1]": its fuel was the input length,
  which ran out before a number's digits were read; fuel is now 2n + 2.
  The duplicate Show JsonValue instance is removed and the remaining one
  is total.
- SafePhone: "+44 20 7946 0958" split as country code 4, letters were
  silently stripped, and code 0 was accepted. Country codes are now split
  by prefix (1 and 7, the 2-digit set, otherwise 3; E.164 codes are
  prefix-free), only visual separators are allowed, and isValidE164
  agrees with parseE164.
- SafeBase64: encode kept each Char's low byte though documented as UTF-8;
  it now encodes RFC 3629 UTF-8, and the string decoders reject
  ill-formed UTF-8 with the new InvalidUTF8 error.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J4QvDi2XA8N9NEXSvCspMr
From the repository root, `idris2 --check src/Proven/Core.idr` reads the
file as module src.Proven.Core and fails on the module-name mismatch, so
this step could not pass for any module; it had never been reached
because earlier steps failed first. Check the same twelve modules from
src/ with module-relative paths, as the proof-check job already does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J4QvDi2XA8N9NEXSvCspMr
Owner ruling, confirmed directly by the owner on 2026-11-18: change
exactly the first-line SPDX tag of these five files, which the rest of
src/ already moved to MPL-2.0 on 2026-05-22. Palimpsest-MPL-1.0 is not
on the standards licence register, and Enforce Trustfile Policies failed
on these files. No other line changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J4QvDi2XA8N9NEXSvCspMr
Owner ruling, confirmed 2026-11-18: a non-positive Max-Age is not an
error but means "expire immediately" (RFC 6265 s5.2.2). withMaxAge stores
it as 0 and the renderer clamps at 0, so no path can emit Max-Age=-1,
which the s4.1.1 server grammar has no form for. buildDeleteCookie
already uses Max-Age=0. The cookie-negative-max-age known-defect entry is
retired as a resolved design question; three ordinary checks replace it,
and the first fails on the previous library.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J4QvDi2XA8N9NEXSvCspMr

Copy link
Copy Markdown
Owner Author

Local evidence, not CI

Head: 1a7da23. The GitHub Actions minutes for this month are used up, so no workflow has run on the two new commits. Everything below was run in a Linux x86_64 container.

New commits (both owner rulings, confirmed directly by the owner on 2026-11-18)

  • 18c1c0a: five first-line SPDX tags changed from Palimpsest-MPL-1.0 to MPL-2.0. The files are SafeRegex/{Parser,Matcher}, SafeSQL/Builder, SafeXML/Parser and SafeYAML/Parser. No other line changed.
  • 1a7da23: a zero or negative cookie Max-Age now means expire-immediately (RFC 6265 §5.2.2). withMaxAge stores it as 0 and the renderer clamps at 0, so Max-Age=-1 can never be emitted. The cookie-negative-max-age known-defect entry is retired, and three ordinary checks replace it.

Toolchain

Tool Version
Idris 2 (build/proof jobs) 0.8.0
Idris 2 (header RefC profile) 0.8.0-b6cbc25d5
Chez Scheme 9.5.8
Zig 0.15.2
OCaml 5.1.1 (ctypes 0.24.0)

Results at 1a7da23

Check (CI step reproduced) Command Result
Trustfile: SPDX compliance the step's own grep … | xargs grep -LE "MPL-2.0|CC-BY-SA-4.0" before (9e4e542): ERROR: 5 files …, exit 1. after: License compliance: PASS, exit 0
Trustfile: believe_me audit step script 50 instances, below the advisory threshold of 300; passes
Trustfile: FFI purity step script 7 files scanned, 0 matching lines; passes
Cookie checks SafeCookieProps driver on the old library, the new check fails (FAIL: negative Max-Age accepted as expire-immediately …, exit 1). On the new library, all 3 new checks pass (exit 0)
build (0.8.0): totality regressions bash scripts/test-totality-regressions.sh exit 0
build (0.8.0): build idris2 --build proven.ipkg exit 0, 0 errors
build (0.8.0): install idris2 --install proven.ipkg exit 0
build (0.8.0): diagnose properties idris2 -p proven -p contrib -p network --check on each *Props.idr 29/29 modules, 0 errors
build (0.8.0): tests idris2 --build tests.ipkg && build/tests/exec/proven_tests exit 0: 318 PASS, 0 FAIL, 19 ledgered expectations from 3 strict entries (toml-parser-stub, yaml-block-collections-rejected, xml-doctype-unparsed)
build (0.8.0): type check the 12 modules, from src/ 12/12
proof-check (0.8.0) every *.Proofs listed in proven.ipkg, from src/ 29/29

Not re-run, and why

  • OCaml CI and "Real Idris header ABI": neither new commit touches bindings/, ffi/, proven.h, the ABI ledger or ByteKernel.idr. The last remote runs, at 9e4e542, passed: Idris 2 CI and OCaml CI. Their local equivalents also passed before.
  • Not run locally: macOS/cross Zig targets, the theory replay and the other language workflows.

Nothing here substitutes for CI. Once minutes are refilled, re-dispatch idris2-ci.yml and ocaml-ci.yml on this branch; they only trigger automatically for PRs into main.


Generated by Claude Code

@sonarqubecloud

Copy link
Copy Markdown

Repository owner deleted a comment from coderabbitai Bot Nov 18, 2026
@hyperpolymath
hyperpolymath merged commit 7a0b11c into codex/real-header-refc-20261116 Nov 18, 2026
26 checks passed
@hyperpolymath
hyperpolymath deleted the claude/trusting-fermat-2msgw7 branch November 18, 2026 14:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants