Skip to content

CORE (signing sitting): chain checkpoints and inclusion proofs - #89

Merged
githubscum merged 1 commit into
mainfrom
lotor-lane/chain-checkpoints
Oct 28, 2026
Merged

githubscum merged 1 commit into
mainfrom
lotor-lane/chain-checkpoints

Conversation

@githubscum

Copy link
Copy Markdown
Owner

WO-CARTRIDGE-SEAL part B. CORE, queued for the signing sitting. Do not merge as a code change: nothing under src/ changes on this branch. The core change is proposals/chain-checkpoints.patch, generated by node proposals/make-chain-checkpoints-patch.mjs (temp copies plus git diff --no-index; never typed, per KNOWN-LIMITS 80). Design, preflight, measurement, sitting steps and draft limits: proposals/chain-checkpoints.md.

What the patch does

  • src/chain/merkle.js (new): RFC 9162 section 2.1 hashing, inclusion proofs (2.1.3) and consistency proofs (2.1.4), plus a compact-range frontier so a tree extends without its leaves. Node builtins only.
  • src/chain/checkpoint.js (new, pure): C2SP signed-note (Ed25519, type 0x01, key ID per spec) and C2SP tlog-checkpoint (origin, size, base64 root; no extension lines). verifyInclusionProof(proof, publicKey) and verifyConsistencyProof(proof, publicKey) need only the proof and the chain's public key.
  • src/store/checkpoints.js (new): sidecar receipts/checkpoints.jsonl; chain.jsonl is never rewritten. Segment sealing every 1024 entries, one lock acquisition per segment. Optional head checkpoint (seal({ atHead: true })) so the newest receipt is provable now. Incremental verify from the newest checkpoint or from one the caller trusts. Full sealed verify. Proofs.
  • src/chain/index.js: verifyRange(entries, key, { startSeq, prevHash }); verifyChain becomes verifyRange from genesis with identical results (asserted against today's live verifyChain on seven chains).
  • src/store/index.js: seal, verifyIncremental, verifySealed, proveInclusion, proveConsistency on the store.

Leaves are the existing entry hashes. Migration of an existing chain is the first seal(): the first checkpoint covers seq 0..1023 of the file as it is.

U+2014 note. The C2SP signature line begins with U+2014, which this repo bans. The code builds it with String.fromCharCode(0x2014); nothing here contains the byte, and the rehearsal asserts the patch carries none. The format is the spec's, unchanged.

Fail-first tests (test/chain-checkpoints.test.js)

Same shape as test/chain-seq-continuity.test.js: the live block runs every case as TODO until the patch lands (landing tripwire: src/chain/merkle.js exists); the queued block applies the patch in a temp repo and runs every case for real.

case on main (live block) patch applied
tamper inside a sealed segment, key-holder rewrite: entry 5 rewritten and re-signed with the same chain key, sidecar re-sealed fails (TypeError, no seal). Control in the same case, true on main: today's verifyChain returns { ok: true } on the rewritten chain kept size-16 checkpoint reports a fork; the old proof's leaf no longer matches; no consistency proof from the kept checkpoint
tamper inside a sealed segment, no key fails (TypeError) verifySealed and a walk from an earlier trusted checkpoint report hash mismatch at entry 10; from the newest checkpoint it is out of sight (documented, asserted)
forged checkpoint: other key, edited root, forged note planted in the sidecar fails (TypeError). Control: both forgeries parse as valid checkpoints when the signature is skipped rejected by verifyCheckpoint, verifyIncremental, verifySealed; seal and proveInclusion refuse
proof against the wrong checkpoint: other size, forged size, other index, other chain fails (TypeError) all rejected with the reason
migration of a 40,000-entry fixture fails (SEGMENT_SIZE undefined, before building the fixture) chain.jsonl SHA-256 unchanged, all 40,000 entry hashes reproduced, 39 checkpoints, first covers 0..1023, full and incremental verify ok, proofs verify at seq 0, 1, 1023, 1024, 20000, 39935
also: RFC 6962 reference roots 1..8; every inclusion and consistency proof to size 40 plus wrong ones; frontier equals MTH 0..70; C2SP spec signed-note example; verifyRange equivalence and absolute seqs; head checkpoint on demand; sidecar caches fail closed (frontier, lastHash, truncation) TODO, fails pass

How checked: (1) branch run, live block 13 TODO, all failing with the errors above; (2) the same test file on a clean clone of 337358f with no patch file: 13 TODO, 15 cancelled, exit 1 (loud); (3) patch applied in a throwaway clone: the live block runs all 13 cases un-todo'd and passes.

Counts

  • git apply --check proposals/chain-checkpoints.patch on 337358f: clean.
  • main 337358f: 1379 tests, 1374 pass, 0 fail, 5 skipped.
  • this branch: 1407 tests, 1389 pass, 0 fail, 5 skipped, 13 todo (+28: the rehearsal file).
  • patch applied in a throwaway clone, then npm run limits-pin -- --stamp as the sitting would: 1393 tests, 1387 pass, 0 fail, 6 skipped, 0 todo.
  • Known flake reproduced, unrelated: running two suites at once fails matcher-history "temp dirs prefixed lotor-mh- are cleaned up" on main too (it counts the shared system temp dir). Every count above is from a run alone.
  • Guard from CI on Linux and Windows, PR intake guard, CONTRIBUTING #88 (bash .github/scripts/pr-guard.sh origin/main HEAD false): guard: clean.

N and its measurement

node proposals/chain-checkpoints/measure-n.mjs, 20,000 entries, three runs: verifyChain ~80 us/entry, sign ~34, Merkle hashing ~2.9, parse ~0.75. N bounds the work an incremental verify does past the newest checkpoint. N = 1024 (power of two, so segments are perfect subtrees) puts the worst case at ~82 ms; a 40k full verify is ~3.2 s.

Byte budget

The sealed segment adds nothing to a cartridge by itself. Measured on the 40k rehearsal chain for the non-core follow-on, which is NOT in this PR (the cartridge carrying proofs): path 16 hashes; signed note 219 B; compact binary proof 616 B for the first and 516 B for each further proof against one checkpoint. Against the 2,808 B useful PAP payload that is at most 5 proofs with an empty spine, about 3 next to a 1 KB compressed spine, before sealed-mode overhead.

Sitting

Apply with each target on the command line (KNOWN-LIMITS 82), one signature for one command:

git apply --include=src/chain/index.js --include=src/chain/merkle.js --include=src/chain/checkpoint.js --include=src/store/index.js --include=src/store/checkpoints.js proposals/chain-checkpoints.patch

then npm run limits-pin -- --stamp, npm test, and append the draft limits from proposals/chain-checkpoints.md with the next free numbers.

Doubts and shortcuts

  • Checkpoints use the chain key (limits 8, 48). They catch a rewrite only for someone who kept an earlier checkpoint; the key holder can re-seal a rewritten chain and verifySealed passes. No cosigner, no anchor (OpenTimestamps out of scope).
  • Nothing seals automatically. No hook or CLI calls seal(). Wiring it into the session-end hook (core) or pap-export (not core) is a separate decision. Sealing in the append path was rejected on purpose.
  • The RFC 6962 reference roots in the test were written from the transparency-dev test data from memory, not re-fetched. All eight matched the implementation on the first run, which cannot happen by chance, but the source was not re-read.
  • Fail-first on main is mostly "the function does not exist". The substantive controls are the two marked above (key-holder rewrite verifies on main; forgeries parse without a signature check).
  • proveInclusion and proveConsistency read every entry the checkpoint covers (O(n), on demand). Not measured separately.
  • Seal against concurrent appends is not stress-tested; it takes the chain lock per segment like an append does.
  • Sealing refuses an unparseable chain line where loadChain drops it silently (stricter, recorded in the proposal).
  • The patch is 1,236 lines for one signature. It splits cleanly if preferred: the two pure files first, then the store wiring.
  • The 40k migration case adds ~11 s to the suite.
  • No KNOWN-LIMITS entry was added on this branch: the three limits are about the patch and are drafted in proposals/chain-checkpoints.md, to be numbered when it lands, so they don't collide with the open part A PR. NEXT-SIGNING-SITTING.md does not exist in the repo, so no entry was added there.

🤖 Generated with Claude Code

WO-CARTRIDGE-SEAL part B, queued as a patch. Nothing under src/ changes on
this branch.

- proposals/chain-checkpoints.patch, generated by
  proposals/make-chain-checkpoints-patch.mjs (git diff --no-index over temp
  copies; never typed). Adds src/chain/merkle.js (RFC 9162 hashing, inclusion
  and consistency proofs), src/chain/checkpoint.js (C2SP signed-note and
  tlog-checkpoint, pure verifiers), src/store/checkpoints.js (sidecar
  receipts/checkpoints.jsonl, segment sealing every 1024 entries, incremental
  and full sealed verify, proofs), verifyRange in src/chain/index.js
  (verifyChain becomes verifyRange from genesis, results unchanged), and
  seal/verifyIncremental/verifySealed/proveInclusion/proveConsistency on the
  store. chain.jsonl is never rewritten.
- proposals/chain-checkpoints/: the new files' sources, the N measurement
  script. proposals/chain-checkpoints.md: design, preflight, measurement,
  sitting steps, draft KNOWN-LIMITS entries.
- test/chain-checkpoints.test.js: rehearsal. Live cases are TODO and fail
  until the patch lands; the queued-patch block applies it in a temp repo and
  runs every case.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@githubscum
githubscum merged commit 8519349 into main Oct 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant