Skip to content

Reduce disjoint verifier peak traced allocation by 62% (no new κ) - #6135

Merged
Mister-Meeseeks merged 1 commit into
CrocSwap:mainfrom
petalas:research/w15-1022aj-frontier-step
Oct 28, 2026
Merged

Mister-Meeseeks merged 1 commit into
CrocSwap:mainfrom
petalas:research/w15-1022aj-frontier-step

Conversation

@petalas

@petalas petalas commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Tooling improvement with a scoped serialization-equivalence proof. An opt-in streaming encoder reduces peak traced allocation during complete DisjointTensor(26).verify() from 107,703,062 to 41,079,356 bytes, about 62%, with identical bytes, checks and complete receipts. This answers Wentao Li's PR #2117 Open next question 1. No new κ or supplier admission is claimed.

 unchanged support, topology, recurrence and physical embedding checks
- materialize every DAG row, complete JSON document and encoded bytes
+ hash the identical compact JSON bytes in bounded chunks
 unchanged complete receipt

Adds only research/disjoint-dag-streaming/. The original checker, historical certificates and selected frontier remain unchanged. Credit Douglas Colkitt, Wentao Li #2001/#2117, Stephan Arrington #1044 and the retained checker lineage. This contribution was produced by Nick Petalas.

Advance CLAIM. Scientific base 721c03822c421b7f7fcbb2598993b947c4d88419, priority revision 8. Published head e4eb97178bdc083cf9e60f8796cf93cddbe27dbf.

Evidence

  • Before: whole-document serialization peaks at 84,685,162 traced bytes; complete verification peaks at 107,703,062 bytes.
    After: serialization peaks at 691,046 bytes; complete verification peaks at 41,079,356 bytes. All five measurements per side give these peaks.
  • Five interleaved fresh-process pairs on CPython 3.12.15/macOS arm64, zero errors, identical complete receipts. Construction plus verification median is 0.399820 s [0.397821, 0.433130] versus 0.377585 s [0.371520, 0.388686]. This is descriptive timing, not a portable speed guarantee. Allocations are measured after circuit construction with tracemalloc, separately from timing; they are not process RSS.
  • Exact bytes on 1,623 schema fixtures and six complete circuits through h=26; 41 rejecting controls. The complete nondigest checker AST is unchanged. Both registered normal and optimized modes pass; direct runs take about 1.29 s and 1.25 s.
  • Exact committed bytes, source history and modes pass for 13 files. The official export passes integrity and declared-input coverage. Read-only Git-free normal and optimized export checks pass without changed bytes.

Reproduce from the repository root with the tested interpreter:

/Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -B scripts/research_package.py --package research/disjoint-dag-streaming
/Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -I -W error -B research/disjoint-dag-streaming/verify.py
/Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -I -O -W error -B research/disjoint-dag-streaming/verify.py
/Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -B scripts/check_research_commit.py --package research/disjoint-dag-streaming --check-source-history --check-modes

Expected scientific output: PASS_DISJOINT_DAG_STREAMING; the registered runner also prints PASS_REGISTERED_RESEARCH_PACKAGE. The benchmark and all five timing pairs are documented in README.md. With an existing external output directory assigned to disjoint_run_dir, its exact reproduction is:

/Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -I -W error -B research/disjoint-dag-streaming/benchmark.py --rounds 5 --output "$disjoint_run_dir/measurements.json"

Limitations

This is an opt-in historical-checker improvement, bounded to one million array entries and 10,000-bit supports. It proves byte equivalence for the stated schema, not correctness of every possible upstream circuit or a complete multiplication theorem. The unchanged compiler allocation remains. Default adoption and source-pin migration are not implemented. Full historical/selected supplier replays, native pricing, all-size compiler, precision/tape, erasure and exact recovery are unrun obligations. Independent human and formal proof-kernel review are unclaimed.

Open next questions

  1. Adopt the encoder in the default checker with an explicit migration that preserves historical source bindings.
  2. Reduce the remaining compiler allocation while preserving every physical embedding and freshness check.

README.md and TASK.json give the first steps and blockers.

Related open work

Final overlap refresh checked this account's open PRs created since the advance CLAIM at 2026-10-28T12:55:34Z. None of the 13 other open PRs studies DisjointTensor DAG serialization or computes this allocation quantity, so no numerical comparison is needed. The earlier refreshed open-PR, Discussion and main searches likewise found no competing serialization contribution. PR #6112 optimizes the selected Clifford replay and PR #6087 optimizes replay planning; their engines and inputs are separate. PR #6050 addresses the shared navigation failures below. No predecessor numbers were independently audited for this contribution.

CI notes

Uncertified by CI at published commit e4eb97178bdc083cf9e60f8796cf93cddbe27dbf. In run 40694782522, plan failed and the package replay was skipped. The package replay did not run in CI. A maintainer must re-run CI after main's documentation is fixed. Local validation above remains local evidence only.

The failure is not caused by this change. Base main commit 721c03822c421b7f7fcbb2598993b947c4d88419, whose subject ends in [skip ci], admitted PR6030 and updated the selected record without matching the current-status and CI navigation documentation. The two onboarding failures are:

  • test_research_current_status.CurrentStatusTests.test_current_section_matches_selected_record expects **0.003125804** in docs/research/current-status.md; the existing bold span also includes = 781451/250000000.
  • test_research_current_status.CurrentCiNavigationTests.test_current_ci_section_matches_selected_record expects make frontier6030-verify in docs/ci-verification.md, the file read by that test; its current section still describes Raise conditional kappa to 0.00310045 with independent helper splits #5599 and make frontier5599-verify.

The failed onboarding job reports 678 tests, exactly these two failures and 23 skips. The same two failures were checked in PR #6120's run 40691776027. They were also reproduced locally before publication. Existing PR #6050 addresses the documentation mismatch. No code was pushed or main documentation changed in response to this CI result.

Merge Danger

Door: two-way. The adapter is opt-in and can be removed independently.

Blast Radius: isolated. Only the new research package and its declared replay are added.

@Mister-Meeseeks
Mister-Meeseeks merged commit c62bc64 into CrocSwap:main Oct 28, 2026
1 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants