Skip to content

Reduce complete replay planning time by 8.27% (no new κ) - #6177

Merged
Mister-Meeseeks merged 1 commit into
CrocSwap:mainfrom
petalas:research/w11-1029a-translation-span
Oct 29, 2026
Merged

Mister-Meeseeks merged 1 commit into
CrocSwap:mainfrom
petalas:research/w11-1029a-translation-span

Conversation

@petalas

@petalas petalas commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Complete replay planning repeats filesystem checks on ancestors shared by different input and manifest paths. This opt-in planner reuses those checks within one stable planning invocation. Complete local median time falls 8.27%, from 0.898166250 s to 0.823856000 s. Claim type: scoped equivalence proof with finite performance evidence. No new kappa or supplier admission.

 plan_details
-  inspect every path component on each distinct path
+  inspect each distinct prefix once within this plan
   retain all manifest reads, hashes, exact path checks and selection causes
   discard cached state on success or failure

Only research/replay-planner-ancestor-cache/ is added. The command authenticates the unchanged shared runner, loads its source into a private module, and replaces its guard for one synchronous plan. The default CI planner is unchanged. A changed shared source fails authentication.

Produced by Nick Petalas. Credit JoyboyBrian's PR #6087 for complete-input reuse, Douglas Colkitt for the runner, Ali Romman for earlier complete-plan benchmark context, and the retained exact-path, symlink/junction, sparse-manifest, Git-source and deadline contributors. Their source and notices are unchanged. Advance CLAIM.

Evidence

  • Before: nine complete commands have median 0.898166250 s, range 0.884204958–0.970985542 s.
    After: nine complete commands have median 0.823856000 s, range 0.801496583–0.867351208 s. All eighteen commands succeed, select the same 3,788 packages and emit identical JSON and warnings, including 3,759 authenticated sparse manifests. Each pair reverses the starting side. The generated BENCHMARK.json binds sources, harness, manifest, environment, samples and output digests.
  • The measured limiter is repeated component traversal and filesystem inspection. An untimed profile puts the original guard at about one quarter of complete planning time. These are warm-filesystem macOS measurements on a shared 18-CPU host; no cold-cache or Linux/Windows speedup is claimed. Startup, source authentication and complete planning are inside the timing.
  • Warning-strict normal and optimized checks pass 18 path cases, four symlink/junction cases, two lookup-failure cases, eight complete fixture-plan cases, source-corruption rejection and generated-receipt validation. Both registered commands pass; locally each direct command took less than 0.2 s against a 120 s timeout.
  • Package metadata, committed-byte/source-history/mode checks and official export integrity/input coverage pass at 07236dccd7dfca0e5e501c77e24f645b7ee523f6. Read-only Git-free normal/optimized exported replays pass with identical stdout; all ten exported files retain their bytes and modes. In-memory syntax and function-size checks pass. No Python linter was installed locally.

From the repository root, with the exact local interpreter:

/Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -I -W error -B research/replay-planner-ancestor-cache/planner.py --changed-since origin/main
/Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -I -W error -B research/replay-planner-ancestor-cache/verify.py
/Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -I -O -W error -B research/replay-planner-ancestor-cache/verify.py
/Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -B scripts/research_package.py --package research/replay-planner-ancestor-cache
/Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -B scripts/check_research_commit.py --package research/replay-planner-ancestor-cache --check-source-history --check-modes

Another standard-library CPython 3.12 executable may replace that path. Expected markers are PASS_REPLAY_PLANNER_ANCESTOR_CACHE, PASS_REGISTERED_RESEARCH_PACKAGE and PASS_RESEARCH_COMMIT_BYTES. Optional timing reproduction:

/Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -I -W error -B research/replay-planner-ancestor-cache/benchmark.py --pairs 9 --output /absolute/scratch/benchmark.json

Future hosts need not reproduce these times or package counts. Registered replay validates the recorded measurement bindings and arithmetic without rerunning timings.

Limitations

The equivalence proof inherits the stable-checkout contract and uses one synchronous caller. The table is bounded by distinct prefixes encountered and is discarded after each plan; peak heap was not measured. Source execution is constrained to an exact SHA256 pin. Python allocation and the authenticated loader deviations are documented in the README.

Native Windows junction execution, a fully materialized checkout benchmark, CI performance, selected supplier replay, Lean, Docker and complete multiplication obligations remain unrun. Independent human and formal proof-kernel review are unclaimed.

Open next questions

Should prefix reuse enter the shared planner? First test native Windows junctions and benchmark a fully materialized checkout. Those platform and workload checks remain the blockers; this PR ships the first verified local improvement as an opt-in command.

Related open work

  • #5721 changes Windows ownership of the shared Git blob reader. This package retains that reader's current pinned implementation and changes only path validation during planning; it supplies no competing process-lifetime or Windows result.

  • #6179 defers an unused validation-source load in the current task-picker CLI. Both improve research coordination tooling, but they change different commands and source files; its picker-startup workload and this complete replay-plan workload are distinct, so their timing percentages are not compared.

CI notes

Both hosted checks passed at exact contribution head 07236dccd7dfca0e5e501c77e24f645b7ee523f6 in run 40880877406: plan in 4m29s and replay (research/replay-planner-ancestor-cache) in 1m29s. The completed check list was read once, and run metadata confirms completed success at this exact head.

Hosted replay certifies the two registered normal and optimized correctness commands, including path/plan differentials, rejection controls, source binding and recorded-measurement validation. It does not repeat the timing experiment or establish a Linux/Windows speedup, physical supplier admission or proof-kernel acceptance. The local source-history/mode and read-only Git-free export checks remain separate passing evidence.

Merge Danger

Door: two-way.

Blast Radius: isolated.

Eight files in one opt-in research package. Shared code, frozen sources and the selected frontier are unchanged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants