Reduce current task-picker startup time by 4.66% (no new κ)

research/current-picker-lazy-validation/BENCHMARK.json

@@ -0,0 +1,68 @@
+{
+  "after_sha256": "04e194b14a1f4157be5be83e8c13995d69c3efe8996773d918acba6bfcabb471",
+  "argv": [
+    "--seed",
+    "00000000000000000000000000000000",
+    "--task",
+    "replay-runtime"
+  ],
+  "before_sha256": "51b9b217cf662557a8303136208ebd1625770dc8f4c8f5141a1fb849282825d4",
+  "benchmark_sha256": "19223030956bd2e8f01886d13e2ad11f9db56bd5b043f0e9b78d26508b1d424b",
+  "cpu_count": 18,
+  "direct_cli_controls": 1,
+  "errors": 0,
+  "fraction_less_wall_time": 0.04655790582170749,
+  "launcher": "import pathlib,sys; p=sys.argv.pop(1); f=sys.argv[0]=str(pathlib.Path.cwd()/'scripts/choose_current_research_task.py'); exec(compile(pathlib.Path(p).read_bytes(),f,'exec'),{'__name__':'__main__','__file__':f})",
+  "load_average": [
+    2.93798828125,
+    2.8505859375,
+    2.7275390625
+  ],
+  "machine": "arm64",
+  "medians_seconds": {
+    "after": 0.09959755828604103,
+    "before": 0.10446104581933469
+  },
+  "pairs": 9,
+  "platform": "macOS-27.0.1-arm64-arm-64bit",
+  "python": "3.12.15 (main, Oct 14 2026, 00:55:37) [Clang 23.1.3 ]",
+  "ranges_seconds": {
+    "after": [
+      0.09396249998826534,
+      0.10557451250497252
+    ],
+    "before": [
+      0.10343423748854548,
+      0.11021292919758707
+    ]
+  },
+  "repetitions_per_batch": 10,
+  "samples_seconds_per_command": {
+    "after": [
+      0.09959755828604103,
+      0.10118455418851227,
+      0.09396249998826534,
+      0.099184066709131,
+      0.10007922500371932,
+      0.09903859999030828,
+      0.09966542089823634,
+      0.09871085421182216,
+      0.10557451250497252
+    ],
+    "before": [
+      0.10914635420776904,
+      0.10519179170951247,
+      0.10352727079298348,
+      0.10446104581933469,
+      0.10507754581049085,
+      0.1042379666119814,
+      0.10343423748854548,
+      0.10353437920566648,
+      0.11021292919758707
+    ]
+  },
+  "schema": "current-picker-complete-timing/1",
+  "scope": "Complete source-executed CLI, fresh interpreter per command; same launcher on both sides. Timing includes JSON decoding and output equality checks. No speed guarantee on other hosts or mapped historical routes.",
+  "stdout_sha256": "de87d316bc9eac6279769b9d6d42adcb001f49f90cff87ad404b9a61a33d125b",
+  "successful_timed_commands": 180
+}

research/current-picker-lazy-validation/README.md

@@ -0,0 +1,134 @@
+# Defer unused task-picker validation
+
+This contribution was produced by Nick Petalas.
+
+The current task picker loads the frozen legacy picker in the parent process
+before launching that same picker in a child. Move the parent load into the
+historical-route branch. Cards already carrying current commands need only the
+child execution. The legacy subprocess, its catalogue checks and output, and
+historical route validation retain their code.
+
+This is an empirical tooling improvement with a scoped equivalence argument
+and finite regression evidence. It supplies no new mathematical bound, supplier
+admission or kappa. [BENCHMARK.json](BENCHMARK.json) records the complete-command
+measurements, source hashes, Python and host, raw timings and error count.
+The recorded median falls from 104.461 ms to 99.598 ms, 4.66% less wall time.
+Every one of the nine paired batches improves. Batch-average ranges are
+103.434–110.213 ms before and 93.962–105.575 ms after; the paired decreases
+range from 3.769 to 9.565 ms. All 180 timed commands succeed with identical
+output, and the direct-CLI control agrees. The raw ranges overlap, so the
+paired observations and host-specific scope matter. The profile attributes
+about 4 ms of the original complete command to the removed parent load;
+child startup and catalogue validation remain the main cost.
+
+## Why the checks survive
+
+Before the moved statement, the resolver identifies the exact historical
+commands. If none occurs, it returns without using the legacy namespace. If a
+historical command occurs, it loads the same exact source and obtains the same
+validation function before validating or changing any card. Loading is not
+cached between resolver calls. Subsequent calls therefore still read current
+source. The CLI always runs its unchanged legacy subprocess before resolving
+successful JSON output, and passes through its help and errors as before.
+
+This argument assumes the retained legacy module's top-level definitions and
+imports. It is not an equivalence claim for arbitrary side-effectful replacements
+of that module, imports by external code of the removed incidental module global,
+or concurrent changes to the source tree. No public helper API is documented for
+that global. Historical mapped routes may pay the same load on every resolver
+call; no performance gain is claimed there.
+
+[verify.py](verify.py) reconstructs the exact pre-change source and checks its
+pinned SHA256, authenticates the unchanged legacy source, checks loading at the
+actual resolver boundary, compares all 22 live first-step cards and both frozen
+route shapes, and exercises seven source/route rejection controls. Explicit
+exceptions keep checks active under optimized Python. The existing routing suite
+also covers CLI help/errors, exact matching, atomic failure, sparse references,
+staged replacements and historical output preservation.
+
+## Reproduction
+
+Use CPython 3.12 and its standard library from the repository root. The local
+interpreter was `/Users/nick/git/integer-mult-bounds/.venv/bin/python3.12`.
+
+```sh
+python3.12 -I -W error -B research/current-picker-lazy-validation/verify.py
+python3.12 -I -O -W error -B research/current-picker-lazy-validation/verify.py
+python3.12 -B scripts/research_package.py --package research/current-picker-lazy-validation
+python3.12 -B scripts/check_research_commit.py --package research/current-picker-lazy-validation --check-source-history --check-modes
+python3.12 -W error -B -m unittest discover -s tests -p test_research_current_assignment.py -v
+```
+
+Expected marker: `PASS_CURRENT_PICKER_LAZY_VALIDATION`. Registered checks are
+bounded correctness checks; they do not assert a host-independent timing ratio.
+To repeat the timing experiment, choose a scratch directory and a new receipt
+path. The writer refuses an existing receipt.
+
+```sh
+python3.12 -W error -B research/current-picker-lazy-validation/benchmark.py --pairs 9 --repetitions 10 --scratch /tmp --write /tmp/new-picker-timings.json
+```
+
+Both sides execute the entire CLI in fresh interpreters, using the real script
+filename and root with different authenticated source bytes. The identical
+launcher reads and compiles one temporary source file per command. It retains
+the legacy child, all its checks and Git queries. Timings include output capture,
+JSON decoding and equality checks. Every complete `(status, stdout, stderr)`
+triple must agree, and a separate direct CLI invocation must match the candidate.
+The script does not write or replace checkout source files. Nine alternating
+batches contain ten complete commands per side, 180 timed commands in total.
+The measurement is of this source-executed CLI, not a subtraction of isolated
+import times. Concurrent host load and the launcher's common overhead limit
+transfer of the measured ratio to other environments.
+
+## Scope and next questions
+
+The original `replay-runtime` card is changed from scientific replay to research
+onboarding startup. This small improvement helps repeated coordination commands;
+it does not accelerate the selected scientific replay. The first improving
+implementation is frozen. Canonical base is
+`3f0152a32eeebac0df11946dbd50cabe5a7f7839`, priority revision 8.
+[The advance claim](https://github.com/CrocSwap/integer-mult-bounds/discussions/1166#discussioncomment-19267797)
+records the scope before implementation.
+
+The existing historical picker, catalogue, selected frontier and predecessor
+packages are unchanged. No historical scientific suite, native replay, Lean,
+Docker, formal proof kernel or independent human review is claimed.
+
+## Open next questions
+
+Does startup loading still matter for the historical mapped routes, where the
+resolver genuinely needs validation? Profile those complete commands separately
+before changing their execution boundary. Preserving child failure propagation,
+source freshness and sparse-reference rejection is the blocker to a broader
+startup change. This package supplies no such change.
+
+## Code constraints
+
+The implementation moves one load and adds no cache, recursive call or loop.
+The package's bounded verifier handles at most 1,000 cards, four route references,
+two route shapes and three source mutants. Benchmark options cap work at 31 pairs
+and 100 commands per batch, with a 20-second timeout on each child. Functions are
+below 60 physical lines.
+
+Power of Ten deviations apply to the package's Python-managed allocation and
+explicit trusted-source execution. Live memory is bounded by the pinned scripts,
+the checked catalogue limit and one captured command output at a time. The
+trusted picker emits a finite catalogue report; the benchmark rejects output
+above 1 MiB after capture. It does not claim a hard capture-memory limit for
+an arbitrarily changed child. Temporary fixtures have four files and are
+removed by their context manager on both success and failure. Dynamic
+execution is needed to compare actual source versions with the same filename;
+both versions are authenticated before execution. No arbitrary submitted program
+is accepted. The benchmark additionally uses the fixed standard-library module
+loader for the local verifier. Replacing these mechanisms would require a new
+benchmark contract; no shared runtime or build configuration is relaxed.
+
+## Credit
+
+Ali Romman / PR5378 supplied the current-command wrapper. Mihir Srivastava /
+PR4906 and PR5520 supplied startup guidance and historical-ceiling routing.
+Douglas Colkitt and the retained sparse-reference contributors supplied the
+frozen picker and its reference checks. Deferred initialization is standard
+programming practice. This contribution is its measured application at this
+specific CLI boundary, with exact-source conformance and reproduction. Existing
+source credit lines retain their bytes. New work is Apache-2.0.

research/current-picker-lazy-validation/SOURCES.json

@@ -0,0 +1,30 @@
+{
+  "schema_version": 1,
+  "base_commit": "3f0152a32eeebac0df11946dbd50cabe5a7f7839",
+  "files": [
+    {
+      "path": "certificates/selected-result.json",
+      "usage": "provenance",
+      "git_blob": "d59b5035e7b4cbaa9d688a1d32dd0d6eb79c466a",
+      "sha256": "9b8d0eea95b3bf0dfcb7938a732a6439b87a2685dec2a5ab9e525b3b34debbfa"
+    },
+    {
+      "path": "docs/research/agent-task-catalog.json",
+      "usage": "provenance",
+      "git_blob": "3404fbdd5e7529a0674eef26114b5071e4586cec",
+      "sha256": "ead6cdc9d0af6d58e8a49744d8ec8c7c15e13d2ba93c276c434bcf9b51b93a81"
+    },
+    {
+      "path": "scripts/choose_research_task.py",
+      "usage": "input",
+      "git_blob": "5c2ed0bab22d1e93c8fd80245b64a0fe89cee32c",
+      "sha256": "484f553c488c6f427ca81f19f3ee2720399dcc28ce61a3886d50bdf28ca72d12"
+    },
+    {
+      "path": "scripts/choose_current_research_task.py",
+      "usage": "provenance",
+      "git_blob": "0402d37f4f65de0f9a84fc7a7209a9f2e934efb1",
+      "sha256": "51b9b217cf662557a8303136208ebd1625770dc8f4c8f5141a1fb849282825d4"
+    }
+  ]
+}

research/current-picker-lazy-validation/TASK.json

@@ -0,0 +1,61 @@
+{
+  "schema_version": 1,
+  "task_id": "replay-runtime",
+  "priority_revision": 8,
+  "base_commit": "3f0152a32eeebac0df11946dbd50cabe5a7f7839",
+  "original_card": {
+    "id": "replay-runtime",
+    "question": "Can one measured replay bottleneck be sped up without dropping coverage?",
+    "starting_points": [
+      "scripts/run_isolated_tests.py",
+      "scripts/verify_replay_coverage.py",
+      "certificates/selected-result.json"
+    ],
+    "methods": [
+      "Profile one isolated replay and optimize the measured bottleneck.",
+      "Design source-bound reuse with explicit dependency invalidation and negative controls."
+    ],
+    "acceptance_gate": "Report before/after wall time on the same fixture, retained checks and invalidation behavior.",
+    "first_step": {
+      "command": [
+        "python3",
+        "-B",
+        "scripts/research_selected_baseline.py"
+      ],
+      "expected": "PASS_RESEARCH_BASELINE_BINDINGS_ONLY",
+      "inspect": "Resolve the current selected review, separate complex/ordinary BIT/admitted BIT/outer budgets, and compare the card against live handoffs before starting. Historical ceilings require exact source/model equivalence.",
+      "scope": "Source-binding orientation only; not a scientific replay or all-size proof. Materialize selected inputs if this checkout is sparse."
+    },
+    "first_experiment": "Measure one unchanged fixture, preserve check coverage, and test one optimization with invalidation controls.",
+    "bottleneck_link": "Faster trustworthy replays increase review throughput and let mechanism candidates be checked sooner.",
+    "allowed_variations": [
+      "Profile one stage or design one source-bound cache boundary."
+    ],
+    "scope_change_triggers": [
+      "Skipping coverage or comparing different fixtures to claim speedup."
+    ],
+    "overlap_terms": [
+      "replay runtime",
+      "cache invalidation",
+      "isolated tests"
+    ]
+  },
+  "scope_status": "changed",
+  "actual_scope": "Defer unused parent-side legacy validation loading in the current task-picker CLI, preserving exact complete outputs, source freshness and historical-route rejection.",
+  "scope_change_reason": "The first BIT suggestion was not assigned by the donor. Live and recent-account overlap excluded the mathematical tour candidate before research. This uses the current verifier-engineering priority for a distinct original onboarding bottleneck rather than auditing a predecessor.",
+  "bottleneck_link": "Repeated coordination commands load the same frozen picker twice even when no historical first-step rewrite is needed. Removing the unused load reduces this bounded startup cost; it changes no supplier or scientific replay.",
+  "budget_minutes": 60,
+  "stop_rule": "Ship the first measured complete-command improvement after exact conformance and retained rejection checks. Do not expand the optimization.",
+  "outcome": "completed",
+  "evidence": "The exact-source conformance checker passes all 22 live cards, two historical route shapes and seven source/route rejection controls. The existing 12 routing tests pass. Generated BENCHMARK.json supplies alternating complete-command wall times, every successful command and source hashes. The recorded nine paired batches of ten fresh commands each have medians 104.461 ms before and 99.598 ms after, 4.66% less wall time, with all nine paired decreases positive and zero errors. The ratio is empirical and host-specific.",
+  "author": "Nick Petalas",
+  "ai_assistance": "Independent human and formal proof-kernel review are unclaimed.",
+  "open_questions": [
+    {
+      "question": "Does startup loading remain a material bottleneck for historical mapped routes?",
+      "why_it_matters": "Those routes still require the legacy validation load, so this optimization does not accelerate that branch.",
+      "first_step": "Profile complete historical mapped CLI commands before changing execution boundaries.",
+      "blocked_by": "Any replacement must preserve source freshness, exact error propagation and sparse-reference rejection."
+    }
+  ]
+}

research/current-picker-lazy-validation/VERIFY.json

@@ -0,0 +1,38 @@
+{
+  "schema_version": 1,
+  "status": "ready",
+  "python": "3.12",
+  "requirements": [],
+  "inputs": [
+    "scripts/choose_research_task.py",
+    "scripts/choose_current_research_task.py",
+    "docs/research/agent-task-catalog.json"
+  ],
+  "commands": [
+    {
+      "argv": [
+        "{python}",
+        "-I",
+        "-W",
+        "error",
+        "-B",
+        "{package}/verify.py"
+      ],
+      "timeout_seconds": 120,
+      "checks": "Exact baseline reconstruction, fresh source loading, unchanged live-card outputs and historical-route rejection controls."
+    },
+    {
+      "argv": [
+        "{python}",
+        "-I",
+        "-W",
+        "error",
+        "-O",
+        "-B",
+        "{package}/verify.py"
+      ],
+      "timeout_seconds": 120,
+      "checks": "Exact baseline reconstruction, fresh source loading, unchanged live-card outputs and historical-route rejection controls."
+    }
+  ]
+}