Repository navigation
perf(buck2): reuse validated CAS files within publication batches - #1831
schickling-assistant wants to merge 1 commit into
Conversation
agent-identity: agent/effect-utils/buck2-migration-dev3 agent-persona: generalist agent-supervisor: unavailable agent-tool: OMP agent-tool-version: 18.6.0 agent-runtime: OMP 18.6.0 agent-model: anthropic/claude-opus-5-5
schickling-assistant
added
origin:agent
Pipeline traces20 jobs; 4 successful, 1 failed, 0 cancelled, 15 skipped
All other jobs (15)
Main baseline samples (by job): tested-tree n=0, default-ref-policy n=7, pr-reviews-resolved n=0, source-shape n=7, quality n=0, trusted-buck2-remote-cache-proof n=0, deploy-storybooks n=7, publish-products n=0, test-live-deploy-ci-tools n=7, test-integration-notion n=7, test[runner=namespace-profile-macos-arm64] n=7, test-integration-restate n=7, test-playwright-tui-react n=0, build-products n=0, test-playwright-utils n=0, weaver n=7, pr-a-inert-buck n=0, cargo n=7, test[runner=namespace-profile-linux-x86-64] n=7, notify-alignment n=7. 15 job(s) without a start time are omitted from the timeline, not measured as zero. |
schickling-assistant
changed the title
|
Parked: superseded by the current mg600 levers (revert #2280, A2a, prefetch #2290, macOS rebalance). Branch kept. Posted on behalf of @schickling
|
|
Parked: superseded by the current mg600 levers (#2280 revert, A2a, #2290 prefetch, macOS rebalance). Posted on behalf of @schickling
|
Problem
The last ten completed merge-group runs had test-lane execution medians of 694 s Linux and 929 s macOS, above the 600 s target. In run 8834,
buck2:editor:publish:testtook 418 s Linux / 361 s macOS, with approximately 358 s / 339 s remaining after whole-workspace authority and the Buck build. Existing package timings do not distinguish mutable-root hashing, private copy/materialization, snapshot proof and CAS candidate rehash.Goal
Measure the bottleneck, then avoid repeated cold copying and CAS candidate hashing by reusing validated CAS-file hints within one locked publication batch. Preserve all source-root and payload integrity proofs. No measured optimized speed-up or satisfaction of the 600 s lane contract is claimed.
Decisions
--metricsis opt-in; default native stdout and hash framing remain unchanged.Verification
Added cold/warm batch counter assertions and five-view reuse, before/after-admission mutation, and non-admitted-source regressions to the editor-view integration fixtures. Added native fingerprint byte-count/framing regressions. No focused test suite or separate validation build was run; integration-owner verification remains required.
gate-slot --class heavy -- devenv --no-tui tasks run genie:run: exit 0; generated files unchanged. Its prerequisite bootstrap successfully exercised the new native metrics and TypeScript parser (431 source walks, 857,769,053 source bytes; 46,838 cold copies/candidate hashes). This was a loaded local Linux bootstrap, not a CI workload or speed benchmark.gate-slot --class gate -- devenv --no-tui shell -- env DEVENV_TASK_PASSTHROUGH=1 oxfmt <four changed TypeScript/Markdown files>: exit 0; 473 ms formatter time. The initial direct oxfmt invocation was refused by the task wrapper; the wrapper's documented passthrough selects the pinned executable for scoped formatting.gate-slot --class gate -- devenv --no-tui shell -- env DEVENV_TASK_PASSTHROUGH=1 cargo fmt --manifest-path rust/buck2-tools/core/Cargo.toml --package buck2-tool-core: exit 0.heavyCiIfadmits merge-group/fallback-main execution, not PRs or dispatch; the heavy-proofs label does not override this.9b843b753ee4e1167f64aea09c663625338b0f16before a latercheck:quickfailed on two instrumentation lint violations. Those explicit-Boolean/named-argument violations are corrected in this slice; the check was not rerun. Retained artifact contains the cold publication and five warm samples.All current Linux and Darwin bounded suites, source complements, module fixtures, native-product smoke checks and final collection census remain unchanged. The macOS lane stays required.
Complexity
No new dependency or persistent cache schema. A numeric work accumulator and optional callbacks expose existing observations; a bounded, batch-owned Map retains validated file hints. Existing store locks, immutable CAS checks and full integrity proofs remain authoritative.
Concerns
The native fingerprint output capability and TypeScript consumer must deploy together. Counter sums include overlapping worker time, while the content-hash counters cover candidate hashing rather than every CAS maintenance read. Unsuccessful publications may expose only completed native walk work. Current policy provides no PR/dispatch admission for the required Linux/macOS test lanes; pre-merge attribution uses the existing empirical publication probe, while final lane execution evidence must come from merge-group runs after merge. This PR remains draft and is not enrolled or armed.
Friction & bottlenecks
Required local generation used normal heavy admission: 11,491 s queued, 919 s held; an earlier 60 s admission budget expired with 20 jobs ahead. No priority slot was used. Pinned formatting subsequently queued for 127 s and held for 4 s. These are shared local capacity costs, not GitHub lane execution measurements. No private runner or store state is used as CI evidence.
Follow-ups
The integration owner must run the added regressions and compare optimized publication on the unchanged Linux/Darwin test censuses, then retain post-merge required-lane execution evidence. Root-digest reuse remains out of scope: it requires an immutable ownership receipt that current
readRootspaths do not provide. Neither the empirical publication probe nor a green PR quality check substitutes for the final required-lane execution evidence.References
packages/@overeng/buck2-tools/README.md: publication integrity and work-counter contractgenie/ci-workflow/test-platforms.json: unchanged platform coverageAuthored by schickling-assistant (coding agent: openai-codex/gpt-6.1-sol).
Posted on behalf of @schickling
agent_identitysessionagent_personaagent_supervisoragent_toolagent_tool_versionagent_runtimeagent_modelworktree