Skip to content

ci: establish dedicated nightly perf validation (#2085) - #2088

Merged
justinjoy merged 1 commit into
mainfrom
codex/2085-hosted-qualification
Oct 5, 2026
Merged

justinjoy merged 1 commit into
mainfrom
codex/2085-hosted-qualification

Conversation

@justinjoy

@justinjoy justinjoy commented Oct 5, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Establish the dedicated Linux nightly performance owner before changing the
hosted PR validation contract tracked in #2085 and blocking #2077.

  • Run Linux performance work only on [self-hosted, Linux, X64, perf],
    scheduled or manually dispatched on canonical main, with a shared queue.
  • Use the runner's local semantic-reasoning:ubuntu26 Docker image without
    pulling it; execute the inspected image ID and record its identity.
  • Require nine actual CRDT/CSPA trials, gold correctness, CoV eligibility,
    host readiness, and absence of CPU throttling. Preserve existing targets.
  • Report acquisition completeness separately from the performance verdict;
    skips, missing samples, and target failures do not become a green result.
  • Prebuild measurement binaries and prevent test-time rebuilding. Preserve
    broader TRACE suite, portfolio, DOOP, and Windows coverage by default.
  • Provide an explicitly partial strict_smoke dispatch for validating the
    CRDT/CSPA acquisition path without running the entire nightly workload.

Evidence and sequencing

Hosted calibration runs 37288534027 and 37307436532 produced no raw timing on
9 of 12 VM observations. Measured identical PR code passed or failed the old
CRDT absolute threshold depending on CPU model; all measured CSPA samples
missed the old absolute threshold. This does not establish a source regression
in #2077.

This PR leaves the hosted required check unchanged and does not close #2085.
After merge, manually dispatch Perf Nightly on main with strict_smoke=true
and require complete, eligible acquisition. An existing target miss remains a
failed performance verdict. Then the next PR can establish hosted correctness
checks with strict performance checks owned by nightly, and #2077 can be rebased
and its required CI rerun.

Validation

Focused Docker, authentic host telemetry, measurement-log, Meson target
selection, governor, DOOP, Windows, TSan, and release RSS contracts are checked
locally. Python checks use a uv virtual environment under HOME.

The actual dedicated runner/local-image acquisition must be verified after
merge: this environment has no Docker daemon. The installed actionlint does
not yet recognize GitHub's concurrency.queue; its other diagnostics are
checked separately.

CI failure follow-up

The first GCC build failed the Python encoding gate and Bash construct ratchet.
The original commit was amended to specify UTF-8 in all 11 reported operations,
including the inherited hosted_perf_calibration.py operation on main, and to
write the telemetry fixture using individual printf calls that the static
shell checker can resolve. Neither gate was weakened.

Local revalidation: encoding scan (115 files, zero findings), seven encoding
self-tests, Bash construct self-test and real setup-only Meson introspection
(78-file closure, zero findings), eight Docker contracts, release RSS contract,
and real capture/check telemetry fixtures including rejection cases all pass.
Full build/ABI validation is being rerun by GitHub CI on the amended commit.

@justinjoy
justinjoy force-pushed the codex/2085-hosted-qualification branch from b965855 to 24ca034 Compare October 5, 2026 13:49
@justinjoy
justinjoy merged commit 793773e into main Oct 5, 2026
14 of 18 checks passed
@justinjoy
justinjoy deleted the codex/2085-hosted-qualification branch October 5, 2026 13:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Establish a trustworthy contract for the hosted required performance gate

1 participant