ci: establish dedicated nightly perf validation (#2085) - #2088
Merged
Merged
Conversation
justinjoy
force-pushed
the
codex/2085-hosted-qualification
branch
from
October 5, 2026 13:49
b965855 to
24ca034
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Establish the dedicated Linux nightly performance owner before changing the
hosted PR validation contract tracked in #2085 and blocking #2077.
[self-hosted, Linux, X64, perf],scheduled or manually dispatched on canonical
main, with a shared queue.semantic-reasoning:ubuntu26Docker image withoutpulling it; execute the inspected image ID and record its identity.
host readiness, and absence of CPU throttling. Preserve existing targets.
skips, missing samples, and target failures do not become a green result.
broader TRACE suite, portfolio, DOOP, and Windows coverage by default.
strict_smokedispatch for validating theCRDT/CSPA acquisition path without running the entire nightly workload.
Evidence and sequencing
Hosted calibration runs 37288534027 and 37307436532 produced no raw timing on
9 of 12 VM observations. Measured identical PR code passed or failed the old
CRDT absolute threshold depending on CPU model; all measured CSPA samples
missed the old absolute threshold. This does not establish a source regression
in #2077.
This PR leaves the hosted required check unchanged and does not close #2085.
After merge, manually dispatch
Perf Nightlyonmainwithstrict_smoke=trueand require complete, eligible acquisition. An existing target miss remains a
failed performance verdict. Then the next PR can establish hosted correctness
checks with strict performance checks owned by nightly, and #2077 can be rebased
and its required CI rerun.
Validation
Focused Docker, authentic host telemetry, measurement-log, Meson target
selection, governor, DOOP, Windows, TSan, and release RSS contracts are checked
locally. Python checks use a uv virtual environment under HOME.
The actual dedicated runner/local-image acquisition must be verified after
merge: this environment has no Docker daemon. The installed actionlint does
not yet recognize GitHub's
concurrency.queue; its other diagnostics arechecked separately.
CI failure follow-up
The first GCC build failed the Python encoding gate and Bash construct ratchet.
The original commit was amended to specify UTF-8 in all 11 reported operations,
including the inherited
hosted_perf_calibration.pyoperation on main, and towrite the telemetry fixture using individual
printfcalls that the staticshell checker can resolve. Neither gate was weakened.
Local revalidation: encoding scan (115 files, zero findings), seven encoding
self-tests, Bash construct self-test and real setup-only Meson introspection
(78-file closure, zero findings), eight Docker contracts, release RSS contract,
and real capture/check telemetry fixtures including rejection cases all pass.
Full build/ABI validation is being rerun by GitHub CI on the amended commit.