You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
perf(test): enforce a per-suite wall-clock budget in CI, warn then fail #10230
Parent: #3716 (slice 3 of 3). Runs in parallel with slice 1.
Runs in: lane.
Problem
#3716 criterion 6: a documented per-suite wall-clock budget that warns above a threshold and fails above a ceiling, so a suite cannot grow past what a contributor can run. Today nothing enforces a per-suite time: scripts/plan-test-lanes.sh uses scripts/suite-seconds.txt only to size and balance CI legs (ceil(seconds / budget)), and fleet-state.test.sh once passed the 600 s tool maximum on its own (#3716 comment). #8757 covers the separate local affected-run budget; this slice is the per-suite CI check.
Fix
Add a CI check that reads each suite's measured wall seconds from the test legs of the run and compares it with a documented threshold and ceiling: above the threshold prints a warning naming the suite; above the ceiling fails. Seed the threshold and ceiling from the current scripts/suite-seconds.txt distribution so no suite on main fails on day one; state the numbers and their basis where the check is documented.
A suite that legitimately needs more gets a named per-suite exception in one listed file, so an exception is visible in review.
Document the budget in the scripts' docs where contributors read how to run tests.
Done when
A test for the check shows a suite above the threshold produces a warning and exit 0, and a suite above the ceiling exits non-zero
The check runs on pull requests and is green on main's current suite times
The threshold, ceiling and their basis are written in one documented place, and the exception list is one file
ci-status is green on the PR head
Split by the backlog HITL session (2026-10-25, owner rule Q93: split large work before a run picks it up). This was generated by AI.
Proposed work class: work-class: scoped
Basis: automated intake triage; the lanes App applied this class.
Size: Medium
Complexity: Medium
Uncertainty: Known
Flags: none
Blocked by: none
Understood: add a CI check that reads each suite's measured wall seconds from the test legs. It warns above a documented threshold and fails above a ceiling. The numbers are seeded from scripts/suite-seconds.txt so main stays green. A single listed file holds the per-suite exceptions, and the budget is documented where contributors read how to run tests. The done-when items are specific, so nothing is missing. Related, not duplicate: #10200 (per-suite timing option and baseline doc), which looks like the sibling slice that runs in parallel.
Activity
github-actions commentedon Oct 26, 2026
Proposed work class: work-class: scoped
Basis: automated intake triage; the lanes App applied this class.
Size: Medium
Complexity: Medium
Uncertainty: Known
Flags: none
Blocked by: none
Understood: add a CI check that reads each suite's measured wall seconds from the test legs. It warns above a documented threshold and fails above a ceiling. The numbers are seeded from scripts/suite-seconds.txt so main stays green. A single listed file holds the per-suite exceptions, and the budget is documented where contributors read how to run tests. The done-when items are specific, so nothing is missing. Related, not duplicate: #10200 (per-suite timing option and baseline doc), which looks like the sibling slice that runs in parallel.
Automated triage, applied by the lanes App.