Repository navigation
perf(pr-merge): batch main's whole-suite runs and stop sleeping after the dispatch - #10491
Conversation
… the dispatch dispatch-full-run and post-merge-dispatch read main's newest pr-require-checks dispatches first and start none within 30 minutes of the last; post-merge-dispatch starts main's tip on its next scheduled run when no run covers it, for any merge and whatever the kill switch says. dispatch-full-run ends after it dispatches; post-merge-dispatch sends a red full run to post-merge-demotion-alert.yml once per run. pr-require-checks.yml gives attempt 1 of a main dispatch one concurrency group with no queue key and cancel-in-progress false; a re-run keeps its own group. The post-merge check and the alert skip a run that group replaced before it started (cancelled, no job). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
847d3dd to
c482234
Compare
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c482234897
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
claude-review reviewed Code review: PR #10491 (batch main's whole-suite runs, drop the post-dispatch sleep)Result: 0 findings. This was a static read of What I checked
Notes, not counted as findings
|
|
claude-security-review reviewed No security findings. I read the whole diff and the PR context. The files that carry a trust control are critical-findings: 0 manifest: no plugin manifest is touched. What I checked
Spend: this only changes spend on GitHub Actions runner minutes. The PR estimates a drop from about 9,740 to at most 4,100 runner-min a day, and no billing setting changes. It is not a hold. Non-security note: the PR itself flags a throughput risk with |
…ed job on the kill switch A cover read refused for a rate limit now stops the step red with rate-limited=true. post-merge-dispatch classifies it through rate-stop.bash's rate_stop_from, as its alert step does, and adds the stop to the job's rate-limited output, which the merge and enable-auto-merge jobs already stop on; its dispatch, re-run and alert steps skip after it. dispatch-full-run has no checkout of the guard, so it matches the pattern rate_stop_from reads, as its removed wait step did, and its dispatch skips. post-merge-dispatch reads the cover and dispatches only while the lane is live (kill switch off, report-only off), as its post-merge check does, and sends a red run to the alert only with the kill switch off. dispatch-full-run's push carve-out stays. The tests that pinned the cover and alert as running whatever the switches say now pin the reviewed behavior instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The kill switch and report-only now stop everything post-merge-dispatch does that spends API or starts a run: the cover read, the pr-require-checks.yml dispatch on main and the alert dispatch. The alert step had only the kill-switch condition; it now also needs the lane live. dispatch-full-run's push carve-out stays. A test runs the kill-switch action with CLAUDE_LANES_DISABLED and with CLAUDE_LANES_REPO_DISABLED set, then evaluates every later step of the scheduled job over its output: none that starts a full run or sends an alert runs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… on main The alert step treated any artifact named pr-merge-alerted-<id> as a send already made, whoever uploaded it, so a pull request run could suppress a red run's alert. It now asks babysit_postmerge.py --lane-alerted, which accepts the record only from a sender the dispatch and re-run records' rule accepts (_lane_sender: this repository's pr-merge.yml, a schedule or workflow_dispatch run, on main, head commit on main's tip). _lane_record and the new lane_alerted share one _lane_proof helper. Exit 75 stops the step red as a rate limit; any other failed read sends nothing for that run, as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
--lane-alerted answered "not recorded" with exit 1, which is also what Python exits with on any uncaught exception, so a crashing check fell through to the send and sent the same red run again every tick. It now answers EXIT_NOT_ALERTED (4), a code no crash (1), argparse refusal (2), failed read (3) or rate limit (75) produces. The alert step sends only on 4; 1 and every other unknown code warn and leave the run to the next run, as a failed read does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
The pr-unblock lane stopped on this pull request and needs a person: the lane could not resolve every conflict.
To resume, deal with the reason above, remove the gh workflow run pr-unblock.yml -R melodic-software/claude-code-plugins -f pr-number=10491 -f activity=resolve-conflicts |
…-batching # Conflicts: # .github/scripts/pr-merge/workflow.test.mjs
|
The pr-unblock lane escalated this pull request as |
Post-merge mutation testingKilled 0, survived 0, timeout 0, no coverage 0, invalid 0, not run 8.
The run's summary has the full report. |
No related issue: efficiency plan items 5 and 7(b)
Summary
Main runs the whole pr-require-checks suite once per merge: 116
workflow_dispatchruns a day at a median 84.1 runner-min, about 9,740 runner-min a day. The push job that starts each run then sleeps on a held runner (up to 13 reads, 3 minutes apart) only so a red run reaches post-merge-demotion-alert.yml: 105 push runs a day at a median 644 s, about 1,130 runner-min a day. Figures are from the efficiency plan of 2026-10-26.This PR caps main's whole-suite runs at one per 30 minutes, each of main's tip and so covering every merge since the last. It also ends the push job once it dispatches. The owner decided both (plan items 5 and 7(b)). The cost is attribution: a red run of main now spans up to about 3 merges instead of 1.
Fix
Item 5: the cap, and a run for every merge. A new step,
Read main's full-run cover, has the same body indispatch-full-runandpost-merge-dispatch. It reads the newest 20 pr-require-checks.yml dispatches on main that github-actions[bot] started, leaving out cancelled ones. It sets two outputs:capped=truewhen one of them was created in the last 30 minutes.uncovered=truewhen none of them is at this run'sgithub.sha.A failed read sets neither output, so the job dispatches as it did before this change.
dispatch-full-run(push, and the merge job's path) dispatches only when not capped.post-merge-dispatch(the 15-minute schedule) dispatches when not capped and either the post-merge check asks for a run (needsDispatch, as before) or main's tip is uncovered. This is the backstop. The cover step has noif:, so it runs while the kill switch or report-only is on.check-kill-switchalways exits 0 (.github/actions/check-kill-switch/action.yml:5), so the steps before it cannot fail the job. Any merge, whoever made it, gets a run of main's tip at most about 30 + 15 minutes after it lands: from the next push after the window, or from the first scheduled run after the window. A merge is never left without a whole run._judge'srelevant()isrun.created_at >= merged_at and ctx.contains(commit, run.head_sha), so one run of the tip verifies every merge in the batch. A merge with no run yet readsdispatch-run-pending, and a run still in progress readsdispatch-run-in-progress. Both are pending, which is exit 5 (verdict_exit: unverified with no failure isEXIT_UNVERIFIED), not a failure. The cap plus the backstop stays far inside the check's 6-hourDISPATCH_DEADLINE.Item 7(b): no sleep after the dispatch.
dispatch-full-runnow has three steps: read the cover, dispatch, record. Its timeout drops from 45 to 10 minutes.post-merge-dispatch, in a new step,Send a red full run to the demotion alert. Each scheduled run reads the pr-require-checks dispatches on main from the last 6 hours. It sends the oldest red one that was not sent before, judged on attempt 1, by run id and head SHA, and records it aspr-merge-alerted-<id>. It sends one run per job, so a red streak costs one alert job per tick. It runs whatever the kill switch says, as the old wait step did.post-merge-dispatchalready holdsactions: write.dispatch-full-runkeepsactions: writeonly.rate-limitedoutput carries the stop. Any other failed read or send leaves the run to the next tick.pr-require-checks.yml concurrency (Automation Lanes, kept minimal). One predicate and one expression change. A
workflow_dispatchonrefs/heads/mainatrun_attempt == 1joins the grouppr-require-checks-main-dispatch, with noqueuekey andcancel-in-progressfalse. Every other run keeps its group andcancel-in-progress: true, as before. A re-run keeps its ownrun_idgroup, so a newer dispatch can never replace the one re-run a red run waits on. The cancel expression follows release-publish-plugins.yml's precedent (${{ github.event_name != 'push' }}). The github-iac#378 reason for dropping shared groups does not apply here. That reason was a requiredci-statusleft unreported on a PR. A main dispatch reports no required check, and theci-lanesstatus readers (fix-ci-dispatch.mjs, the contract-only run) read PR heads only.The docs this relies on (Control the concurrency of workflows and jobs, source
data/reusables/actions/actions-group-concurrency.mdin github/docs, read 2026-10-27):With
queue: max, pending runs would pile up and nothing would batch. A workflow test pins that the group sets noqueuekey.A replaced run is no signal. GitHub creates the replaced run and then cancels it before any job starts. Runs 40192695442 and 40112740615 (work-resolve-issue.yml, a workflow-level group with cancel false) read
conclusion: cancelled,total_count: 0jobs, andrun_started_at == created_at.run_started_attherefore cannot tell such a run from one that started. The jobs count can. Each reader of a cancelled main full run:babysit_postmerge.py: changed. On main it read a replaced run as RERUN, so the merge was heldattribution-pending, the run was marked for a re-run of no jobs, and a replaced run could become the baseline._Context.replaced()(attempt 1, cancelled, 0 jobs) filters it out ofcheck()'s full runs, the dispatch-listing completeness check,_lane_rerunsand_baseline. A run cancelled after its jobs started still reads red.collect.mjs:133: no change. It already skipscancelled.report.mjs:246andred-main.mjs:81: no change. They act onfailureonly.Known race. A push job and a scheduled run can both read the cover as uncapped and both dispatch. That costs at most one extra run per window, and the group absorbs it: the second run waits, or replaces a waiting one.
API cost. About 2 reads per scheduled run (cover, alert listing) and 1 per push (cover). Before, each merge cost up to 13 polls.
Verification
Tests that fail on main and pass here. Each was checked by swapping in origin/main's three workflow files:
.github/scripts/pr-merge/workflow.test.mjsgithub.shaas the tip.queuekey and do not cancel. A re-run keeps its own group. The control test (every other run keeps its group and cancel) passes on main and here..github/scripts/pr-merge/merge-job.test.mjs: these port the wait step's alert-send tests to the new step.if:and position.plugins/source-control/skills/babysit-prs/scripts/tests/test_babysit_postmerge.pyReplacedRuns: a replaced run neither holds nor re-runs the merge, and is no baseline. The control (cancelled after jobs started still waits for its re-run) passes on both.scripts/post-merge-demotion-alert.test.sh: a dispatched run cancelled before any job started files nothing, and one cancelled once its jobs started files a record.Runs:
node --teston both pr-merge test files: 148 pass.scripts/post-merge-demotion-alert.test.sh: PASS=156.pytest: 3191 passed. One earlier-xrun stopped on 1 failure that the full run did not repeat. Its name was lost, so it is UNVERIFIED as a flake.actionlinton the three workflows: clean.scripts/ci-review-lanes.test.sh: PASS=98.scripts/run-ruff.shcheck and format: clean.scripts/check-affected.sh: 14 passed, after merging origin/main.Expected re-measure (from the plan):
workflow_dispatchruns of pr-require-checks on main per day: 116 → at most 48.limits/concurrency.jsonl).pushsuccess median: 644 s → under 60 s, saving about 1,130 runner-min a day.Throughput risk the plan did not weigh.
merge.max-in-flightis 5 (docs/conventions/pr-pipeline.yaml). It counts queued lane entries plus merged lane PRs not yet verified (babysit_merge_policy.in_flight). A merge now waits up to 30 minutes for its batch run, plus the run, plus a tick. Main's whole runs take about 8 minutes of wall time: p50 8 and p90 9 over the last 53 attempt-1 dispatch runs, read 2026-10-27. So verification of a merge moves from about 8-23 minutes to as much as about 68 minutes (30 cap, 15 to the next tick, 8 run, 15 to the tick that reads it). At about 5.5 lane merges an hour, the lane can reach the in-flight cap and hold new enqueues for part of each window. Re-measure the gate'sunverified-auto-mergerefusals after merge. The owner's options are to keep 30 minutes, raisemax-in-flight, or shorten the cap.Related
🤖 Generated with Claude Code