Skip to content

perf(pr-merge): carry completed run attempts across post-merge-dispatch ticks - #10834

Merged
kyle-sexton merged 3 commits into
mainfrom
perf/post-merge-dispatch-runs-listing
Oct 29, 2026
Merged

kyle-sexton merged 3 commits into
mainfrom
perf/post-merge-dispatch-runs-listing

Conversation

@kyle-sexton

Copy link
Copy Markdown
Member

No related issue: an API-budget cut found in the 2026-10-28 GITHUB_TOKEN breaker trips on pr-merge's post-merge-dispatch job; no issue tracks it.

Hold paths: this changes babysit_postmerge.py and .github/workflows/pr-merge.yml, both hold paths, so the owner approves the merge. No permission, secret or token change.

Summary

The post-merge check in pr-merge's post-merge-dispatch job re-read the earlier attempts of every re-run full run in its window on each 15-minute tick: 8 GET .../actions/runs/{id}/attempts/{n} calls per tick on all five full checks measured on 2026-10-28. A completed attempt cannot change, so the check now carries those attempts from one tick to the next in the report artifact the job already keeps.

Calls per run: 50-57 → about 42-49, measured from each job's babysit-gh-calls line. That is 8 × 4 ticks/h ≈ 32 of the job's ≈ 200 GITHUB_TOKEN calls/h (≈ 16%). A run that is newly re-run still costs its one read, on the first tick that sees it.

Fix

  • babysit_postmerge.py: _Context._attempt takes an attempt from the file named in BABYSIT_POSTMERGE_ATTEMPTS when the entry has the same run id, attempt number and head SHA as the listed run. Otherwise it reads the attempt as before. save_attempts writes back only the completed attempts this check used.
    • Fails closed. A file with the wrong version or repository, a key that does not match its entry, an attempt not marked completed, a malformed SHA, or a field the reader does not use is ignored whole, and every attempt is read live.
    • Without the variable nothing changes, so the merge job's check behaves as before.
  • pr-merge.yml, post-merge-dispatch only: the check step copies attempts.json from the last run's kept report (last-report.mjs downloads it only from a trusted pr-merge run on main, the same boundary as the lane records) to the path it names. It keeps the file in the report whether the check runs or an idle tick reuses the report.
  • Changelog fragments for source-control and pr-pipeline.

Not done, and why. The brief proposed indexing dispatch_runs_at by head SHA in one dated listing. That finding did not reproduce:

  • None of the six runs read made a repos/{}/{}/commits call, so dispatch_runs_through, the per-SHA loop, never ran. The six runs are 40783844800, 40786123150, 40787501102, 40789255688, 40790836092 and 40795766929.
  • That path fires only for a merge past the 6-hour deadline with no dispatch of its own.
  • It also exists on purpose as a read independent of the dated listing, so folding it into a dated listing would defeat its point.

Three other cuts were rejected because each would change what the check decides:

  • --branch main removes the report's shared block, so the output changes.
  • Reusing the main dispatch listing in _lane_reruns opens a race for a run created between the two reads.
  • Caching failing_checks jobs and annotations is unsafe: the jobs API can list a completed job's steps short, so a cached answer can differ from a later live read.

Verification

  • tests/test_babysit_postmerge_attempts.py (new) is a pin. The source of truth is the check with no attempts file, which is the code before this change.
    • A carried check returns report() identical to it, in 2 fewer attempt reads (2 → 0).
    • Garbage, another repository or version, an in-progress, misfiled or extra-field entry, and an entry at another head SHA each give the same report with every attempt read live.
    • An in-progress attempt is never written.
    • A check that looks up no attempt leaves the file as it was.
  • Each of four mutants fails the new tests: skip loading the file; skip validating entries; drop the head-SHA match; write attempts that are not completed.
  • Full babysit-prs suite: 3401 passed, 4523 subtests passed. node --test .github/scripts/pr-merge/merge-job.test.mjs: 82 pass, with a new test that the step hands the file to the check and keeps it on both the check path and the reuse path. scripts/run-ruff.sh check and format --check are clean.
  • Steady state: 10 runs of pr-require-checks.yml on main at run_attempt 2 sit in the check's window, which starts at 2026-10-27T16:01Z. The newest finished at 2026-10-28T22:37Z, before the first measured tick (22:57Z), so the same attempts recur and each tick after the first reads none of them.

Related

🤖 Generated with Claude Code

kyle-sexton and others added 3 commits October 28, 2026 21:27
…ch ticks

Every post-merge check re-read the earlier attempts of each re-run full
run in its window (8 attempt reads per tick on the 2026-10-28 runs). A
completed attempt never changes, so the check now answers one from a file
pr-merge.yml's post-merge-dispatch carries in its kept report, for the
same run, attempt and head SHA, and writes back the completed attempts it
used. Any unusable file is no file, and every attempt is read as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kyle-sexton
kyle-sexton marked this pull request as ready for review October 29, 2026 02:13
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-29T02:20:41.259889Z c663723 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

Copy link
Copy Markdown
Contributor

claude-security-review reviewed c663723fc7450d0f94eed914b8fec2c38b9da2a0: 0 finding(s), 0 as inline comments.

No security issues found.

critical-findings: 0

I read the whole diff, pr-merge.yml's post-merge-dispatch job and last-report.mjs. Both pr-merge.yml and babysit_postmerge.py are on the trusted-path list, so I read each change and what it gates.

  • Where the carried file comes from: the new attempts.json is copied only from $RUNNER_TEMP/post-merge-last/. last-report.mjs fills that directory, and only from newestTrustedRun filtered by onMain: a scheduled or hand-started run of main's pr-merge.yml whose head commit is on main. Only that run's own upload step writes the artifact (.github/scripts/pr-merge/last-report.mjs:9-11, :51-52). This is the same boundary as the lane records, so a pull request, fork or comment author cannot place content in it.
  • What a forged file could do, if one got through: it could hide a failed attempt 1. load_attempts rejects it unless the file's version and repo match, each key equals <run id>:<attempt>, the entry is completed with a full 40-hex SHA, and the entry holds no extra fields. _Context._attempt then uses an entry only when its head SHA equals the listed run's. Any rejected file is dropped whole and every attempt is read live. A completed attempt's conclusion, actor and head SHA do not change, so a genuine carried entry decides the same as a live read.
  • Merge hold: the change does not weaken or remove a hold. Without BABYSIT_POSTMERGE_ATTEMPTS, the merge job's check behaves as before. The tests pin the report as identical to the no-file check.
  • Workflow and token scope: GH_TOKEN and the job's permissions are unchanged. The new env var is a fixed runner.temp path, with no untrusted context in run: and no expression injection. cp copies between fixed paths. Writes use a pid-suffixed temp file and os.replace, and the log line strips \r\n% so it cannot inject workflow commands.
  • Cost: this removes GITHUB_TOKEN calls and adds no billed resource or paid plan, so there is no cost hold.

manifest fields: none touched.

@github-actions

Copy link
Copy Markdown
Contributor

claude-review reviewed c663723fc7450d0f94eed914b8fec2c38b9da2a0: 0 finding(s), 0 as inline comments.

No findings. I read the full diff and PR context. I also read Run.from_api and grepped for other callers of the module-level _attempt, which I confirmed are only in _Context._attempt. I did not run the tests, so the PR's pass counts are author-claimed and unverified.

  • Carry logic: _Context._attempt uses a carried entry only when run id, attempt number and head SHA all match the listed run. load_attempts validates the whole file or rejects it. _attempt_fields covers exactly the fields Run.from_api reads (id, run_attempt, status, conclusion, head_sha, head_branch, created_at, html_url, actor.login, triggering_actor.login). The entry == _attempt_fields(entry) check therefore rejects only entries with extra or missing fields. A carried attempt decodes to the same Run as a live read, so the report is unchanged.
  • Workflow: The cp steps are guarded by [ ! -f … ] ||, so they are safe under set -e. The env path and $RUNNER_TEMP/post-merge-attempts.json are the same file. The attempts file is kept alongside the lane records and travels the same trusted-run path through last-report.mjs, so its trust boundary matches the lane records'. The merge job names no file and is unchanged.
  • Tests: The Python tests pin the carried report to the no-file report and cover the fail-closed cases. The Node test covers the hand-off and keep on both the check path and the idle-reuse path.

Minor, not counted: save_attempts runs in finally and writes only the attempts used so far. If a check stops early (for example a rate-limit stop, code 75), the carried set shrinks to what was used before the stop. The next tick re-reads the rest live. This costs only extra calls and cannot change a decision, so I did not flag it.

Spend: The change removes GITHUB_TOKEN API calls and adds no spend. It changes no permissions or billing, and I returned no hold.

@kyle-sexton
kyle-sexton added this pull request to the merge queue Oct 29, 2026
Merged via the queue into main with commit 38293d4 Oct 29, 2026
76 checks passed
@kyle-sexton
kyle-sexton deleted the perf/post-merge-dispatch-runs-listing branch October 29, 2026 02:27
@github-actions

Copy link
Copy Markdown
Contributor

Post-merge mutation testing

Killed 0, survived 0, timeout 0, no coverage 0, invalid 0, not run 7.
Covered-code score n/a; mutation score n/a.

File Killed Survived Timeout No coverage Invalid Not run
plugins/source-control/skills/babysit-prs/scripts/babysit_postmerge.py 0 0 0 0 0 7

The run's summary has the full report.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant