Skip to content

perf(ci): log pr-merge's job-token calls when a run stops before the lane token - #10618

Merged
kyle-sexton merged 3 commits into
mainfrom
perf/pr-merge-superseded-api-accounting
Oct 28, 2026
Merged

kyle-sexton merged 3 commits into
mainfrom
perf/pr-merge-superseded-api-accounting

Conversation

@kyle-sexton

@kyle-sexton kyle-sexton commented Oct 28, 2026 •

Copy link
Copy Markdown
Member

No related issue: follow-up to #10601, job-token calls of superseded pr-merge runs unlogged

Summary

Between 04:45 and 05:45Z, 16 of 31 pr-merge runs logged no api-calls notice. All 16 were pull_request_target runs whose merge or enable-auto-merge job stopped at "Check for a newer run that does this work" (one, run 40635471723, at the lane token's superseded refusal). Those jobs still called GitHub with the job token: who.mjs reads the run record, and superseded.mjs lists the pending pr-merge runs and reads each candidate's jobs. Nothing counted those calls, for two reasons:

  • The job-end log step ran only if: always() && steps.lane-token.outputs.token != '', so it never ran in a job that stopped before it minted the lane token.
  • The steps before the lane token set no API_BUDGET_TOKEN. createGitHub already records every response's rate-limit reading in $RUNNER_TEMP/api-budget/<token> through the rate guard's latch (rate-guard.mjs guard → recordReading), but only when API_BUDGET_TOKEN names the bucket. Without it, it writes nothing. So the shared client needed no change.

Fix

  • Every job-token createGitHub step in merge and enable-auto-merge now sets API_BUDGET_TOKEN: github-token. That is 14 steps: who, superseded, rotation, and the kept-file readers (head refs, verdict reads, artifact listing, lane records, verifier answers). The new test also checks every other pr-merge job and finds no other step missing it. Steps that pass the lane token keep their existing lanes-app bucket.
  • "Count the sweep's refusals" now reads the run's start through the wrapped gh. It sources api-budget.bash and calls api_budget_wrap_gh github-token, so that read leaves a reading. Its script, sweep-summary.mjs, makes no GitHub call, so the step sets no API_BUDGET_TOKEN.
  • Both job-end log steps now run if: always() && needs.post-merge-dispatch.outputs.rate-limited == ''. They no longer depend on the lane token. tokens: lanes-app github-token is unchanged, because the action skips a token with no reading: the existing log-api-budget action test passes lanes-app with no readings and gets output only for github-token. The rate-limit exclusion keeps the existing rule that nothing runs after post-merge-dispatch's rate-limit stop. That stop is the job's first step, so the job made no call and never checked out .base.

Still not counted: refire-refused.mjs, which holds both tokens in one step, so a single bucket would mislabel one of them.

Verification

  • New test, <job>: a run that stops as superseded still logs its github-token calls, for merge and enable-auto-merge. It runs the real who and superseded steps, with the env read from pr-merge.yml, against a local server whose pending run has the job waiting. It checks that the output says superseded=true. It then runs the log-api-budget action body with the job-end step's tokens: and expects ::notice title=api-calls::github-token=3 not-modified=0 and no lanes-app line. It fails on main.
  • New test, every createGitHub step that calls GitHub with the job token alone names API_BUDGET_TOKEN github-token. On main it lists the 14 steps; it passes on this branch. It also fails a step that sets the variable when its script builds no client.
  • New test, the merge job's sweep-count read leaves one github-token reading. It runs the step's body with a fake gh. Before the wrapper it recorded no reading; now it records one. The existing rate-limit-stop test still shows the step makes no read after the gate's stop.
  • every job that calls GitHub with the job token logs its api-calls is tightened. It now rejects a log step whose condition depends on anything but always() and the rate-limit stop. On main it fails with ['merge', 'enable-auto-merge'].
  • Two older tests now pin the new condition (merge-job.test.mjs and workflow.test.mjs). The rate-limit test (<job> sends nothing and ends red when post-merge-dispatch stopped on a rate limit) passes unchanged.
  • node --test .github/scripts/pr-merge/*.test.mjs .github/actions/log-api-budget/*.test.mjs .github/actions/check-trusted-trigger/*.test.mjs: 1081 pass, 0 fail. scripts/check-affected.sh --tests: 10 passed, 6 skipped.

This PR touches pr-merge.yml, which is on the merge-holds path list, so a person lifts the hold before it merges.

Related

🤖 Generated with Claude Code

kyle-sexton and others added 2 commits October 28, 2026 02:03
…lane token

A merge or enable-auto-merge run that stopped as superseded made
GITHUB_TOKEN calls (who.mjs, superseded.mjs) but logged no api-calls
notice: the job-end log step ran only once the lane token was minted,
and the steps before it set no API_BUDGET_TOKEN, so createGitHub's rate
guard kept no readings.

Every job-token node step in both jobs now names API_BUDGET_TOKEN
github-token, and the job-end log step runs under always() unless
post-merge-dispatch's rate-limit stop ended the job. A token with no
reading still gets no line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kyle-sexton
kyle-sexton marked this pull request as ready for review October 28, 2026 06:20
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-28T06:27:14.890649Z f8802bb Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

github-actions Bot commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

claude-review reviewed f5da74960a3ce7f34d349b16da6b057a241bc157: 0 finding(s), 0 as inline comments.

Incremental review of f8802bb..f5da749 (pr-merge.yml, merge-job.test.mjs, workflow.test.mjs): no findings. I read the diffs and the surrounding code. I did not run the tests, because this lane has no shell.

  • Workflow change. "Count the sweep's refusals" now sources api-budget.bash and calls api_budget_wrap_gh github-token before its one gh api … --jq .run_started_at read. It drops the step-level API_BUDGET_TOKEN, since sweep-summary.mjs makes no GitHub call. This follows the same pattern as the other job-token steps (for example pr-merge.yml:317-319). The wrapper re-issues the call with -i, records the reading, and returns the body unchanged, so --jq still works. The LIMITED == true guard still skips the read after the gate's rate-limit stop.
  • merge-job.test.mjs. The rate-limit-stop test now gives the fake gh a header block, links .base, and sets RUNNER_TEMP, which the wrapped read needs. The gh-called-or-not assertion is unchanged.
  • workflow.test.mjs. The bucket test is narrowed to steps whose script calls createGitHub(. It also fails a step that names a bucket without building a client. Its text-match heuristic would mislabel a script that reaches createGitHub only through another module, and would then flag that script's bucket as "needless". I found no such step in the current set. The new sweep-count test runs the step body against a fake gh and checks the exact api-budget/github-token line (200 5000 4000 1000 9999999999 core). The printf escaping and the imports (symlinkSync, fileURLToPath, tempDir) check out.

Spend: no billable change. The new gh read in the sweep-count step counts against the job token's rate limit only.

@github-actions

github-actions Bot commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

claude-security-review reviewed f5da74960a3ce7f34d349b16da6b057a241bc157: 0 finding(s), 0 as inline comments.

Incremental review of f8802bb..f5da749, covering pr-merge.yml, merge-job.test.mjs and workflow.test.mjs. I found no security issues.

critical-findings: 0

  • pr-merge.yml ("Count the sweep's refusals" step): The step no longer sets API_BUDGET_TOKEN: github-token. It now sources .base/.github/actions/log-api-budget/api-budget.bash and calls api_budget_wrap_gh github-token, so its one gh read is counted in the job-end notice.
    • .base is the default-branch tip (ref: refs/heads/main, path: .base), not pull-request code. Other steps in the same file source the library the same way.
    • No token, permission, trigger or if: condition changed. The step still uses GH_TOKEN: ${{ github.token }}, and continue-on-error: true and timeout-minutes: 1 are kept.
    • The LIMITED guard that skips the read after the gate's rate-limit stop is unchanged. A rate-limit hold is therefore not weakened, and the wrapped gh only records readings.
    • The step also sets set -euo pipefail, so a failed library checkout fails the step. That is non-blocking because of continue-on-error.
  • merge-job.test.mjs and workflow.test.mjs: These are test-only changes that use temp directories, stubbed gh and node binaries, and a symlink to the repository root. They run no untrusted input and widen no control. The reworked API_BUDGET_TOKEN test now checks createGitHub( usage in the called script, which is a sound refinement.

The step runs the pr-merge.yml merge-hold path. I read its whole change and found no weakened hold, no widened trust control and no kill-switch change. No hold is set.

@kyle-sexton
kyle-sexton marked this pull request as draft October 28, 2026 06:25
@kyle-sexton
kyle-sexton marked this pull request as ready for review October 28, 2026 06:25

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

claude-review: 1 inline finding(s) on f8802bb65ca42906a6c7f6f8372d7b42c1dfb707; the summary is in this lane's pull request comment.

claude-lane: claude-review

Comment thread .github/workflows/pr-merge.yml Resolved
The API_BUDGET_TOKEN this PR set on "Count the sweep's refusals"
counted nothing: sweep-summary.mjs makes no GitHub call, and the
step's one read, the run's start, was a bare gh api. The step now
sources api-budget.bash and wraps gh, so that read leaves a
github-token reading, and the useless env line is gone.

The step-bucket test now asks for API_BUDGET_TOKEN only of a script
that builds a createGitHub client, and flags one set where the script
calls nothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kyle-sexton
kyle-sexton added this pull request to the merge queue Oct 28, 2026
Merged via the queue into main with commit caa0bfa Oct 28, 2026
84 of 86 checks passed
@kyle-sexton
kyle-sexton deleted the perf/pr-merge-superseded-api-accounting branch October 28, 2026 06:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant