Skip to content

perf(ci): measure unattributed GITHUB_TOKEN calls and cut three without changing behavior - #10141

Merged
kyle-sexton merged 6 commits into
mainfrom
perf/cut-github-token-calls
Oct 26, 2026
Merged

kyle-sexton merged 6 commits into
mainfrom
perf/cut-github-token-calls

Conversation

@kyle-sexton

Copy link
Copy Markdown
Member

No related issue: GITHUB_TOKEN budget work against the owner's 1,000/h plan (R26). The study ran in the session and was not filed.

Summary

GitHub documents GITHUB_TOKEN at 1,000 REST requests per hour per repository (REST rate limits). The lanes plan to that number, and the observed 5,000 header limit is headroom, never a target.

The 2026-10-25 budget study measured core on two hourly counters with different reset minutes. Over 07:00 to 15:00Z their mean window peaks were 2,674 and 1,792, about 4,470/h combined. That is roughly 4.5 times the plan. About 1,850 core/h (41%) came from jobs that post no api-budget notice, so it could not be attributed.

This PR has two parts. First, it measures what was unattributed. Second, it ships the three cuts that leave behavior unchanged. Cron cadence is not changed here. The pr-merge.yml edit is one wait-step hunk, and the rest of that file belongs to the concurrent pr-merge work.

Fix

Commit Change Calls removed (study estimate, core/h)
perf(ci): log the api-budget notice in the jobs that post none Every call through the rate guard leaves a reading under API_BUDGET_TOKEN. The log-api-budget action adds an exclusive api-calls <token>=N not-modified=M line next to the shared-counter cost. The notice is wired into intake-triage born-triaged and refine-select, pr-verify's two judge jobs, pr-unblock's four select/context jobs, post-merge-demotion-alert report-red-main, and the audit-lane-runs, delete-stale-branches, monitor-signals and reconcile-lost-runs jobs. 0. It adds no call. It turns the 41% unattributed into measurements.
perf(ci): poll rerun-ci-status's run with conditional GETs pr-feedback-dispatch rerun-ci-status sends each repeat poll with If-None-Match on the last ETag. A 304 stands for the last 200's body. Pace (60 s), cap (600 s) and decisions are unchanged. Study cut 7, estimated at 20-40. Not claimed, because whether GitHub counts an authorized 304 against GITHUB_TOKEN is unverified. The new not-modified count settles it on the first live run.
perf(ci): read the full run with conditional GETs in dispatch-full-run pr-merge dispatch-full-run sends each read of the full run after the first as a conditional GET. A 304 means the run is still going. The three-minute reads, 39-minute ceiling and red-run dispatch are unchanged. Study cut 10, estimated at 15-40. Up to 12 calls per merge burst. Not claimed, for the same 304 reason.
perf(ci): read report-escalation's comments 100 to a page report-escalation.sh's --paginate comments read asks for 100 per page, not the default 30. ceil(n/30) - ceil(n/100) calls per escalation on a PR with n comments, under 1/h. Mostly lanes App calls, not GITHUB_TOKEN.

Changelog fragment: .changes/pr-pipeline/perf-cut-github-token-calls-3f9a1c62.md (patch).

Verification

  • Rebased onto main. One conflict, in rate-guard.test.mjs: main's stop-file tests and this branch's reading test were both added. The resolution keeps both, with the union of their node:fs imports.
  • node --test over every *.test.mjs the branch touches or that names a touched workflow or script: 2256 tests, 2256 pass.
  • scripts/ci-rerun-ci-status.test.sh: 157 pass, 0 fail.
  • .github/scripts/pr-unblock/report-escalation.test.sh: 571 ok, 0 failed.
  • shellcheck and actionlint are clean on the touched files. scripts/check-changelog-fragments.sh --check passes.

Related

🤖 Generated with Claude Code

kyle-sexton and others added 6 commits October 25, 2026 16:32
Measurement first (report row 8 and the 41% unattributed, about 1,850
core/h): the rate guard every Node GitHub call goes through now keeps a
reading of each answer under API_BUDGET_TOKEN, and the log-api-budget
action adds an exclusive `api-calls <token>=N not-modified=M` line next
to the shared-counter cost. Wired into intake-triage born-triaged and
refine-select, pr-verify check-intent and check-intent-subscription
(step-level env; the log step sits before "Fail without a verdict"),
pr-unblock's four select/context jobs, post-merge-demotion-alert
report-red-main, and the audit-lane-runs, delete-stale-branches,
monitor-signals and reconcile-lost-runs jobs.

Estimated saving: 0 calls/h; it turns the estimates into measurements.
Behavior unchanged: recording writes a file under RUNNER_TEMP only when
the step names API_BUDGET_TOKEN, and adds no GitHub call. budgetedFetch
drops its own recordReading so the guard is the one recorder (its
existing test pins one line per answer). recordReading moves into
rate-guard.mjs (api-budget.mjs re-exports it) so the HIGH-floor import
closure does not grow. Not covered: intake-triage's main job (a
ci-workflows reusable) and bash `gh` calls left unwrapped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Report cut 7 (estimate 20-40 core/h). The poll of the pr-require-checks
run (pr-feedback-dispatch.yml rerun-ci-status, every 60 s up to 600 s)
now sends each repeat read with If-None-Match on the last 200's ETag; a
304 (gh exits 1 on it) returns that 200's body. The brief's "one wait"
or event-driven path would change when ci-status is re-run, so the poll
keeps its pace, cap and decisions; only the answer form changes.

Saving: up to (polls - 1) calls per waiting run, but NOT claimed: GitHub
documents that an authorized 304 does not count against the primary
limit, which is unverified for GITHUB_TOKEN. The 304 readings land in
this job's api-budget notice (api-calls ... not-modified=M with
x-ratelimit-used), the measurement path that settles it on the first
live run. Behavior unchanged: same reads, sleeps and outcome; a 304
stands for an unchanged run. Pinned by scripts/ci-rerun-ci-status.test.sh.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Report cut 10 (estimate 15-40 core/h). The wait step reads the
dispatched full run every 180 s, 13 times at most. Each read after the
first now carries If-None-Match on the last ETag; a 304 makes gh exit 1,
which the loop already treats as "read again next time", and a 304 can
only mean the run is unchanged, so still not completed. The brief's
"wait once" would hold the push group past the run's end and delay the
red-run dispatch, a behavior change, so the pace and ceiling stay.

Saving: up to 12 calls per merge burst, NOT claimed: an authorized 304
not counting is documented but unverified for GITHUB_TOKEN; the
rerun-ci-status readings settle it. Behavior unchanged: same reads,
sleeps, group hold and dispatch. A small hunk in the wait step only, to
ease the concurrent pr-merge.yml change. Pinned by pr-merge
workflow.test.mjs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Coordinator cut (b), per_page=100 on paginated reads that use the
default 30. A sweep of .github found one: report-escalation.sh's
`gh api --paginate "$issue/comments"`; the shared Node paginate() and
the other bash --paginate reads already ask for 100.

Saving: ceil(n/30) - ceil(n/100) calls per escalation on a pull request
with n comments (0 under 31 comments, 1 at 31-60). That read runs only
outside status mode, which the lanes App token drives, so it saves App
calls, not GITHUB_TOKEN ones; estimate under 1 call/h. Behavior
unchanged: --paginate still reads every page and jq -s merges them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kyle-sexton
kyle-sexton marked this pull request as ready for review October 26, 2026 02:42
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review 🔄 Running since 2026-10-26T02:42:49.309250Z b5568de Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

Copy link
Copy Markdown
Contributor

claude-security-review reviewed b5568dea92867b5ba27e561d95ba83bfe91d2e05: 0 finding(s), 0 as inline comments.

No security findings in this pull request.

critical-findings: 0

Scope read: all 22 files of the diff and the PR context. No instruction-surface deletions and no plugin manifests (plugin.json or marketplace.json) are touched. The protected-class register was not needed.

Trusted paths. No file under .github/standards/trusted-actors/** or .github/actions/check-trusted-trigger/** is edited, and there is no CODEOWNERS or .gitmodules change. pr-verify.yml does set API_BUDGET_TOKEN: github-token on the check-trusted-trigger gate step, so I read what that gates.

  • The only new behavior on that path is recordReading inside the rate guard's guard() wrapper.
  • recordReading runs after the response arrives and never throws. Its readingLine call sits inside the try.
  • It only appends to $RUNNER_TEMP/api-budget/<token>. It has no effect on the response or on the gate's decision.
  • The <token> path segment is validated by ^[a-z][a-z0-9-]{0,39}$ before join, so it cannot traverse paths.

Shell changes.

  • pr-merge.yml dispatch-full-run: a 304 exits gh with status 1, so || continue keeps waiting. That is correct, because 304 means the run is unchanged. The ETag comes from GitHub's own response header and is passed as a single array element to -H, so it cannot inject anything. The red-run dispatch and its head-SHA check are unchanged.
  • pr-feedback-dispatch.yml rerun-ci-status: a 304 returns the cached body only when a conditional header was actually sent. The old retry, rate-limit and decision logic is untouched.
  • report-escalation.sh: only ?per_page=100 is added to a fixed endpoint. No user input is involved.
  • log-api-budget/action.yml awk line: it counts lines and prints only integers plus the workflow-supplied token name. It adds no notice-injection surface.

Workflow changes.

  • The new log-api-budget steps run the action from the .base checkout, which is the trusted default-branch tip. They use if: always(), continue-on-error, and a 1-minute timeout.
  • The job-level API_BUDGET_TOKEN env is a constant. No github context is interpolated into any run: block.
  • The new steps add no permissions.

Cost. None. The new steps spend a few seconds of existing Actions minutes. The change removes no calls and adds none.

@github-actions

Copy link
Copy Markdown
Contributor

claude-review reviewed b5568dea92867b5ba27e561d95ba83bfe91d2e05: 0 finding(s), 0 as inline comments.

No findings on b5568dea. I read the full diff (22 files) and the PR context. I did not run any tests, shellcheck or actionlint; the pass counts in the PR description are the author's claims and I did not verify them.

What I checked by reading:

  • Rate-guard move. recordReading and readingLine now live in rate-guard.mjs, and latch.guard records every answer. request-budget.mjs no longer records itself. budgetedFetch reaches the guard through retryServerErrors, which wraps the fetch with latch.guard (check-trusted-trigger.mjs:150). So the existing request-budget.test.mjs reading test should still see one reading per wire call, with no double count. The 304 replay in send happens after the guard has already recorded the real 304.
  • API_BUDGET_TOKEN wiring. Every newly labelled job spends GITHUB_TOKEN. LANES_APP_ID in the audit-lane-runs workflow is an id, not a second token. The guard's one-name-per-process caveat does not bite here. The scripts I looked at (reconcile.mjs, monitor.mjs, cleanup.mjs, report.mjs, triage-sweep.mjs, collect.mjs, file-findings.mjs) go through the guard or createGitHub.
  • log-api-budget action. The new api-calls line is computed from the same per-token file that already produced lines, so it only runs when a reading exists. An empty count prints as 0 through %d. The cost= parsing is unchanged because the new line is not appended to $lines. The updated action test matches this: 2 counted calls and 1 304.
  • Conditional GETs.
    • In pr-merge.yml dispatch-full-run, a 304 makes gh exit 1 and the loop continues, which is the intended "still going". cond keeps the last ETag across failures.
    • In pr-feedback-dispatch.yml read_api, a 304 that was sent with a conditional header returns the stored body. A 304 without one still falls into the retry path. Pace, cap and decisions are unchanged. The ETag is tracked per path, so reads of other paths are never made conditional by mistake.
    • Both rely on bash 4.4+ for "${cond[@]}" under set -u, which the hosted runners have.
  • report-escalation.sh. ?per_page=100 with --paginate is correct, and the test stub's endpoint was updated to match.
  • post-merge-demotion-alert/workflow.test.mjs. The assertion now allows more than one run: line and still checks that none contains ${{.
  • Closing issue. None; the PR says it closes no issue.
  • Manifests, paved paths and rules. No plugin manifest changed. I did not resolve a standards index or read .claude/rules/ files.
  • Trusted paths. No changed file matches the listed globs.

Spend: the added log-api-budget steps are 1-minute continue-on-error steps on existing jobs. They add no API calls. The PR itself says the 304 savings are unverified and not claimed.

@kyle-sexton
kyle-sexton added this pull request to the merge queue Oct 26, 2026
Merged via the queue into main with commit 0ec4b67 Oct 26, 2026
67 checks passed
@kyle-sexton
kyle-sexton deleted the perf/cut-github-token-calls branch October 26, 2026 03:13
kyle-sexton added a commit that referenced this pull request Oct 26, 2026
Conflicts composed:
- .github/workflows/pr-merge.yml (wait for the full run): keep main's
  ETag-conditional `gh api -i` poll (#10141) and wrap it in this branch's
  stderr capture, so a rate-limited read still stops the wait red while a
  304 (gh exit 1, no rate-limit text) keeps polling.
- .github/scripts/pr-merge/workflow.test.mjs: runWait takes both
  notModified and limited/limitedMessage; reads is main's array of read
  lines, so the rate-limit tests assert reads.length.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant