Skip to content

ci: count every GitHub request the Node clients send, GraphQL split out - #10811

Merged
kyle-sexton merged 3 commits into
mainfrom
ci/count-every-node-github-request
Oct 29, 2026
Merged

kyle-sexton merged 3 commits into
mainfrom
ci/count-every-node-github-request

Conversation

@kyle-sexton

@kyle-sexton kyle-sexton commented Oct 29, 2026 •

Copy link
Copy Markdown
Member

No related issue: measured uncounted GITHUB_TOKEN spend, the Node half of #10765 (counted 1,359 against the header's 4,000 used for the 22:55-23:55Z pool-B hour)

Summary

Every GitHub request a lane script's Node client sends now reaches the job-end api-calls notice. #10765 did this for gh.

The premise this started from was off: onResponse never did the counting. Every createGitHub request already went through the rate guard (retryServerErrors → latch.guard → recordReading), which records a reading whenever the step names a bucket in API_BUDGET_TOKEN. The real gaps were these:

  • Factories that bypassed the guard. address-feedback's createGitHubApi, used by its select main, fix-ci-context and review-dependency. These jobs set API_BUDGET_TOKEN: github-token and still counted 0. Also the dispatch and feedback-dispatch factories when not handed budgetedFetch, and pr-verify's raw job-log read.
  • Node steps that named no bucket. pr-merge's Re-fire step, ci-status's verdict-reads restore, and work-unblock's check-closer.
  • Wrong bucket. A client on the job token inside a lanes App step counted as lanes-app. This hit enable-auto-merge's verify reader and clear-escalations' CHECKS_TOKEN and verify reader. That was pool-B traffic showing up in the other pool.

Call sites counted, of the 49 createGitHub/createGitHubApi calls: 43 before (given a bucket), 46 now, plus pr-verify's log read. The three steps above now name their bucket, and the second-token clients in 5 pr-merge steps move to github-token.

Fix

  • One recorder. rate-guard.mjs countRequests(fetch) keeps each answer's reading and changes nothing else: no latch and no retry. The guard sends through it, and so do the factories that never had the guard. A fetch that already counts carries a mark (countsRequests) and is not wrapped again, so budgetedFetch and guarded fetches handed to a factory are counted once.
  • Bucketing. budgetToken: a pinned token wins. Otherwise a request whose bearer is not the step's own token (GH_TOKEN, else GITHUB_TOKEN) counts under API_BUDGET_OVERRIDE_TOKEN, the rule babysit_gh.py already applies to gh calls. Otherwise it counts under API_BUDGET_TOKEN. Without an override set, nothing moves.
  • address-feedback is opt-in, not counted at the seam. publish-feedback and apply-dependency-verdict import it from a directory copy (collect-base-activity), which cannot reach .github/actions. So countedFetch() loads the recorder at run time, and only the three base-run mains call it. The copy-run scripts stay uncounted. They also run on the lane token in a job with no bucket.
  • No consolidation of the five factories. Their retry rules differ: safeToResend, Connection: close, attempt timeouts and caps. Merging them would change request behavior.
  • Notice. api-calls now ends in graphql=K, the GraphQL requests among N, which spend GraphQL's own pool. Comparing N with the core header's used therefore needs K subtracted. A reading's resource gives K, and the gh wrapper writes request graphql for a /graphql request. The first field is unchanged.
  • Expected shift on the hub. In pr-merge's merge, enable-auto-merge and clear-escalations steps, github-token rises and lanes-app falls with no new traffic. Read it as re-bucketing, not a regression.

Hold paths: rate-guard.mjs, request-budget.mjs, address-feedback.mjs, fix-ci-context.mjs, feedback-dispatch.mjs, pr-merge.yml and pr-require-checks.yml are on the hold list, so this needs the owner's merge approval. No secret, token or permission changes. The new env lines only name counting buckets.

Verification

  • New log-api-budget/count-requests.test.mjs (11 tests). One reading per request through createGitHub, address-feedback (each attempt, a resent 500 included), dispatch, and feedback-dispatch, raw and handed budgetedFetch (once, not twice). Override bucketing: GH_TOKEN, then GITHUB_TOKEN as own, and no override means no move. The three mains count one reading per stubbed request.
  • resolve-pr.test.mjs: one reading per API request, the log's 302 among them, and none for the storage read.
  • log-api-budget.test.mjs: pr view with one GraphQL and one REST request gives github-token=2 graphql=1, and a header-read gh api graphql gives graphql=1.
  • pr-merge/workflow.test.mjs: a step that holds both tokens and builds a client must name lanes-app and the github-token override.
  • Mutation check: removing each hook fails its test. That covers the dispatch, feedback-dispatch and address-feedback wraps, each main's countedFetch, the budgetedFetch mark, the guard's recording, idempotence, the override rule, the own-token rule, resolve-pr's wrap (and counting the storage read), both graphql markers, and Re-fire's override env.
  • Every .github node suite: 4,584 tests. After installing resolve-config's locked deps, the 21 that had failed for a missing ajv/yaml pass. scripts/ci-gh-read.test.sh PASS=40, scripts/ci-rerun-ci-status.test.sh PASS=157. shellcheck on api-budget.bash and actionlint on the three workflows are clean.

Not met by this PR alone: the ~10% criterion. The uncounted volume in that hour sits mostly elsewhere: 30 runs of pr-review-hosted and pr-review-security-hosted posted no notice, and the jobs newly counted here ran a handful of times.

Owner merge decision

Q14, 2026-10-29: merge at e39b4f91ce6184682f3cf4f8f4cda49385b46ef6 now, and fix the security review's one P5 finding in a follow-up PR (option A). The owner's answer, verbatim: "yes". That finding: refire-refused's API_BUDGET_TOKEN: lanes-app also moves the job-token client's margin entry. The follow-up pins that client to github-token and corrects the changelog's "No request changes" line. The independent security review found no HIGH or CRITICAL.

Related

🤖 Generated with Claude Code

kyle-sexton and others added 3 commits October 28, 2026 20:29
The rate guard's reading recorder (rate-guard.mjs) becomes countRequests,
a count-only fetch wrapper the guard and every client factory send their
attempts through. A fetch that already counts is marked and not wrapped
twice. A request whose token is not the step's own GH_TOKEN (else
GITHUB_TOKEN) counts under API_BUDGET_OVERRIDE_TOKEN, as babysit_gh.py
does, so the job token's calls in a lanes App step are no longer counted
as lanes-app.

Newly counted: the address-feedback client in its select, fix-ci-context
and review-dependency mains (loaded at run time, so the directory-copy
script activities still load), the dispatch and feedback-dispatch
factories' own fetch, pr-verify's job-log read, and three Node steps that
named no bucket. The api-calls notice adds graphql=K.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…quests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@gitguardian

gitguardian Bot commented Oct 29, 2026

Copy link
Copy Markdown

⚠️ GitGuardian has uncovered 1 secret following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secret in your pull request
GitGuardian id GitGuardian status Secret Commit Filename
39229828 Triggered Generic CLI Secret e39b4f9 plugins/developer-experience/evals/sources/harness/score-deterministic.py View secret
🛠 Guidelines to remediate hardcoded secrets
  1. Understand the implications of revoking this secret by investigating where it is used in your code.
  2. Replace and store your secret safely. Learn here the best practices.
  3. Revoke and rotate this secret.
  4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider


🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

@kyle-sexton
kyle-sexton marked this pull request as ready for review October 29, 2026 00:57
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-29T00:59:40.339907Z e39b4f9 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

Copy link
Copy Markdown
Contributor

claude-security-review reviewed e39b4f91ce6184682f3cf4f8f4cda49385b46ef6: 0 finding(s), 0 as inline comments.

No security findings in PR #10811 at head e39b4f9.

critical-findings: 0

I read the whole diff (19 files) and the PR context. I checked the trusted-path files and what they gate: rate-guard.mjs, request-budget.mjs, address-feedback.mjs, fix-ci-context.mjs, feedback-dispatch.mjs, pr-merge.yml and pr-require-checks.yml. Nothing in the diff widens a trust control.

  • Trusted-actor check, lane token scope and kill switch: none are touched. The pr-merge.yml hunk adds two env lines to the Re-fire step: API_BUDGET_TOKEN: lanes-app and API_BUDGET_OVERRIDE_TOKEN: github-token. These only name counting buckets. The step's GH_TOKEN and ACTIONS_TOKEN sources, permissions: and the KILL_SWITCH condition are unchanged.
  • Merge holds: merge-holds.mjs, enable-auto-merge.mjs, clear-escalations.mjs and superseded.mjs are not in the diff. The other pr-require-checks.yml hunks are a graphql=K awk change that repeats api-budget.bash verbatim, plus one new API_BUDGET_TOKEN: github-token env line. No hold step is altered.
  • Rate guard: guard() still runs check(url, method) before each request and observe(...) after each answer, in the same order, so the guard still fails closed. The only change is that recording moves into countRequests. A fetch already marked countsRequests is returned as is, so the pinned token is skipped for it. That affects which bucket is counted, not what is allowed.
  • Token handling:
    • budgetToken compares the request's bearer to GH_TOKEN/GITHUB_TOKEN only to pick a bucket name. Neither token is logged, written or sent anywhere new. The bucket names come from workflow env, not from untrusted input.
    • countRequests never changes the URL, headers, body or redirect mode. resolve-pr.mjs still sends the same Authorization header with redirect: "manual", and the storage redirect is still read without the token.
  • Runtime import in address-feedback.mjs: countedFetch imports a fixed relative path, ../../actions/log-api-budget/rate-guard.mjs. It is called only from the base-checkout mains: address-feedback select, fix-ci-context and review-dependency. The copy-run scripts publish-feedback and apply-dependency-verdict never call it. No PR-controlled path or specifier reaches the import.
  • Manifests: no plugin.json or marketplace.json changes, so no manifest <field> lines apply.
  • Instruction surfaces: no CLAUDE.md, skill, agent or hook content is deleted or narrowed.
  • Test changes: the relaxed workflow.test.mjs assertion replaces the old "skip lane-token steps" exclusion with a stricter rule. A step that holds both tokens and builds a client must name lanes-app and the github-token override. This tightens coverage.
  • Spend: the PR adds only local accounting. No paid plan, billing or billed resource is involved, so there is no cost hold.

@github-actions

Copy link
Copy Markdown
Contributor

claude-review reviewed e39b4f91ce6184682f3cf4f8f4cda49385b46ef6: 0 finding(s), 0 as inline comments.

Code review: no findings

I read the full diff (.claude-lane/pr.diff) and the PR context. I checked the changes against the scripts they touch. I did not run the tests. The test results in the PR description are author-claimed and unverified by me.

What I checked

  • rate-guard.mjs: countRequests, countsRequests and budgetToken look consistent.
    • The guard sends through countRequests, so each answer records one reading.
    • The COUNTED mark keeps a fetch handed in from budgetedFetch or from a guarded client from being counted twice.
    • The bucket rule is pinned token, then API_BUDGET_OVERRIDE_TOKEN when the bearer is not GH_TOKEN or GITHUB_TOKEN, then API_BUDGET_TOKEN. With no override set, nothing moves.
  • address-feedback.mjs countedFetch: the dynamic import path ../../actions/log-api-budget/rate-guard.mjs resolves from .github/scripts/pr-unblock. The copy-run scripts that import address-feedback.mjs from a directory copy stay unaffected, because only the three base-run mains call it.
  • dispatch.mjs and feedback-dispatch.mjs: countRequests wraps the raw fetch inside each client's own retry layer. A resent 5xx counts per attempt, and a budgetedFetch handed in is not re-wrapped.
  • api-budget.bash and the pr-require-checks.yml ci-status copy: the awk edit is identical in both. A request graphql line has graphql in $2, and a header reading has it in $6. request lines still count toward N. Both copies match on the /graphql URL pattern.
  • Bucket env lines:
    • Re-fire in pr-merge.yml: its GH_TOKEN is the lane token and its ACTIONS_TOKEN is the job token, so lanes-app plus the github-token override fits refire-refused.mjs.
    • The carry-run-cache read (pr-require-checks.yml) and the check-closer step (work-unblock.yml) build their clients on ACTIONS_TOKEN alone, so github-token is right for both.
    • The other API_BUDGET_OVERRIDE_TOKEN lines the new workflow.test.mjs rule needs are already present at head (pr-merge.yml, lines 1313, 1369, 1771 and 2145).
  • resolve-pr.mjs: only the API request is counted. The redirected storage read is not.

Manifests: the diff changes no plugin.json or marketplace.json. The changelog fragment is bump: patch, which fits a counting fix with no request changes.

Closing issue: none. The PR says "No related issue", so there are no acceptance criteria to check. The PR body says the "~10%" criterion from #10765 is not met by this PR alone. That is a stated scope limit, not a defect.

Embedded instructions in PR text: none found.

Spend: none. This change adds no paid plan, billing setting or billed resource. The PR names itself as touching hold-list paths and needing the owner's merge approval, and that is a merge-gate matter, not a cost hold.

@kyle-sexton
kyle-sexton added this pull request to the merge queue Oct 29, 2026
Merged via the queue into main with commit 0219230 Oct 29, 2026
102 of 103 checks passed
@kyle-sexton
kyle-sexton deleted the ci/count-every-node-github-request branch October 29, 2026 02:07
@github-actions

Copy link
Copy Markdown
Contributor

Post-merge mutation testing

Killed 1, survived 0, timeout 0, no coverage 0, invalid 0, not run 6.
Covered-code score 100%; mutation score 100%.

File Killed Survived Timeout No coverage Invalid Not run
.github/actions/log-api-budget/api-budget.bash 0 0 0 0 0 2
.github/actions/log-api-budget/rate-guard.mjs 0 0 0 0 0 3
.github/scripts/pr-unblock/dispatch.mjs 1 0 0 0 0 0
.github/scripts/pr-unblock/request-budget.mjs 0 0 0 0 0 1

The run's summary has the full report.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant