Skip to content

perf(source-control): batch the demotion record reads - #10668

Merged
kyle-sexton merged 2 commits into
mainfrom
perf/batch-demotion-reads
Oct 28, 2026
Merged

kyle-sexton merged 2 commits into
mainfrom
perf/batch-demotion-reads

Conversation

@kyle-sexton

Copy link
Copy Markdown
Member

No related issue: post-merge-dispatch GITHUB_TOKEN cut, measured on 4 schedule runs

Summary

pr-merge's post-merge-dispatch job made 83, 86, 23 and 89 GITHUB_TOKEN calls in 4 schedule runs (2026-10-28, 12:25-12:57Z). In 3 of them, the "Check earlier lane merges" step could not reuse the last report, so it ran babysit_postmerge.py --lane-merges-only in full. Each of those full runs made 66-68 calls, 47 of them GraphQL.

A trace of every gh call in one full run (read-only, against this repository on 2026-10-28) shows where the 47 GraphQL calls go:

Read GraphQL calls
read_demotion, inside default_floor: one PolicyDemotionRecord read per candidate issue 39
read_demotion: issues-by-author listings 2
merged pull requests walk 3
pin batches 3

The total, 39+2+3+3 = 47, matches "default graphql": 47 on the babysit-gh-calls line of each of the 3 full-run job logs. The demotion record read is the cost. The merged walk and the pins are 6 calls.

Fix

  • read_demotion reads its candidate records 20 to one aliased GraphQL query (_demotion_records, DEMOTION_BATCH). This follows the pattern babysit_postmerge.read_pins uses for pins.
  • Records are still processed in sorted order, and the read still stops at the first record that holds. Batches are fetched one chunk at a time, so a hold in an early chunk skips the later chunks' reads.
  • A record whose batch query fails, or that the batch answer leaves out (missing, null, or with another number), is read on its own with the unchanged DEMOTION_RECORD query. Every outcome is therefore the per-issue read's outcome, including its error text.
  • DEMOTION_RECORD and the batch query share one field list (DEMOTION_FIELDS).
  • read_demotion also serves the merge gate (assess), settle_shared's demotion-changed path and lane_room. They get the same reduction, with the same answers.
  • GraphQL point cost stays about the same, since the same issues are read. What drops is the number of calls, which the token budget counts.

Before and after:

Before After
read_demotion, 39 candidate issues (measured live) 42 calls 5 calls (2 listings, 2 batches, 1 permission read)
babysit_postmerge.py --lane-merges-only full run 66-68 calls (CI logs) about 29-31 (computed: 42 → 5)
post-merge-dispatch job when reuse fails 83-89 calls (CI logs) about 46-52 (computed)
post-merge-dispatch job when reuse holds 23 calls (CI log) 23 (unchanged: reuse makes no demotion read)

Verification

  • New tests in test_babysit_merge_policy.py, written first. On the unchanged babysit_merge_policy.py the new test and all 6 equality subtests failed: 7 failed, 20 passed.
    • ReadDemotionBatched.test_45_records_take_two_listings_and_three_batches pins the call count. 45 records take 2 listings and 3 batch queries, where the per-issue read took 47 calls.
    • test_batched_reads_equal_per_issue_reads runs the batched read and the per-issue read (_demotion_records answering nothing) over the same 30-record fixtures and asserts equal Demotion results. It covers: all clear, a hold before an unreadable record, a truncated edit history, a batch error that falls back (it asserts the fallback reads one at a time up to the hold), an issue missing from a batch answer (one single read), and a missing issue that no single read finds either.
    • test_the_cases_reach_their_answers checks that those cases are not all one answer: #12 edit-history-incomplete and demotion-unreadable: issue #30 returned nothing.
  • demotion_fake now answers the batch query too, so test_role_label_removed_is_still_off keeps its 3-query count: 2 listings and 1 batch, where before it was 2 listings and 1 single read.
  • The whole babysit-prs suite passes: 3336 tests. scripts/run-ruff.sh check is clean, and check-changelog-fragments.sh --check-required origin/main passes.
  • Live check against this repository, read-only. The shipped read_demotion and a batched read returned equal Demotion results over 39 candidate issues and 15 records. With this branch's code, read_demotion took 5 calls and returned the same Demotion.

Related

🤖 Generated with Claude Code

kyle-sexton and others added 2 commits October 28, 2026 10:00
read_demotion read each candidate demotion issue in its own GraphQL
query: 39 of the 47 GraphQL calls of a full post-merge check. It now
reads them 20 to an aliased query, as read_pins does, and reads on its
own any record a batch fails or leaves out, so the answer is the
per-issue read's. Live: 42 calls to 5 over 39 candidates, equal result.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kyle-sexton
kyle-sexton marked this pull request as ready for review October 28, 2026 14:15
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-28T14:21:35.069633Z 2d808e5 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

Copy link
Copy Markdown
Contributor

claude-security-review reviewed 2d808e5c3aba2683f5b818acdad42ea83f91eaf1: 0 finding(s), 0 as inline comments.

No security issues found in this pull request.

critical-findings: 0

What I read: the whole diff (3 files), the PR context, and the surrounding code of _graphql, _candidate_issues and read_demotion in babysit_merge_policy.py. babysit_merge_policy.py is a trusted path: it backs the merge hold (the demotion record read behind the merge gate, post-merge floor and lane room). I read its whole change and what it gates.

Trust control check: the demotion hold is not weakened.

  • Batch errors fail closed into the per-issue read. _graphql raises PolicyReadError on any GraphQL errors entry, a non-zero exit, or missing data (babysit_merge_policy.py:476-481). _demotion_records turns that into {}, so every record in the chunk is re-read one at a time with the unchanged DEMOTION_RECORD query. A partial NOT_FOUND inside a batch (i5: null plus errors) therefore gets the same treatment, and the per-issue error text is preserved.
  • A record is accepted from the batch only if it is a JSON object and its number equals the requested one. A null, a non-object, an empty object or a mismatched number is dropped and re-read singly. If that single read also returns nothing, read_demotion still raises issue #N returned nothing, which holds the rung off (:1592-1593).
  • Everything after the record is fetched is unchanged: the marker and truncated-history check, _record_hold, ordered iteration and early return on the first hold. A bad answer from the batch cannot clear a record the per-issue path would have held.
  • Query injection: the alias text interpolates only n, which comes from _candidate_issues and is an int from the GraphQL number field. The isinstance(..., int) check rejects anything else (:1456). The owner and repo go in as variables, not interpolated. No attacker-controlled string reaches the query text.
  • Batch size is 20, so the added query complexity and response size are bounded, and the per-record userContentEdits(first:100) is unchanged.

Other surfaces: the change fragment under .changes/ and the tests have no security surface. No manifests, workflows, tokens or permissions changed.

Spend: none. The change reduces GITHUB_TOKEN API calls.

@github-actions

Copy link
Copy Markdown
Contributor

claude-review reviewed 2d808e5c3aba2683f5b818acdad42ea83f91eaf1: 0 finding(s), 0 as inline comments.

No findings. I read the full diff and the surrounding read_demotion and _graphql code in babysit_merge_policy.py. I did not run the tests; the author's 3336-test pass and live-read results are author-claimed and unverified by me.

  • Batching and fallback: read_demotion fetches a chunk of 20 candidates when i % DEMOTION_BATCH == 0. It still walks the sorted candidates in order and still returns at the first hold. A record the batch fails on, omits, nulls or answers with another number falls back to the unchanged DEMOTION_RECORD single read, so its outcome and error text are the per-issue ones. _graphql raises on any GraphQL errors, including a partial NOT_FOUND for one alias, so such a batch falls back as a whole. That costs only extra calls, not a different answer.
  • Query: the aliased query, the shared DEMOTION_FIELDS and the fragment d on Issue spread are well formed. The aliases come from integer issue numbers, so there is no injection surface.
  • Tests: they exercise the call count, equality with the per-issue read across the fallback and hold cases, and distinct answers across the cases. The demotion_fake ordering comment matches how the fake matches query names.
  • Changelog fragment: the patch bump suits a call-count change with the same answers.
  • Closing issue: none; the PR body says "No related issue".
  • Cost: no hold. This only reduces GitHub token calls and adds no spend.

@kyle-sexton
kyle-sexton added this pull request to the merge queue Oct 28, 2026
Merged via the queue into main with commit 2b1d7d1 Oct 28, 2026
77 checks passed
@kyle-sexton
kyle-sexton deleted the perf/batch-demotion-reads branch October 28, 2026 14:36
@github-actions

Copy link
Copy Markdown
Contributor

Post-merge mutation testing

Killed 0, survived 0, timeout 0, no coverage 0, invalid 0, not run 8.
Covered-code score n/a; mutation score n/a.

File Killed Survived Timeout No coverage Invalid Not run
plugins/source-control/skills/babysit-prs/scripts/babysit_merge_policy.py 0 0 0 0 0 8

The run's summary has the full report.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant