[kyle-sexton]

Closes #10486

Summary

The pr-verify verdict reader (babysit_verify_evidence.py) paged every repository artifact on each cold read: 69 pages on 2026-10-27, on the job's GITHUB_TOKEN. pr-merge's merge job, enable-auto-merge and ci-status's merge-queue step each list cold, at about 5 to 12 listings an hour. That is 380 to 910 of the token's 1,000 requests an hour, and the count grows with the artifacts.

The reader now builds each listing on a trusted earlier one. It reads only the pages since that listing, and keeps the result only when it holds GitHub's live total_count exactly. Otherwise it lists every page.

It also closes #10486, a HIGH gap in today's code: a whole listing can skip a live artifact at a page boundary when an earlier one expires while it pages, and on 2026-10-27 two of three whole listings skipped live FAILs, so the gate could allow past a FAIL. Now no judgment uses a listing, whole or incremental, whose unique ids are not exactly the live total_count.

Fix

• Carry. The merge job of a scheduled or hand-started pr-merge run uploads its last saved listing as pr-merge-verify-listing (one day). .github/scripts/pr-merge/listing-carry.mjs fetches it the way the head-ref map and the run cache are fetched (fix(pr-merge): carry the run cache through a trusted artifact, not an Actions cache #10292, fix(pr-merge): carry the verifier reads' run cache through its own trusted artifact #10327): rotation.mjs newestTrustedRun plus onMain, never actions/cache. It writes the file to BABYSIT_VERIFY_LISTING_CARRY in the merge job, enable-auto-merge and ci-status.

• Increment (Reader._incremental). The base is the newest whole listing begun within LISTING_CARRY_MAX_AGE (3 h): this Reader's own (so refresh() before an allow is incremental too), the run's saved one, or the carry. The listing comes newest id first, but ids are not ordered by created_at. Pages are read down past the base's highest id until an artifact created LISTING_OVERLAP (2 h) before the base was read. Those pages decide every id down to the lowest one they hold, so an overwrite's deleted copy drops out. Duplicates are merged by id.

• Prune. A base entry below that point is dropped when it is marked expired or its expires_at has passed by the first page's Date header. babysit_gh.gh_api_with_headers returns the headers.

• Live completeness guard. The result must hold exactly the first page's total_count. It lists every page as today on any of these:

  • another count
  • no base, a base older than 3 h, or one of another shape
  • a page or entry of another shape, or no Date
  • any failed read

  A forged carry that drops a FAIL must also add a fabricated entry to match the live count, and forging it needs code running in main's pr-merge run. This rule now sits in the module docstring ("Carried reads") as the one exception to "a carried read only counts a FAIL". The _run_state comment points there.

• Whole listings held to the live count (Reader._complete_listing, fix(source-control): the verdict reader's full artifact listing skips live artifacts at page boundaries, FAILs included #10486). A whole listing is deduplicated by id and added to the whole listings read before it, then checked by the same top-up from page 1 (fresh pages decide the newest ids, expires_at is pruned by the page's Date, and the total must equal total_count). A mismatch reads the whole listing again and unions it. After LISTING_READS = 3 whole listings that never hold the count, the candidate refuses verify:unreadable; it never allows. Basis for 3: of three whole listings on 2026-10-27, two skipped live artifacts (1 and 3), and every skipped id was in every other whole listing, so a second read restores a skip and a third is margin (judgment on that data). The reads and top-ups share the one LISTING_TIMEOUT_SECONDS. A listing entry with no id now refuses too, since it cannot be counted. The refresh before an allow (babysit_merge.py refresh()) reads through the same path.

• Consumers. The saved listing keeps its shape (fetchedAt, artifacts), so these work unchanged: babysit_merge.py refresh, babysit_merge_skip.py, ci-status's PASS-name loop, head-ref-map.mjs and lane-records.mjs pruning. Page calls show in the existing babysit-gh-calls lines as GET repos/{}/{}/actions/artifacts.

• Gate-file lists. listing-carry.mjs is added to the gate-file lists: babysit_risk.py, merge-holds.mjs, ci-status hold_patterns, both hosted review lanes, and ci-status's sparse checkout.

Bases.

• LISTING_OVERLAP = 2 h. Across 6,835 artifacts listed on 2026-10-27 (created 10-16 to 10-27), the largest lead of a lower id's created_at over a higher id's was 58m49s. Each day's largest lead was 34 to 59 min. 2 h is twice the largest (judgment on that data).
• LISTING_CARRY_MAX_AGE = 3 h:
  • Scheduled pr-merge runs come 4 times an hour (pr-merge.yml cron).
  • newestTrustedRun asks at most 10 trusted runs (MAX_RUNS), about 2.5 h of runs.
  • At the most uploads seen in one hour that day (255), 3 h plus the overlap is at most about 13 pages, against 69 for a full listing (judgment on that data).
• Expiry as GitHub counts it. I read the first page (total_count and Date) 17 times, about 49 s apart, from 15:36 to 15:50 UTC. Every time, total_count equalled the last reading, plus the new uploads on page 1, less the artifacts whose expires_at had passed by Date. Five artifacts expiring at 15:40:47-55 left the count by the next reading. The per-id GET of one artifact answered 404 a second before its expires_at, while the count still held it until the next reading.

Verification

Live probe (read-only, my token). The real reader ran an incremental read on a listing I took at 15:32, chained twice, and each run was followed by a full listing:

Run (UTC)	Incremental	Full listing right after	Set comparison
15:49	2 calls, 1.3 s, 6,856 ids = live count	69 calls, 51.7 s, 6,855 unique of 6,860 rows	Incremental equals full plus 1 live FAIL the full listing skipped at index 6100
15:53	2 calls, 1.3 s, 6,862 ids = live count	69 calls, 50.3 s, 6,862	Equal
16:02	2 calls, 1.6 s, 6,866 ids = live count	69 calls, 58.1 s, 6,863	Incremental equals full plus 3 live artifacts the full listing skipped at indexes 3400, 3500 and 4600 (one a FAIL)

The four skipped ids sit at page boundaries, and a GET after each probe confirmed every one is live. Each skipped id was present in every other whole listing. This is #10486, now closed here: a whole listing is kept only when it holds the live count.

Tests. TheListingBuildsOnACarriedOne (12 tests) and listing-carry.test.mjs (4 tests). Each fallback has its own test, and each was proven to fail when the matching mechanism is removed:

Mutation	Test that fails
no incremental read	every new-behavior test
no age bound	test_a_carry_past_its_age_bound_lists_every_page (3 h kept, 3 h 1 s lists every page)
no entry-shape check	test_a_carry_that_does_not_parse_lists_every_page, test_a_page_with_no_date_or_an_odd_entry_lists_every_page
no overlap	test_the_overlap_reads_a_lower_id_created_after_the_base, test_a_fail_uploaded_since_is_read_from_the_first_page_alone
no count guard	test_a_count_other_than_githubs_lists_every_page (a forged carry drops an old FAIL), test_a_live_count_no_listing_can_hold_refuses
expiry by the runner's clock, not Date	test_an_expired_carried_entry_is_dropped_by_githubs_date
no in-range drop	test_a_carried_entry_the_pages_no_longer_hold_is_dropped

AWholeListingMustHoldTheLiveCount (4 tests, each failing with the check removed): a whole listing that skips a FAIL is restored by the next read and the FAIL refuses (verify:earlier-fail, where the first listing alone would allow); one that never holds the live count refuses verify:unreadable after 3 reads and saves nothing; a duplicate at a page boundary is counted once; the refresh before an allow is held to the same rule. A listing entry with no id refuses (test_a_listing_with_an_artifact_of_no_id_refuses).

Two fallbacks have their own tests: a failed page read (502 and 404), in test_a_failed_page_read_lists_every_page, and an untrusted or off-main run, in listing-carry.test.mjs. A module-level patch fails any other test that reads a page.

Suites.

• python3 -m unittest (babysit-prs scripts): 3,214 tests with real gh blocked on PATH; test_babysit_verify_evidence (305) and test_verify_rules_sync OK.
• node --test in .github/scripts/pr-merge: 530 pass. In pr-verify: 158 pass.
• scripts/ci-lanes-pr-hold.test.sh: 216 pass.
• scripts/ci-review-lanes.test.sh: 98 pass.
• actionlint on the changed workflows: clean.
• scripts/check-affected.sh: 14 passed.

Related

• fix(source-control): give the verify reader's artifact listing its own timeout #10465: the listing timeout this replaces as the main cost lever
• fix(pr-merge): carry the run cache through a trusted artifact, not an Actions cache #10292, fix(pr-merge): carry the verifier reads' run cache through its own trusted artifact #10327: the trusted-carry path this reuses
• fix(source-control): the verdict reader's full artifact listing skips live artifacts at page boundaries, FAILs included #10486: the full listing skips live artifacts at page boundaries (closed here)

🤖 Generated with Claude Code

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-27T18:53:51.936556Z	4e8695c	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[github-actions]

claude-security-review reviewed dfa762ccb10d2e5e59717f2c79af1d828e07788f: 0 finding(s), 0 as inline comments.

No security issues found in this change. Review was read-only: I had no shell and ran no tests.

critical-findings: 0

Trusted paths read. I read the whole change of listing-carry.mjs, merge-holds.mjs, babysit_risk.py, babysit_verify_evidence.py, babysit_gh.py, pr-merge.yml and pr-require-checks.yml, and the gates they feed. There are no plugin.json or marketplace.json changes, so no manifest <field> lines apply.

Carried listing as a trust boundary. A forged or stale carried listing could hide a FAIL from the verdict reader, but nothing untrusted reaches it.

• listing-carry.mjs fetches it with newestTrustedRun and onMain, the same path already used for the head-ref map and run cache. It never goes through actions/cache, and a run whose head commit is off main is rejected.
• The upload step in pr-merge.yml runs only for schedule or workflow_dispatch. The file it uploads is the Reader's own atomic save.
• listing-carry.mjs removes any pre-existing file at the carry path before writing, and the workflow points that path at runner.temp.
• Forging a carry would need code running in main's pr-merge run, which already holds more authority than the listing.

Live-count guard fails closed.

• _incremental returns a result only when len(listing) == total_count from the live first page (babysit_verify_evidence.py line 1026 of the head file).
• A missing or malformed base or entry, a missing Date, a failed read, a timeout or a count mismatch all return None, which falls back to a full listing.
• A stale base entry can only inflate the count, which also falls back. I found no compensating-error path an outside party can trigger.
• _complete_listing refuses verify:unreadable after 3 whole listings that never hold the count. This tightens the existing path: a whole listing that skipped a FAIL at a page boundary (fix(source-control): the verdict reader's full artifact listing skips live artifacts at page boundaries, FAILs included #10486) can no longer allow.

Gate-file registration. listing-carry.mjs is added to merge-holds.mjs and babysit_risk.py as gate, to ci-status hold_patterns, to both hosted review lanes' lists and to the sparse checkout. That is consistent, so changes to it route to a person.

Token and permissions. I saw no token scope or permissions: widening. The new steps use github.token with the existing actions: read and add no new secrets.

Spend, not a hold. The cold-read headroom check in pr-require-checks.yml rises from 900 to 1,520 GITHUB_TOKEN requests per queue entry. That only makes the check stricter and draws on API rate budget, not billing.

[github-actions]

claude-review: 1 inline finding(s) on 4e8695c887b3ba05c8287cb1dacdf8195c5d26fd; the summary is in this lane's pull request comment.

claude-lane: claude-review

[github-actions]

claude-review reviewed dfa762ccb10d2e5e59717f2c79af1d828e07788f: 0 finding(s), 0 as inline comments.

No findings on this head (dfa762c). I read the full diff and the surrounding code it depends on. I did not run any tests: this lane has no shell, so every test count and live-probe result in the PR body is author-claimed and unverified by me.

Scope

• Read the whole diff, covering babysit_verify_evidence.py (_incremental, _complete_listing, _incremental_base, _listed_entry), babysit_gh.gh_api_with_headers, listing-carry.mjs, the pr-merge.yml and pr-require-checks.yml wiring, and the gate-file lists.
• Read the helpers the new code relies on: _send/_includes_headers, run_command, _when, _recent, json_object, newestTrustedRun/trustedRun/ghDownload/sharedReads.
• Did not read the roughly 700-line test_babysit_verify_evidence.py addition line by line.

Closing issue #10486 has no separate acceptance-criteria list. Its Scope paragraph is met in code, as far as reading shows:

• A whole listing is deduplicated by id, added to earlier whole listings, and checked by a top-up from page 1 (_complete_listing).
• After LISTING_READS (3) reads that never hold the live count, it raises _Unreadable, so the candidate refuses verify:unreadable.
• refresh() reads through _current_listing and so uses the same path.
• A listing entry with no id raises _Unreadable.
• The PR body names a test for each of these.

Trusted-path scrutiny (merge-holds.mjs, listing-carry.mjs, babysit_risk.py, babysit_gh.py, babysit_verify_evidence.py, pr-merge.yml, pr-require-checks.yml)

• Carry source.
  • listing-carry.mjs accepts only runs that trustedRun passes: schedule or workflow_dispatch, on main, workflow pr-merge.yml.
  • It also requires the run's head commit to be on main (onMain), and the artifact must be unexpired.
  • It never uses actions/cache. It removes any file already at the output path first, and every failure writes nothing and warns.
• Reader checks. The reader re-checks the carry's shape and age (_listed_entry, LISTING_CARRY_MAX_AGE). It keeps a result only when its unique ids equal the first page's live total_count. Every anomaly I traced falls back to a full listing: no Date header, a page of another shape, a failed read, a deadline overrun, or an empty page. None of them reaches an allow.
• Remaining forgery risk. A forged carry has to add a fabricated entry to match the live count, and that needs code running in main's pr-merge run. The PR states this in the module docstring "Carried reads", the one exception to "a carried read only counts a FAIL".
• Rate-limit exceptions. RateLimitExhausted is a BaseException, so the new except (RuntimeError, ValueError, OSError) in _incremental does not swallow it. The guard still stops the run, as it does today.
• Workflows. The step ids gate and candidates used in the new if: conditions exist in the merge job. The new upload step uses the same pinned actions/upload-artifact SHA as the job's other uploads. The ci-status sparse-checkout list and all six gate-file lists carry listing-carry.mjs.
• Cold-read headroom. The need figure moves from 900 to 1,520 per queue entry and is pinned in a test (793 + 13 + 3 x (3 x 77 + 7)). The README, workflow comments and shell test agree on 1,520 and 7,700.

Spend. This uses only the job's GITHUB_TOKEN request budget and a 1-day, roughly megabyte-sized artifact per scheduled pr-merge run. It enables no paid plan, billing setting or billed resource, so there is no cost hold.

Left for a person. Nothing blocks this review. The size and age constants (LISTING_OVERLAP 2 h, LISTING_CARRY_MAX_AGE 3 h, LISTING_READS 3) rest on the author's own 2026-10-27 measurements. I cannot re-measure them here.

Manifests. The diff changes no plugin.json or marketplace.json. Both changelog fragments are bump: patch, which fits a behaviour-preserving perf change plus a fail-closed fix.

[melodic-automation-lanes]

The pr-unblock lane stopped on this pull request and needs a person: the merge stages a change under .github/workflows/, which the lane does not push.

• Reason: workflow-staged
• Resume key: b08200f7584c86b46fa258b4a6b19dcae4b821c5 (the head the lane stopped on)

To resume, deal with the reason above, remove the needs-human label (and do-not-merge, if set), then dispatch the pr-unblock workflow with pr-number=10488 and activity=resolve-conflicts:

gh workflow run pr-unblock.yml -R melodic-software/claude-code-plugins -f pr-number=10488 -f activity=resolve-conflicts

[melodic-automation-lanes]

The pr-unblock lane escalated this pull request as workflow-staged, and that no longer holds at 03b57cb5cd80e3e9a80b62a6754a7242ac057ddc: main changes nothing under .github/workflows/ since its merge base with 03b57cb. pr-merge removed needs-human, so the lanes run on it again. To keep a person on it, add needs-human back: a label a person adds is never removed.

[melodic-automation-lanes]

The pr-unblock lane stopped on this pull request and needs a person: a conflicted path is a workflow, submodule config or a settings file, which the lane does not resolve.

• Reason: protected-path-conflict
• Resume key: 03b57cb5cd80e3e9a80b62a6754a7242ac057ddc (the head the lane stopped on)

To resume, deal with the reason above, remove the needs-human label (and do-not-merge, if set), then dispatch the pr-unblock workflow with pr-number=10488 and activity=resolve-conflicts:

gh workflow run pr-unblock.yml -R melodic-software/claude-code-plugins -f pr-number=10488 -f activity=resolve-conflicts

[github-actions]

claude-review: 1 inline finding(s) on 7035e06ccc74474cd8871336ba3f2da59c9f0221; the summary is in this lane's pull request comment.

claude-lane: claude-review

[melodic-automation-lanes]

The pr-unblock lane escalated this pull request as protected-path-conflict, and that no longer holds at 7035e06ccc74474cd8871336ba3f2da59c9f0221: GitHub reports no conflict with main. pr-merge removed needs-human, so the lanes run on it again. To keep a person on it, add needs-human back: a label a person adds is never removed.