[kyle-sexton]

No related issue: review re-runs after the 2026-10-25 credential outages stayed rerun-not-allowed

Summary

During today's two Claude credential outages (20:41-20:47Z and 22:44-23:09Z), review runs that failed with failure-class=auth were re-run once the secret was fixed. The review-evidence binder accepted a second attempt only when the first failed on a rate limit, so every such attempt 2 was refused as rerun-not-allowed. One PR was escalated to needs-human (evidence-unbound ... rerun-not-allowed), and another holds only attempt-2 reviews that never bind.

Fix

• babysit_review_evidence.py adds AUTH_FAILURE_RE and rerun_binds(lane, job). Attempt 2 binds when attempt 1 failed on a rate limit, or when a failure-level annotation opens with the lane's own publish-review line <lane>: failure-class=auth: (ci-workflows publish.cjs fail). The match is anchored and lane-checked the way the publish rate-limit line is, so class=auth-x, a class quoted in detail text, another lane's line, or the warning-level <lane> key: class=auth cannot bind. _bound_record now uses rerun_binds.
• A spent key needs no entry: publish.cjs judge() reports a key credit-stop as failure-class=rate-limit, which already binds, and credit-stop is not in its CLASSES.
• rate_limited and rerun_binds share one failure-annotation scan.
• The merge gate's automatic rerun still uses rate_limited, so the gate never reruns an auth failure itself: a rerun before the secret is fixed would spend the one allowed attempt.
• reference/safety.md states the new binding rule; a changelog fragment records the fix.

Verification

• Fail-first: test_second_attempt_after_a_credential_failure failed before the fix and passes after it. The three subtests of test_an_auth_lookalike_does_not_bind (auth-x, a quoted class, another lane) failed against the first, unanchored version and pass now.
• test_a_warning_level_auth_does_not_bind (a warning-level auth line plus a real failure-class=timeout failure stays rerun-not-allowed) fails with the annotation_level == "failure" guard removed and passes with it.
• test_a_credential_failure_is_not_a_rate_limit (auth is not a rate limit, so the gate does not rerun it) passes before and after.
• babysit-prs Python suite on the merged head: 2974 tests OK; the pinned run-ruff.sh check passes.
• Left out: nothing.

Related

• Affected by the 20:41-20:47Z outage: fix(app-control): find plugin control skills at manifest-declared paths #10092, fix(harness-ops): skip docs-sourced markers in the native-drift recheck #10090 (needs-human: evidence-unbound, rerun-not-allowed), fix(harness-ops): follow destructuring defaults, Function.prototype.constructor and commented importers in the parser #10064
• Affected by the 22:44-23:09Z outage: fix(harness-ops): bound the OTEL hot tier's total size, not only each file #10137, feat(planning): give acceptance criteria ids and trace them to plan phases and decompose slices #10136, feat(pr-pipeline): stop holding on trusted paths and lane-clearable refusals #9826 (only attempt-2 reviews at 98df63c)

🤖 Generated with Claude Code

[kyle-sexton]

Held for the owner: this widens which re-run review attempts count as merge evidence (auth and credit-stop failures join rate limits), so it waits for the owner's explicit decision (Q-REBIND in the poteto session) before merging.

[kyle-sexton]

Decision recorded. The owner answered Q-REBIND in the poteto session (2026-10-26, "go with whatever the facts/data/metrics + /discovery:research says and ensure we're aligned with SOLID, DRY/SSOT principles"): option A. A review lane's second attempt binds after the first failed on a rejected credential (failure-class=auth) or a spent key (credit-stop), as it already does after a usage limit. The one-rerun-per-head cap stays, and the merge gate still never starts those reruns. Basis: attempt 1 produced no verdict, the same rationale as for rate limits. Context: ci-workflows v0.65.13 (#10151) adds the credential fallback chain, so this covers only the residual cases (all credentials gone, 403, the last leg).

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-26T02:38:43.091357Z	53195ce	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[github-actions]

claude-security-review reviewed 624c2922abde542ec1978bfed1fb1cd1e407070b: 0 finding(s), 0 as inline comments.

Incremental security review of the changes since 53195ce (changelog fragment, safety.md, babysit_review_evidence.py, and its tests): no security issues found.

• AUTH_FAILURE_RE is applied with .match, so it is anchored at the start of the message. It also requires the lane name to equal the lane under review, so it is narrower than the old unanchored INFRA_CLASS_RE search. A class quoted inside detail text, class=auth-x, another lane's line, and warning-level annotations cannot bind a second attempt. The new tests cover each of these cases.
• rate_limited keeps its semantics and now shares the _failures and _lane_line helpers with rerun_binds. The merge gate's automatic rerun still uses rate_limited, so the gate never starts an auth rerun by itself.
• Dropping credit-stop from the binding set narrows it. The PR says publish.cjs already reports a spent key as rate-limit, which still binds. I could not check that claim here because ci-workflows is not in this tree.
• Forged annotation text: matching is on annotation message text. An attacker who could write a failure-level annotation on the lane's job could already forge the existing class=rate-limit and publish-rate-limit lines. This diff adds no new reach, and the result is only that a second attempt binds.
• The changelog fragment and safety.md edits are prose and agree with the code.
• No plugin manifest fields were touched.
• No trusted-path files changed.

critical-findings: 0

[github-actions]

claude-review reviewed 624c2922abde542ec1978bfed1fb1cd1e407070b: 0 finding(s), 0 as inline comments.

Incremental review of the changes since 53195ce: no findings.

I read the incremental diff and checked it against the head file, babysit_review_evidence.py:98-120. I did not run the tests; the PR's test results are the author's claims and I have not verified them.

• Anchoring: AUTH_FAILURE_RE is matched with .match and a lane check, the same way PUBLISH_RATE_LIMIT_RE is. A warning-level <lane> key: class=auth line can't match: after the lane name the regex needs : failure-class=auth: , and that line has key: there. The auth-x lookalike, a class quoted in detail text and another lane's line are all rejected. The annotation_level == "failure" filter in _failures also excludes warning-level lines.
• Spent key: the diff drops credit-stop from the binding set. It now relies on publish.cjs reporting a spent key as rate-limit, which already binds. That is the author's claim about code in ci-workflows. I couldn't read that repo from here, so I haven't verified it.
• Merge gate: the gate's automatic rerun still goes through rate_limited, which does not include auth. The new test test_a_credential_failure_is_not_a_rate_limit pins that.
• Docs: the changelog fragment and the safety.md paragraph match the code. Both now say a spent key reports as rate-limit.
• Tests: they cover the positive case, the warning-level case with a real timeout failure, and three lookalikes.

No manifest files changed and there is no closing issue. The PR body says "No related issue", so no acceptance criteria apply. No trusted-path files were touched and nothing needs a hold.