Skip to content

fix(pr-merge): route repeated-fail and too-old verdicts to the lanes, not a person - #10481

Merged
kyle-sexton merged 2 commits into
mainfrom
fix/route-repeated-fail-to-lanes
Oct 27, 2026
Merged

kyle-sexton merged 2 commits into
mainfrom
fix/route-repeated-fail-to-lanes

Conversation

@kyle-sexton

Copy link
Copy Markdown
Member

No related issue: owner rule 2026-10-27 on needs-human (intent, design-altering findings, HIGH/CRITICAL security only)

Summary

Under the owner rule of 2026-10-27, a person is pulled in only for an intent change, a finding that alters intent or the primary design, or a HIGH or CRITICAL security issue. verify:repeated-fail and verify:too-old are none of these, but pr-merge still put needs-human on them. This PR routes both to the lanes. It reuses the paths #10249 and #10198 built and adds no new route.

  • Merge gate. Both reasons still refuse the merge for good. Neither clears at a later head, so such a pull request never merges unattended. The merge queue holds them as well: merge-holds.mjs evidenceHold holds on every refusal except the three a pass decides, and it never reads needsHuman.
  • Native path (merge.lane-prs: native, lanes App pull requests). The reader now reports needsHuman: false, so verifyClears returns lane. The pull request stays unarmed with the lane comment. Once the refusal stands for merge.risk-escalation-minutes, a failing pr-merge/unarmed status goes on the head. It gets no label. This is fix(pr-merge): route non-h38 needs-human holds to bounded retries and a failing status #10249's route.
  • Gate path (Dependabot and adopted pull requests). No escalation comment and no label. The neutral pr-merge / gate check run shows the refusal, as it already does for verify:spec-changed.
  • Labels already placed. LANE_CAUSES.verdict in clear-escalations.mjs adds both reasons. On its next tick, pr-merge's clear step removes a native label for either one at its head (refusal-cleared) when the reader names only lane causes there. This is fix(pr-merge): clear a native gate's needs-human label once its cause is gone #10198's lift.
  • What stays with a person. verify:untrusted-spec does: an untrusted spec is a trust boundary. It is now the whole of NEEDS_HUMAN.

How the repeated-fail bound holds. The cap still refuses the merge, and pr-merge spends nothing on a lane refusal beyond its scheduled read. Nothing dispatches from the status: fix-ci reacts only to check runs, and no workflow runs on status events (checked in #10249). Fix rounds on verifier notes stay bounded by address-feedback's own cap: loop.review-rounds (default 2) plus FRESH_FIXER_ROUNDS 1 (address-feedback.mjs:42-49).

Fix

  • babysit_verify_evidence.py: NEEDS_HUMAN = (UNTRUSTED_SPEC,). The module docstring and the constant's comment now state the new rule.
  • clear-escalations.mjs: LANE_CAUSES.verdict adds verify:repeated-fail and verify:too-old. The comment now says why both are lane-class.
  • lane-causes-person-codes.test.mjs also reads NEEDS_HUMAN. It asserts that LANE_CAUSES.verdict shares nothing with it, that verify:untrusted-spec stays in it, and that both reasons are lane causes.
  • Restatements updated: the enable-auto-merge.mjs header and the verifyClears comment; the pr-pipeline README (refusal-cleared row, native-path label list, verifier-verdict section); pr-pipeline.schema.json max-verify-fails; the pr-pipeline.yaml comment; and babysit_merge_policy.py max_verify_fails.
  • Removed as unreachable: the repeated-fail escalation's quote of the head FAIL's summary. escalate_verify runs only on needsHuman, and the quote applied only to verify:repeated-fail. That covers babysit_merge._fail_summary, _carried_quote, _QUOTE_RE, VERIFY_SUMMARY_MAX, the summary arguments of verify_escalation_body and escalate_verify, Reader.fail_summary, _listed_id, SummaryUnreadable, Evidence.head_fail, and the report's headFail field. No consumer in .github/ read headFail.
  • Changelog fragments under .changes/source-control/ and .changes/pr-pipeline/.

Verification

Each test below fails against main's sources and passes on this branch. They were run by copying the branch's test files into a checkout of main.

  • test_babysit_verify_evidence.py: 23 failures on main.
    • test_only_an_untrusted_spec_goes_to_a_person
    • LaneRequiresAVerdict.test_the_cap_and_an_old_pull_request_hold_with_no_escalation: gate path, no comment and no label for either reason
    • the cap and too-old needs_human assertions
    • TheByIdFetchDecidesAsGhRunDownloadDid, whose docstring now names its two changed fields
  • clear-escalations.test.mjs: "enable-auto-merge's label comes off when the reader now reads a refusal the lanes clear for the same cause" now includes refactor(guardrails): resolve the commit convention from docs conventions source-control.yaml #10134's shape: a repeated-fail label lifted at its head while the reader still refuses repeated-fail, and the same for too-old.
  • lane-causes-person-codes.test.mjs: "the repeated-FAIL cap and an old pull request are lane causes the clear lifts".
  • The enable-auto-merge.test.mjs fixture flips (needsHuman: false, lane comment, no label) pass on main as well, because the fixtures set the flag directly. They keep the tests in line with the contract. The Python needs_human tests and the clear and pin tests above are the actual proof.

Suites, run locally on 2026-10-27:

  • python3 -m unittest in babysit-prs/scripts: 3174 tests OK.
  • node --test in .github/scripts/pr-merge: 526 pass, 0 fail.
  • scripts/ci-lanes-pr-hold.test.sh: PASS=215 FAIL=0.
  • scripts/check-affected.sh --tests: 13 passed, 2 skipped.
  • scripts/run-ruff.sh check: clean. format --diff on the changed Python shows the same 6 hunks on main and on this branch, so none of them come from this change.
  • scripts/check-changelog-fragments.sh --check-required origin/main: OK.
  • No workflow changed.

Security check: nothing decides a merge from needsHuman. It is read only by enable-auto-merge.mjs verifyClears, which chooses between a label and a comment, and by babysit_merge.unattended_writes, which decides whether to escalate. verifyHold and merge-holds.mjs evidenceHold hold on the refusal itself.

Related

  • #10249: the lane route reused here (lane comment, failing pr-merge/unarmed status, no label). It kept both reasons with a person as C1a and C1b. Owner rule 2026-10-27 reverses that.
  • #10198: the same-head refusal-cleared lift, which LANE_CAUSES drives.
  • #10397: a trusted approval at the head lifts a hold-path hold.

Left out, for the owner:

  • Only #10134 lifts. Its standing refusal is verify:repeated-fail. #10377, #10414 and #10439 hold on a hold path, which is a person's by design. A trusted approval at their heads lifts them (fix(pr-merge): arm a hold-path pull request a trusted person approved at its head #10397).
  • No lane closes or re-admits a pull request at the cap or past TOO_OLD_DAYS. Such a pull request stays open with the failing status. Closing it and re-admitting the issue would not help at the cap: the count follows the issue across pull requests (README, Verifier verdict), so a new pull request for the same issue is refused again as soon as a pass decides. Whether to add a close step is the owner's decision.
  • address-feedback still sends verifier-note threads to a person once past its round cap (address-feedback.mjs:84-93, feedback-round-cap). The result is needs-human on the pull request, not needs-decision on the issue. Under the same owner rule, that looks like the next candidate.
  • pr-verify has no "spec cannot be met as written" signal. Its output is {verdict, summary, notes}, so this PR adds no reason-2 (needs-decision) detection.
  • Gate-path escalations already placed for these reasons lift only on head-moved. The same-head lift applies on the native path only.

🤖 Generated with Claude Code

kyle-sexton and others added 2 commits October 27, 2026 11:36
… not a person

Owner rule 2026-10-27: needs-human only for an intent or design change or
a CRITICAL security issue, trust holds kept. NEEDS_HUMAN is now
verify:untrusted-spec alone. verify:repeated-fail and verify:too-old still
refuse the merge for good; they get the lane comment and the failing
pr-merge/unarmed status (#10249's route) and no label.

clear-escalations.mjs LANE_CAUSES.verdict adds both, so labels already
placed for them come off at their head (#10198's refusal-cleared lift).
The pin test now also checks LANE_CAUSES.verdict against NEEDS_HUMAN.

The repeated-fail escalation's summary quote is unreachable and removed:
_fail_summary, _carried_quote, VERIFY_SUMMARY_MAX, Reader.fail_summary,
SummaryUnreadable and the report's headFail.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kyle-sexton
kyle-sexton marked this pull request as ready for review October 27, 2026 16:02
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-27T16:05:24.481402Z 7d8367c Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

Copy link
Copy Markdown
Contributor

claude-review reviewed 7d8367c6136ef5a188ea74df104df1a3fef9d3cf: 0 finding(s), 0 as inline comments.

No findings. I read the full 1286-line diff and the PR context. I did not run any tests; the suite results in the PR description are author-claimed and I did not re-run them.

Checked against the change

  • NEEDS_HUMAN is now (UNTRUSTED_SPEC,) (plugins/source-control/skills/babysit-prs/scripts/babysit_verify_evidence.py:261). Evidence.needs_human is only a membership test on it (line 579), and verifyClears in .github/scripts/pr-merge/enable-auto-merge.mjs:521-535 maps needsHuman === false to lane. verify:repeated-fail and verify:too-old therefore get the lane comment and no label, as the PR describes.
  • The refusal itself is unchanged. The reader still returns verify:repeated-fail and verify:too-old, and the skip logic in babysit_merge_skip.py still lists both as skippable. refusal is not an allow answer, so the merge gate still refuses. I found no path where needsHuman decides whether to merge, which matches the PR's security note.
  • clear-escalations.mjs LANE_CAUSES.verdict now includes both codes. The new test pins LANE_CAUSES.verdict against NEEDS_HUMAN (no overlap, verify:untrusted-spec stays in it), so the lane and person lists cannot drift apart. verify:untrusted-spec stays with a person.
  • The removals (_fail_summary, _carried_quote, _QUOTE_RE, VERIFY_SUMMARY_MAX, Reader.fail_summary, _listed_id, SummaryUnreadable, Evidence.head_fail, headFail) leave no references in non-test, non-changelog code. The only headFail matches are unrelated headFailing identifiers in fix-ci-dispatch.mjs.
  • Docs and comments (README, schema, pr-pipeline.yaml, babysit_merge_policy.py) match the new routing. The changelog fragments use bump: patch, which fits a routing change with no config or schema shape change.

Trusted paths touched (clear-escalations.mjs, enable-auto-merge.mjs, babysit_verify_evidence.py, babysit_merge.py, babysit_merge_policy.py, docs/conventions/pr-pipeline.yaml): none of them widens what the gate allows. The change only removes a label and an escalation comment for two refusals that keep refusing. Lifting a needs-human label already placed for these refusals is intended and is covered by the clear-escalations.test.mjs cases. The PR lists as unaddressed that gate-path escalations already placed lift only on head-moved.

Closing issue: none (the PR states an owner rule, not an issue), so there are no acceptance criteria to check.

Cost: no billing change. The change only removes needs-human routing.

Manifests: no plugin.json or marketplace.json change, so no manifest <field> lines apply.

@github-actions

Copy link
Copy Markdown
Contributor

claude-security-review reviewed 7d8367c6136ef5a188ea74df104df1a3fef9d3cf: 0 finding(s), 0 as inline comments.

No security findings in PR #10481 (head 7d8367c).

critical-findings: 0

What changed. verify:repeated-fail and verify:too-old leave NEEDS_HUMAN, which is now verify:untrusted-spec alone. Both join LANE_CAUSES.verdict in clear-escalations.mjs. The repeated-fail escalation's quote of the head FAIL's summary is removed.

Trust-control check (merge hold weakened?). I read the changed trusted-path files and the code they gate. The change does not weaken any merge hold.

  • verifyHold in enable-auto-merge.mjs refuses on any non-null evidence.refusal. It never reads needsHuman, so both refusals still keep the pull request unarmed.
  • verifyClears reads needsHuman only to pick lane or person. That decides whether a label is written, not whether the merge is held. An answer missing the flag still falls to person, so it fails closed.
  • evidenceHold in merge-holds.mjs holds on every refusal except verify:missing, verify:spec-changed and verify:untrusted-spec. verify:repeated-fail and verify:too-old are not in that list, so the merge queue still holds them.
  • In babysit_merge.py, needsHuman only gates escalate_verify. It does not gate the merge decision (unattended_writes, near line 4039).
  • verify:untrusted-spec, the trust-boundary refusal, stays person-class. lane-causes-person-codes.test.mjs now pins LANE_CAUSES.verdict against NEEDS_HUMAN, so the two lists cannot overlap.
  • Lifting an existing needs-human label for these two reasons (refusal-cleared) removes a signal, not a hold. The reader must still name only lane causes at that head, and the merge refusal stands.

Removed quote path. The deleted code (Reader.fail_summary, _carried_quote, the headFail field) only passed model-authored text into a comment body. Removing it shrinks the untrusted-text surface, and I found no remaining consumer of headFail.

Manifests. No plugin.json or marketplace.json changed, so there are no manifest flags. No workflow files changed.

Spend note. With the label gone, a pull request at the repeated-fail cap can still get address-feedback rounds on verifier notes. These use prepaid credit and are bounded by loop.review-rounds plus the fresh-fixer round, so this is not a hold.

I did not read docs/conventions/instruction-exception-register/README.md. The diff changes no standing instruction file, only escalation routing and the docs that describe it.

@kyle-sexton
kyle-sexton added this pull request to the merge queue Oct 27, 2026
Merged via the queue into main with commit 3541d2a Oct 27, 2026
78 checks passed
@kyle-sexton
kyle-sexton deleted the fix/route-repeated-fail-to-lanes branch October 27, 2026 19:46
@github-actions

Copy link
Copy Markdown
Contributor

Post-merge mutation testing

Killed 4, survived 1, timeout 0, no coverage 0, invalid 0, not run 3.
Covered-code score 80%; mutation score 80%.

File Killed Survived Timeout No coverage Invalid Not run
.github/scripts/pr-merge/clear-escalations.mjs 0 0 0 0 0 1
plugins/harness-ops/skills/plugins/scripts/sync-run.sh 4 1 0 0 0 0
plugins/source-control/skills/babysit-prs/scripts/babysit_merge.py 0 0 0 0 0 1
plugins/source-control/skills/babysit-prs/scripts/babysit_verify_evidence.py 0 0 0 0 0 1

Surviving mutants

Location Operator Disposition Cause
plugins/harness-ops/skills/plugins/scripts/sync-run.sh:1332 LCR productive input-gap

The run's summary has the full report.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant