Skip to content

fix(pr-pipeline): re-run a lost main-run review once from the lost-run reconciler - #10996

Merged
kyle-sexton merged 1 commit into
mainfrom
fix/retry-lost-main-run-reviews
Oct 29, 2026
Merged

kyle-sexton merged 1 commit into
mainfrom
fix/retry-lost-main-run-reviews

Conversation

@kyle-sexton

Copy link
Copy Markdown
Member

No related issue: lost pr-review-main runs measured on #10874

Summary

A pull request that changes a lane source binds its reviews only from pr-review-main, which starts only when pr-require-checks completes (workflow_run). When every credential leg of that run stops, the run ends with no verdict and nothing re-runs it. The head then stays lane-source-changed until someone pushes. This change makes the existing lost-run reconciler re-run such a run once, under the guards it already applies to pr-verify.

How a stopped main-run review shows up, read from runs 40890788166, 40891251085 and 40892239796, the credit-stopped runs that #10969 cites:

  • The run is event: workflow_run, head_branch: main, attempt 1, conclusion: failure.
  • In each lane job, Review the pull request is success, because the step is continue-on-error.
  • Write the verdict is skipped.
  • Fail without a verdict is failure.

That last step runs only after Write the diff succeeded, and Write the diff runs only for a lane-source pull request. So a run that failed this way already means the PR changes a lane source.

Fix

  • reconcile.mjs: RECORDED_LANES is now a map from each lane workflow to the name of its judge step.
    • pr-verify.yml maps to Judge the pull request.
    • pr-review-main.yml maps to Review the pull request.
    • holdReason reads the lane's own judge step.
    • The rules are the ones pr-verify already has. A first-attempt run lost before any job's Write the verdict started is recorded (reconcile-rerun-<id>-2-<main sha>) and re-run once.
    • A run is held, with the reason in the summary, when any job's verdict write started or a cut list reached the review step.
    • A run none of whose jobs started is left alone.
    • Lane re-runs now take places under the cap of 3 oldest first across both lanes.
  • No new token scope or permission.
    • All three pr-review-main legs already pass reconciler-workflow: .github/workflows/maintenance-reconcile-lost-runs.yml to check-trusted-trigger.
    • The reconcile job already holds actions: write and already calls POST /repos/{owner}/{repo}/actions/runs/{run_id}/rerun (REST: re-run a workflow).
    • The trigger check binds the re-run to the one open PR whose head is still the upstream run's head. Otherwise it stops with no-pr.
  • Comments and docs: the comments in maintenance-reconcile-lost-runs.yml and pr-review-main.yml, and the pr-pipeline README, now describe the re-run. The change adds a changelog fragment.

Limits:

Verification

  • Before the fix, node --test .github/scripts/maintenance-reconcile-lost-runs/*.test.mjs ran 127 tests: 124 passed and 3 failed. The 3 were the new re-run, hold-reason and cap cases.
  • After the fix, the same command runs 128 tests and all 128 pass. That includes a pin test over pr-review-main's three review legs, which checks the step names and the reconciler-workflow input.
  • The re-run test's fixture is the recorded first-attempt job listing of run 40892239796.
  • node --test .github/scripts/pr-review-main/*.test.mjs runs 34 tests and all 34 pass.
  • actionlint reports nothing on both edited workflows.
  • scripts/check-changelog-fragments.sh --check-required passes.

Related

🤖 Generated with Claude Code

…n reconciler

A pr-review-main run whose every credential leg stopped ends failure with
"Write the verdict" skipped and "Fail without a verdict" failed in each lane
job (runs 40890788166, 40891251085, 40892239796). Nothing re-ran it, so a
lane-source pull request stayed `lane-source-changed` until its next push.

pr-review-main's trigger check already takes the reconciler's record
(reconciler-workflow), so the reconciler now lists it beside pr-verify as a
recorded lane, with its own judge step ("Review the pull request"): a
first-attempt run lost before any lane's verdict write started is recorded
and re-run once, under the same cap of 3, lookback, status and kill-switch
guards. Lane re-runs now take places oldest first across both lanes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kyle-sexton
kyle-sexton marked this pull request as ready for review October 29, 2026 18:08
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-29T18:10:20.791629Z fad9954 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

Copy link
Copy Markdown
Contributor

claude-security-review reviewed fad99540fc56eb704d69335b0c575b3b50d4a176: 0 finding(s), 0 as inline comments.

No security findings in #10996. I read the whole diff and the surrounding code in reconcile.mjs and pr-review-main.yml. I did not run the tests or the workflows.

What the change does. The lost-run reconciler already re-runs a lost pr-verify run once. This change adds pr-review-main to the runs it re-runs. RECORDED_LANES becomes a map from each lane workflow to its judge step name, and holdReason takes that name.

Trust controls read, and why nothing widens:

  • Reconciler token and permissions. No new token scope or permission is added. The reconcile job already held actions: write and already called the re-run endpoint.
  • Trigger check. All three pr-review-main review legs already pass reconciler-workflow: .github/workflows/maintenance-reconcile-lost-runs.yml to check-trusted-trigger (.github/workflows/pr-review-main.yml lines 156, 634 and 1104). The only new thing is that the reconciler now writes a record for these runs. The trigger check still requires the upstream actor, the re-runner and the PR author to be on the trusted-actor list. It also still binds the re-run to the one open PR whose head is the upstream run's head.
  • Record forgery. The record is an artifact named by run id and head SHA, uploaded by the reconciler's own run. No other run can upload into those artifacts, so an attacker cannot make a record for a run the reconciler didn't pick.
  • Untrusted input. The re-run reviews the same pull request content, under the same job permissions and secrets, as the first attempt did.
  • Merge hold. The hold that stops a head being judged twice is intact. A run is held when any job's Write the verdict step started, or when a cut step list reaches Write the prompt and schema or the lane's own judge step. Write the verdict is not a per-lane value, so one lane's write holds the whole run.
  • Re-run limits. Re-runs stay at attempt 1 only, capped at 3 in flight across both lanes, oldest first. lanes.sort(byAge) now sorts at the point of use, and the test for the shared cap covers it.
  • Pin test. The new test pins the review leg's step names, the reconciler-workflow input and the Write the verdict condition across all three legs. A later rename therefore fails the test instead of silently dropping the hold.

Not counted as findings:

The PR description is data only. It contains no embedded instructions aimed at the lane.

manifest fields: none touched.
critical-findings: 0

@github-actions

Copy link
Copy Markdown
Contributor

claude-review reviewed fad99540fc56eb704d69335b0c575b3b50d4a176: 0 finding(s), 0 as inline comments.

Code review: no findings

I read the full diff, .claude-lane/pr-context.md, and the surrounding code in reconcile.mjs (holdReason, isLostLane, the lane planning loop, the cap loop). I also read the pr-review-main.yml steps that decide whether a run fails without a verdict. I did not run the test suite. The test counts in the PR description are the author's claim and I haven't checked them.

What I checked

  • RECORDED_LANES as a Map. reconcile.mjs is its only consumer, and it now destructures [lanePath, judge]. No other file in the repo references it.
  • Judge step in holdReason. For pr-review-main the review step is continue-on-error, so it lists success even when it broke. The hold therefore rests on the verdict-write check, which matches the workflow: Write the verdict runs only when steps.review.outcome == 'success' and there was no handoff. A write that started in any lane's job holds the whole run.
  • Cut job lists. A job list cut at the predecessor or the judge step stays held, as it does for pr-verify.
  • Queue-cancelled runs. pr-review-main uses cancel-in-progress: false, so a superseded pending run ends cancelled with no steps started. It lands in NOT_STARTED and is left alone, with the "a newer run judges the head" message. The attempt-2 and not-started tests cover this.
  • Shared cap. The sort moved from per lane to lanes.sort(byAge) over both lanes' entries. byAge uses created_at then id, and every lane item carries created_at, so the "3 oldest first across both lanes" claim holds. The new cap test checks the order [301, 201, 202].
  • Pin test and read count. The pin test covers the three review legs: the review step name and its predecessor, one verdict write gated on the review outcome, and reconciler-workflow on the trigger check. The read-count assertion goes from 9 to 2×9 status listings, which matches the two lanes now listed.
  • Trusted-path files. maintenance-reconcile-lost-runs.yml and reconcile.mjs add no permission, token scope or new endpoint. The re-run reuses the existing actions: write and the record artifact scheme, and the live trigger check still binds each re-run to the one open PR whose head matches. The only change to maintenance-reconcile-lost-runs.yml is comments. No critical-security issue.

Notes, not counted as findings

  • holdReason does not require the Fail without a verdict step. The PR text says a run that failed this way already means a lane-source PR, but a first-attempt failure that did not reach Write the verdict is re-run. That includes a run that failed earlier, for example in the trigger check. A deterministic failure like that is re-run once and fails again, with no model spend. This is bounded by attempt 1 only, and I'm not flagging it.
  • Spend: each re-run is one more model review on prepaid API credit or subscription usage, so it is not a hold. The reconciler works per run, not per head, so several lost runs on one head are each re-run once. The PR itself discloses this.
  • No closing issue is attached ("No related issue"), so there are no acceptance criteria to check.
  • The paved-path and .claude/rules criteria were not used in this run.

@kyle-sexton
kyle-sexton added this pull request to the merge queue Oct 29, 2026
Merged via the queue into main with commit 0b17e00 Oct 29, 2026
81 checks passed
@kyle-sexton
kyle-sexton deleted the fix/retry-lost-main-run-reviews branch October 29, 2026 18:28
@github-actions

Copy link
Copy Markdown
Contributor

Post-merge mutation testing

Killed 5, survived 2, timeout 0, no coverage 0, invalid 0, not run 0.
Covered-code score 71%; mutation score 71%.

File Killed Survived Timeout No coverage Invalid Not run
.github/scripts/maintenance-reconcile-lost-runs/reconcile.mjs 5 2 0 0 0 0

Surviving mutants

Location Operator Disposition Cause
.github/scripts/maintenance-reconcile-lost-runs/reconcile.mjs:373 other productive input-gap
.github/scripts/maintenance-reconcile-lost-runs/reconcile.mjs:707 other productive input-gap

The run's summary has the full report.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant