Skip to content

fix(source-control): pass over skipped-validation reds in review-venue - #8878

Merged
kyle-sexton merged 3 commits into
mainfrom
fix/review-venue-one-pr-red
Oct 21, 2026
Merged

kyle-sexton merged 3 commits into
mainfrom
fix/review-venue-one-pr-red

Conversation

@kyle-sexton

Copy link
Copy Markdown
Member

Closes #8867

Summary

review-venue took a lane's newest completed pull request run as the lane's health. A pull request that edits a review caller workflow ends both lanes red by design: claude-code-action skips itself because the caller differs from the default branch's copy, and the reusable workflow's last step fails with failure-class=skipped-validation. That one run (#8800's runs 39610040287 and 39610040502) read both lanes as local failing for every other pull request.

Fix

  • plugins/source-control/skills/pull-request/scripts/review-venue.sh: a run that did not succeed gets its jobs and failed jobs' annotations read. When every failed or timed-out job carries a failure-level annotation starting failure-class=skipped-validation, the run reaches no verdict and the lane's health comes from the run before it, like a skipped or cancelled run. Any other class (rate-limit, auth, ...) or a failed run with no failed job keeps its verdict, so a real outage still reads failing. After 10 such runs in a row the lane reads local stale. A failed jobs or annotations read exits 2, like every other read. A successful newest run costs no extra call.
  • The signal is the annotation the reusable workflows write: echo "::error title=$LANE failed::failure-class=${FAILURE_CLASS:-unknown}: $why" (ci-workflows .github/workflows/pr-review.yml:1392, pr-review-security.yml:1183).
  • Rule doc (skills/pull-request/reference/review-venue.md), GitHub provider doc (the two new reads, with pointer, as-of and recheck trigger) and a .changes/source-control patch fragment.

Of the issue's options, this is the first. Judging over the newest N runs would also hide a real outage seen on only one pull request, so a review could be skipped. Reading the run for the pull request in hand doesn't work because the venue is often read before the pull request exists.

Verification

  • review-venue.test.sh: 49 pass. The fix(source-control): stop one workflow-editing PR from marking both review lanes failing in review-venue #8867 reproduction (newest run a skipped-validation red, an older success → ci healthy) failed on the unfixed script and passes now. New cases: a real failure behind a skipped-validation red reads failing; several real failures read failing; a rate-limit red reads failing; only skipped-validation reds read stale; the window applies to the run behind the skipped one; unreadable jobs or annotations exit 2; a healthy lane reads no jobs.
  • Live: the classifier returns "skip" for runs 39610040287 and 39610040502 and "verdict" for a success run; review-venue.sh --repo melodic-software/claude-code-plugins reads both lanes ci healthy.
  • shellcheck clean on both files; markdownlint-cli2 clean on the three docs.

Related

🤖 Generated with Claude Code

kyle-sexton and others added 3 commits October 21, 2026 10:51
A pull request that edits a review caller workflow ends both review lanes
red by design (failure-class=skipped-validation). review-venue took that
run as the lane's verdict and marked both lanes local failing for every
other pull request. A failed run whose failed jobs all carry that
annotation now reaches no verdict; other failures still count.

Closes #8867

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kyle-sexton
kyle-sexton marked this pull request as ready for review October 21, 2026 15:21
@github-actions

Copy link
Copy Markdown
Contributor

claude-review reviewed 1a58c0e14e6b1712ec806bba322a61660e9434d7: 0 finding(s), 0 as inline comments.

No findings on PR #8878 at head 1a58c0e.

I read the full diff (.claude-lane/pr.diff) and the PR context, which holds the closing issue #8867. I did not run review-venue.test.sh: this session's permission system declined to run it. The PR body's "49 pass", shellcheck and live-run claims are author-claimed and unverified by me.

Closing issue #8867. The issue lists three options and the criterion that a real outage, where several recent runs fail, must still read failing. The PR takes the first option (skip the designed skipped-validation red).

  • The skipped_validation check passes over a run only when every failed or timed-out job carries a failure-level failure-class=skipped-validation annotation. A failed run with no failed job, or any other class (rate-limit, auth), keeps its verdict.
  • The new tests cover the reproduction, a real failure behind a skipped-validation red, repeated real failures, a rate-limit red and a failed-closed read. A failing annotation read is also covered. The "several real failures" and rate-limit cases would fail if the outage criterion broke.

Script logic. I traced review-venue.sh by hand for each outcome:

  • A newest run that succeeded breaks the loop at once with no extra gh call.
  • A skipped-validation red falls through to the next verdict run.
  • A real failure breaks and reads local failing.
  • Ten passed-over runs, or running out of runs, leaves rid empty and reads local stale.
  • The made timestamp is set only when rid is non-empty.
  • die inside skipped_validation exits the script, because the function is not run in a command substitution.

Docs and fragment. The rule doc, the GitHub provider doc and the patch fragment agree with the script on the 10-run bound and the fail-closed reads. The provider doc carries the pointer, as-of date and recheck trigger that the skill-body rule asks for.

Minor, not worth a block. After ten passed-over runs the stale line still says "has no run with a verdict among its newest 100". It is accurate enough, since the lane has no usable verdict.

Not checked. I did not look at the path-scoped rules (cost-claims.md and others) beyond the diff text. No prices or cost claims appear in it.

@github-actions

Copy link
Copy Markdown
Contributor

claude-security-review reviewed 1a58c0e14e6b1712ec806bba322a61660e9434d7: 0 finding(s), 0 as inline comments.

No security issues found in PR #8878 at head 1a58c0e.

The PR changes review-venue.sh so that a run whose failed jobs are all marked failure-class=skipped-validation is passed over, and the lane's health comes from the newest earlier run. The rest of the diff is docs, tests and a changelog fragment.

  • Forged annotation: a PR that edits the caller workflow could emit a forged skipped-validation annotation on its own failed run. The effect is limited to hiding that one run. Older real failures stay newest-first behind it and still read local failing. The pass-over is capped at 10 runs (MAX_PASSED); beyond that the lane reads local stale, which is not healthy, so it fails closed. This was not worth a finding.
  • Injection: run and job ids from the API are checked against ^[0-9]+$ before they go into later gh api paths. --repo is validated as before. The annotation message goes only into a jq test() regex, not into a shell command.
  • Error handling: a failed jobs or annotations read, or an unparseable response, exits 2 instead of reading healthy. The new tests cover both cases.
  • Instruction surfaces: the PR deletes or softens no standing instruction. The documentation edits only describe the new pass-over rule. I did not consult the instruction exception register, because no standing rule is removed.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-21T15:32:37.113617Z 1a58c0e Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@kyle-sexton
kyle-sexton added this pull request to the merge queue Oct 21, 2026
Merged via the queue into main with commit 79e2c83 Oct 21, 2026
59 checks passed
@kyle-sexton
kyle-sexton deleted the fix/review-venue-one-pr-red branch October 21, 2026 15:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(source-control): stop one workflow-editing PR from marking both review lanes failing in review-venue

1 participant