Skip to content

fix(pr-pipeline): record the main-run review's size stop as an outcome and collapse exact renames #10961

Closed

Description

@kyle-sexton

Problem

pr-review-main.yml "Write the diff" (lines ~167-191) refuses a verdict when the PR diff is over MAX_DIFF_BYTES (300000), holds a binary file, or has a line over LONG_LINE_BYTES. The refusal is correct (the lanes stay unbound and the merge gate holds the PR for a person), but:

  1. It shows as a failed run. Both review jobs exit 1 with ::error::, so a designed stop counts as a lane failure on the dashboard. Example: run https://github.com/melodic-software/claude-code-plugins/actions/runs/40894121985 (docs(contracts): move plugin-author contracts to docs/contracts #10952, head 08bd36a).
  2. A large PR has no path forward but a person. docs(contracts): move plugin-author contracts to docs/contracts #10952 moves 232 files (+421/-345 lines of real change). --no-renames prints every moved file whole, so the patch passes 300 KB, though almost all of it is unchanged content.

Proposal

  1. Emit the stop as a recorded outcome, not a failure: the step writes verdict=none reason=<too-large|binary|long-line> and exits 0, the record job posts it, and the binder holds for a person by reading that reason (fail closed when the record is missing). Keep ::notice::, not ::error::.
  2. Collapse exact renames: diff with rename detection limited to 100% similarity (-M100%), so a pure move prints only its rename from / rename to lines. Keep the whole-file view (today's --no-renames rule) for any rename whose destination is a lane source, or whose source and destination differ in lane-source status, so a move into a lane source is still read whole. Apply the size cap after collapsing.

Acceptance

  • A designed stop leaves the run green and the PR held with a named reason; a real step failure stays red.
  • A test fixture with N pure renames outside lane sources stays under the cap; a pure rename into a lane source still prints whole.
  • The security review of the change finds no HIGH/CRITICAL (this edits a gate workflow, so it is person-held).

Refs: #10743, #10952

Activity

  1. added
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Oct 29, 2026
  2. github-actions commented on Oct 29, 2026

    @github-actions
    Contributor

    Proposed work class: work-class: structural
    Basis: automated intake triage; the lanes App applied this class.

    Size: Medium
    Complexity: Medium
    Uncertainty: Known
    Flags: none
    Blocked by: none

    Understood: the main-run review's size, binary and long-line stops should be recorded as a named verdict=none reason=... outcome on a green run, with the merge gate still holding the PR for a person. Exact renames should also collapse (-M100%), so a pure-move PR like #10952 stays under the size cap. Renames into a lane source would still print whole.

    The refinement comment already lists the acceptance criteria, and the approach is clear. I found no duplicate. #10912 edits the same workflow's start trigger, so expect a possible merge conflict with it.

    Automated triage, applied by the lanes App.

  3. added
    priority: mediumReal value, no hard deadline; normal backlog flow.
    and removed
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Oct 29, 2026
  4. added
    status: readyTriaged, unblocked, and fully specified; eligible to pick up.
    work-class: structuralRefactors, migrations, contract changes; cross-cutting and hard to reverse.
    status: confirmedA maintainer confirmed the item is real and in scope; ready to be worked.
    and removed
    status: readyTriaged, unblocked, and fully specified; eligible to pick up.
    on Oct 29, 2026
  5. melodic-automation-lanes commented on Oct 29, 2026

    @melodic-automation-lanes
    Contributor

    Not admitted to the issue lane: a criterion names .github/workflows/pr-review-main.yml, which the issue lane cannot push (patch-protected-path); a builder session takes it. Admission takes the issue once each of those is met, when intake triages it again (.github/actions/resolve-config/admission-rules.json).

  6. melodic-automation-lanes commented on Oct 29, 2026

    @melodic-automation-lanes
    Contributor

    The latest triage verdict holds nothing: admission moved the issue to status: confirmed; its comment says why.

  7. added
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    and removed
    status: confirmedA maintainer confirmed the item is real and in scope; ready to be worked.
    on Oct 29, 2026
  8. melodic-automation-lanes commented on Oct 29, 2026

    @melodic-automation-lanes
    Contributor

    Refinement. A read-only run explored this repository, searched its issues and pull requests, and read the allowed docs to draft what follows. On an issue a listed human filed, it counts as the issue's spec once posted, while unedited; it is still model text, so check it.

    Intended outcome: The main-run review's size, binary and long-line stops are recorded as a named verdict=none reason on a green run that the merge gate reads as a hold, and exact renames collapse so a pure-move PR stays under the cap.

    Acceptance criteria

    • In .github/workflows/pr-review-main.yml the Write the diff step exits 0 for too-large, binary and long-line stops, prints ::notice:: not ::error::, and writes a reason of too-large, binary or long-line.
    • The empty-diff and part-line-over-PART_BYTES branches, and any real git or fetch failure, still exit 1 so a real step failure stays red.
    • Steps after Write the diff that gate on steps.diff.outcome == 'success' (trusted text, criteria copy, models, credential) are also gated on the step not having stopped, so no model leg runs after a designed stop; the Fail without a verdict step does not fire for a designed stop.
    • The record job posts the stop as a check run per lane on the reviewed head, titled with verdict=none and the reason, for example 'verdict=none reason=too-large', and the Record the counts loop validates the reason against ^(too-large|binary|long-line)$ and rejects anything else.
    • The stop reaches the record job without a model turn: the step passes it through a verdict artifact or job output that only this run can write, validated in the record job, never interpolated into a command.
    • babysit_review_evidence.py reads a verdict=none record for the head and returns the lane unbound with a named hold reason, and a missing, malformed or forged record still returns the same unbound result (fail closed).
    • A pytest case in plugins/source-control/skills/babysit-prs/scripts/tests passes for a verdict=none record on each of the three reasons, and for a missing or malformed record, each leaving the lane unbound.
    • The diff command uses -M100% in place of --no-renames, so a pure rename prints only its rename from / rename to lines.
    • A rename whose destination is a lane source, or whose source and destination differ in lane-source status, is still printed whole, using the same babysit_review_evidence.lane_source predicate that the select step uses.
    • MAX_DIFF_BYTES is applied to the diff after collapsing renames.
    • A fixture test builds a repository with N pure renames outside lane sources and shows the patch under MAX_DIFF_BYTES, and shows a pure rename into a lane source printed whole; the test passes in CI.
    • docs/conventions/pr-pipeline/README.md (around line 1077-1082, which says the diff uses --no-renames) and a .changes/pr-pipeline fragment describe the recorded stop and the rename rule, and a security review of the change finds no HIGH or CRITICAL.

    External state: none

    Dependencies

    • #​10743 (merged; it added the main-run binding this edits), so nothing is open.
    • #​10912 touches the same workflow's start trigger, so land or rebase in either order to avoid a conflict.

    Related work

    Sources read

    • .github/workflows/pr-review-main.yml:156-220: Write the diff step uses --no-renames and exits 1 with ::error:: on the binary (186), size (190) and long-line (195) stops; the step is also used by every later step gating on steps.diff.outcome == 'success'.
    • .github/workflows/pr-review-main.yml:435-439: Fail without a verdict fires only when the diff step succeeded and no verdict exists.
    • .github/workflows/pr-review-main.yml:448-485: the record job posts a neutral check run titled findings=N critical=N from verdict.json artifacts and fails on a malformed file.
    • plugins/source-control/skills/babysit-prs/scripts/babysit_review_evidence.py:405-439: _main_lane returns the lane unbound as lane-source-changed when no counted record binds, so a missing record already holds.
    • plugins/source-control/skills/babysit-prs/scripts/babysit_review_evidence.py:453-494: _main_bound binds only on a counted verdict plus a matching record-job check run counts pair.
    • plugins/source-control/skills/babysit-prs/scripts/babysit_review_evidence.py:583-594: lane_source(path) is the predicate for lane-source status.
    • docs/conventions/pr-pipeline/README.md:1077-1082: documents the long-line hold and that the diff uses --no-renames.

    Automated refinement by the lanes App. Intake triages the issue again with it.

  9. removed
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Oct 29, 2026
  10. added
    status: readyTriaged, unblocked, and fully specified; eligible to pick up.
    status: confirmedA maintainer confirmed the item is real and in scope; ready to be worked.
    and removed
    status: readyTriaged, unblocked, and fully specified; eligible to pick up.
    on Oct 29, 2026
  11. kyle-sexton commented on Oct 30, 2026

    @kyle-sexton
    MemberAuthor

    Closing as done: every acceptance criterion is met on main at b6dcb4a, by three merged pull requests, so no new change is needed.

    • #10969 collapses exact renames: .github/scripts/pr-review-main/write-diff.py prints --no-renames and, only when that patch is over the cap, re-runs with -M100%, printing whole any move into or out of a lane source (ev.lane_source(src) / ev.lane_source(dst), the gate's own predicate), a mode change, a symlink or submodule, a loaded path, or a move an added line names. The caps (binary, long-line, chunk split) run on its output (pr-review-main.yml, Write the diff step).
    • #11084 records the stops: the step's stop() prints ::notice::, writes <lane>-stop / <lane>-head outputs and exits 0; the record job's "Record the stops" step validates ^(too-large|binary|long-line|chunk-failed)$ and a 40-hex head, then posts the neutral check run verdict=none reason=<reason>. No model turn is involved. babysit_review_evidence._main_lane returns the lane unbound as main-review-stop:<reason>.
    • #11160 replaced the single size stop with chunked review; too-large now means a file over one chunk or more than MAX_CHUNKS chunks.

    Tests on main (run locally at b6dcb4a):

    • node --test .github/scripts/pr-review-main/*.test.mjs: 46 pass, 0 fail. These include "a diff the review cannot read whole stops green and names its reason for the record job", "after a stop no later step of the leg runs", "the record job posts each stop as a neutral verdict=none record, from a closed reason", "over the size cap a file moved unchanged outside the lane sources prints as its rename lines alone" (five 400-line moves under a 1000-byte cap), and "a move into or out of a lane source, or one that changes the mode, prints whole".
    • pytest test_babysit_review_evidence.py test_babysit_gh.py -k 'stop or main': 39 pass. They cover verdict=none for too-large, binary and long-line, plus a malformed reason=other, each leaving the lane unbound.

    Docs: docs/conventions/pr-pipeline/README.md (around lines 1126-1148) describes the rename rule and the verdict=none record. The changelog fragments shipped with those PRs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    priority: mediumReal value, no hard deadline; normal backlog flow.status: confirmedA maintainer confirmed the item is real and in scope; ready to be worked.work-class: structuralRefactors, migrations, contract changes; cross-cutting and hard to reverse.

    Type

    Fields

    No fields configured for Bug.

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions