Skip to content

pr-review: the bubblewrap install step times out at 1 minute and fails review lanes with no outcome #9073

Description

@kyle-sexton

The claude-review lanes fail with no review outcome when their "Install subprocess isolation" step (apt install of bubblewrap) exceeds its 1-minute timeout. On PR #9067 this happened twice within about 30 minutes on 2026-10-22:

Each failure blocks the PR until someone re-runs the job, and the job reports no-outcome rather than an infrastructure class.

Possible fixes: cache the bubblewrap .deb (or use a runner image that ships it), raise the step timeout with a measured basis, or retry the install step and classify a timeout there as infrastructure so the status job can re-run it automatically.

Done when a timed-out install no longer fails a lane without a retry, measured over a week of runs.

Activity

  1. added
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Oct 22, 2026
  2. github-actions commented on Oct 22, 2026

    @github-actions
    Contributor

    Proposed work class: work-class: scoped
    Basis: automated intake triage; the lanes App applied this class.

    Size: Medium
    Complexity: Medium
    Uncertainty: Known
    Flags: none
    Blocked by: #10142

    Decision recorded. The owner answered in the Refinement and Issue Triage sessions (2026-10-25), deciding the open items below on the facts:

    go with whatever the facts/data/metrics + /discovery:research says and ensure we're aligned with SOLID, DRY/SSOT principles

    [quote truncated; the full decision is at the link]

    #9073 (comment)

    The decision settles the open questions: the timeout fix lives in ci-workflows#893 with a pin bump here, the babysit merge gate re-runs install-failed on attempt 1 with no 5-hour wait, this repo's setup-runner-isolation stays unchanged, and the week-of-runs check becomes a person check. Nothing is left to ask. The work depends on #10142 (rerun_binds).

    Automated triage, applied by the lanes App.

  3. added
    needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.
    and removed
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Oct 22, 2026
  4. removed
    needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.
    on Oct 25, 2026
  5. kyle-sexton commented on Oct 25, 2026

    @kyle-sexton
    MemberAuthor

    Released for re-triage under the owner's 2026-10-24 rule: an issue stops for a person only for an intent or vision question, a security call, a host or attended run, or spend outside prepaid credit. The hold label came off and intake triage runs again now; it re-applies a hold, with a numbered question, when one of those applies.

    To hold this issue again, add needs-human (or status: needs-decision) and say why in a comment.

  6. added
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    priority: highSignificant impact, or blocks an imminent release; staff this cycle.
    and removed
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Oct 25, 2026
  7. added
    status: readyTriaged, unblocked, and fully specified; eligible to pick up.
    work-class: scopedA briefed fix or small feature; blast radius bounded by the brief, tests exist.
    status: needs-decisionAwaiting a human or maintainer judgment call.
    and removed
    status: readyTriaged, unblocked, and fully specified; eligible to pick up.
    on Oct 25, 2026
  8. melodic-automation-lanes commented on Oct 25, 2026

    @melodic-automation-lanes
    Contributor

    Not admitted to the issue lane: no trusted-human acceptance criteria: a listed human must put the confirmed - [ ] criteria in the body or post them as their own comment, or, on an issue a listed human filed, intake's newest refinement comment must hold them unedited. Admission takes the issue once each of those is met, when intake triages it again (.github/actions/resolve-config/admission-rules.json).

  9. 5 remaining items

  10. melodic-automation-lanes commented on Oct 25, 2026

    @melodic-automation-lanes
    Contributor

    Refinement. A read-only run explored this repository, searched its issues and pull requests, and read the allowed docs to draft what follows. On an issue a listed human filed, it counts as the issue's spec once posted, while unedited; it is still model text, so check it.

    Intended outcome: The review lanes' bubblewrap install retries on timeout, or has a measured longer timeout, and a timed-out install is classified as infrastructure rather than no-outcome, so a lane no longer fails without a retry.

    Related work

    Open questions

    1. Where does the 1-minute timeout live? The failing lanes (pr-review and pr-review-security) call melodic-software/ci-workflows pr-review-cli.yml, which is outside this repo, so is the fix there, with this repo only bumping its pin?
      RECOMMENDED: Treat the fix as a ci-workflows change plus a pin bump here, and file or link that work as a dependency. Basis: .github/workflows/pr-review-hosted.yml:172
    2. Which fix should be used: a retry with infrastructure classification, a longer timeout, or caching the .deb?
      RECOMMENDED: Retry the install and classify a timeout as infrastructure so the status job re-runs it. Do not cache, and do not raise the timeout without measured install times. Basis: judgment
    3. The local setup-runner-isolation action already retries apt 3 times with no step timeout, so should it also be changed, since pr-run-activity jobs share it?
      RECOMMENDED: Leave it as is. Adding a per-attempt timeout there is optional and would need a documented change to step 9 of the contract. Basis: .github/actions/setup-runner-isolation/action.yml:26
    4. The done-when 'measured over a week of runs' cannot be checked in one run, so may it be replaced with checkable criteria?
      RECOMMENDED: Yes. Use: a test or workflow assertion that a timed-out install step yields an infrastructure failure-class and triggers a re-run, with the week-long measurement kept as a follow-up person check. Basis: judgment

    Sources read

    • .github/actions/setup-runner-isolation/action.yml:22: the Install subprocess isolation step retries apt-get update and install up to 3 times with sleep 5, has no timeout-minutes, and fails if bwrap is missing
    • docs/conventions/pr-pipeline/pr-run-activity.md:277: step 9 documents the apt-get install with three tries and the fail-red check for bwrap
    • .github/workflows/pr-review-hosted.yml:172: the review lane calls melodic-software/ci-workflows pr-review-cli.yml at v0.65.9, so the failing step is defined outside this repo
    • .github/workflows/pr-verify.yml:265: the local lanes use a 22-minute step timeout, not 1 minute

    Automated refinement by the lanes App. Answer with a comment whose first line is **Decision recorded.**.

  11. kyle-sexton commented on Oct 26, 2026

    @kyle-sexton
    MemberAuthor

    Decision recorded. The owner answered in the Refinement and Issue Triage sessions (2026-10-25), deciding the open items below on the facts:

    go with whatever the facts/data/metrics + /discovery:research says and ensure we're aligned with SOLID, DRY/SSOT principles

    I should only ever be in the loop ONLY when necessary (intent/vision changed, something came up during implementation that impacts trajectory/alignment, OR a high security risk) - I've covered this all before

    No owner ledger (.work/*/interview-checklist.md, .work/gh-monitor-owner-decisions/) names this issue.

    Questions from intake's refinement (2026-10-25), each decided against melodic-software/ci-workflows at the pin this repo uses (pr-review-cli.yml@e5167fc2, v0.65.11, .github/workflows/pr-review-hosted.yml:172):

    1. Where does the timeout live?
      Decided: In ci-workflows (ci-workflows#893), with a pin bump here. The step is now "Isolate the runner" (pr-review-cli.yml:398, :1486; pr-review-security.yml has its own copy). Part of the fix has already landed there (feat(naming): fold running-retro tournament lessons into name-it-better #862, 2026-10-23): the step limit is 2 minutes, the first install skips apt-get update, every apt call shares a 100 s deadline (50 s in pr-review-security.yml), and a slow mirror ends in a named ::error.
    2. Which fix?
      Decided: Report the existing install-failed class from a failed "Isolate the runner" step (ci-workflows#893, fix(review): report a failed runner isolation install as install-failed ci-workflows#893, all five copies), and re-run it through the one re-run path that already exists for review lanes: the babysit merge gate. That gate re-runs a failed AI review check under the owner's token today, but only for class=rate-limit or failure-class=publish-failed, after a 5-hour wait, on attempt 1 only (plugins/source-control/skills/babysit-prs/reference/safety.md:847-858). Every other class is refused ("failure class is not rate-limit; never re-run", babysit_merge.py:403). Add install-failed to that allow-list with no 5-hour wait: the wait exists for the session-limit window, and an apt mirror failure is short. Keep the attempt-1-only, one-re-run-per-check-per-head and pinned-live-head rules as they are.
      Not the reconciler: it leaves review lanes to a person because "their gates or the lanes token broker refuse a re-run github-actions[bot] starts" (reconcile.mjs:44-46). Adding them there would widen a trust gate, which is the owner's call. Reusing the gate's path changes no trust boundary (SOLID/DRY: one re-run path). No .deb cache and no longer timeout: the shared 100 s deadline already bounds the install, and no measurement shows it is too short.
    3. Change this repo's setup-runner-isolation too?
      Decided: No. It retries apt three times with no step limit (.github/actions/setup-runner-isolation/action.yml:22-26) and is not the failing step.
    4. Replace "measured over a week of runs"?
      Decided: Yes. The week of runs becomes a person check; the lane criteria below can be checked in one run.

    Depends on #10142 (#10142), which adds rerun_binds(lane, job). A re-run only counts if the review evidence binds attempt 2, and today _bound_record binds attempt 2 only after a rate limit (babysit_review_evidence.py:348, :379).

    Acceptance criteria:

    • ci-workflows#893 is released, and this repo's pr-review-hosted.yml and pr-review-security-hosted.yml pin it (Lane Funding's repin PR).
    • babysit_merge.py re-runs the whole run of an AI review check whose failure is install-failed on attempt 1, with no 5-hour wait, under the existing head, attempt and job-binding checks; every other class except rate-limit and publish-failed is still refused.
    • A test in the babysit-prs script tests asserts that an install-failed check gets one re-run, a second attempt gets none, and an unlisted class stays refused.
    • safety.md:847-858 names install-failed beside rate-limit and publish-failed.
    • install-failed is defined once in babysit_review_evidence, and rerun_binds accepts it, so attempt 2 after an install failure binds.
    • The merge gate's re-run trigger set (rate-limit, publish rate-limit, install-failed) is read from that one definition, and a test asserts it is a subset of the classes rerun_binds accepts.

    Checks a person runs:

    • Over a week of review-lane runs, no lane stays red on the install step without a re-run.
  12. added
    status: readyTriaged, unblocked, and fully specified; eligible to pick up.
    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.
    and removed on Oct 26, 2026
  13. melodic-automation-lanes commented on Oct 26, 2026

    @melodic-automation-lanes
    Contributor

    The work-resolve-issue run that agent-ready started ended over 3 hours ago with no report and no pull request. work-unblock removed and added agent-ready back to start a fresh run on main's workflow files. If this run ends the same way, it re-fires the issue again, waiting longer each time.

  14. added
    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.
    and removed
    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.
    on Oct 26, 2026
  15. melodic-automation-lanes commented on Oct 27, 2026

    @melodic-automation-lanes
    Contributor

    The work-resolve-issue run that agent-ready started failed or was cancelled before it reported, or it stopped or the token broker refused it because its workflow files differ from main's. work-unblock removed and added agent-ready back to start a fresh run on main's workflow files. If this run ends the same way, it re-fires the issue again, waiting longer each time.

  16. added
    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.
    and removed
    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.
    on Oct 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.priority: highSignificant impact, or blocks an imminent release; staff this cycle.status: readyTriaged, unblocked, and fully specified; eligible to pick up.work-class: scopedA briefed fix or small feature; blast radius bounded by the brief, tests exist.

    Type

    Fields

    No fields configured for Bug.

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions