Skip to content

ci: standards sync PR #9076 waited 3.5 h behind a hosted ubuntu-24.04 runner queue #9139

Open
Task
Open
ci: standards sync PR #9076 waited 3.5 h behind a hosted ubuntu-24.04 runner queue#9139
Task
Labels
priority: mediumReal value, no hard deadline; normal backlog flow.status: confirmedA maintainer confirmed the item is real and in scope; ready to be worked.status: needs-decisionAwaiting a human or maintainer judgment call.work-class: read-onlyAudits, research, reports. No repository mutation; tracker and queue writes only.

Description

@kyle-sexton
kyle-sexton
opened on Oct 22, 2026

What happened

On 2026-10-22, standards sync PR #9076 waited about 3.5 hours to merge: opened 14:23Z, merged at about 17:55Z. The previous sync PR, #9068, took 10 minutes. The checks themselves passed; the time was spent waiting for runners.

  • At 16:38Z this repository had 20 workflow runs queued and 6 in progress.
  • chore: sync standards components #9076's ci-status job (job 119949584561, label ubuntu-24.04) had been queued for 8 minutes with no runner assigned.
  • At the same time, the sync app pushed a new head to chore: sync standards components #9076 (bac1cfa, 16:25Z), which restarted its CI.
  • The superseded ci-status run on the earlier head failed by design ("superseded by full run … in flight"). A REST watcher that treats any failed check as final stops on that.

At 18:05Z the queue was down to 3 runs, so the backlog was temporary.

Why it matters

Every cross-repo change that needs a standards sync first, such as a ci-workflows pin bump whose runner-policy contract arrives through the sync, waits behind this queue. It cost #9074 about 3.5 hours.

Open question

Which runs filled the hosted ubuntu-24.04 queue between 14:30Z and 17:30Z? Two possible causes:

  • the org's hosted-runner concurrency limit;
  • one workflow fanning out.

The fix differs: move jobs to the managed fleet, or add a concurrency group to the workflow that fans out. A per-workflow breakdown of queued time for that window would settle it.

Refs: melodic-software/claude-code-account-rotation#288

🤖 Generated with Claude Code

Activity

  1. github-actions
    added
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Oct 22, 2026

  2. github-actions commented on Oct 22, 2026

    @github-actions
    github-actionsBot
    on Oct 22, 2026 – with GitHub Actions
    Contributor

    Proposed work class: work-class: read-only
    Basis: automated intake triage; the lanes App applied this class.

    Size: Small
    Complexity: Medium
    Uncertainty: Knowable
    Flags: none
    Blocked by: none

    Understood: on 2026-10-22, standards sync PR #9076 spent about 3.5 hours waiting on the hosted ubuntu-24.04 queue. The checks passed. The open question is what filled that queue between 14:30Z and 17:30Z: the org's hosted-runner concurrency limit, or one workflow fanning out. A per-workflow breakdown of queued time for that window would settle which fix applies: move jobs to the managed fleet, or add a concurrency group.

    Nothing else is missing. The first step is read-only: pull the run data for that window and attribute the queued time. No likely duplicates found among the open issues.

    Automated triage, applied by the lanes App.

  3. github-actions
    added
    priority: mediumReal value, no hard deadline; normal backlog flow.
    on Oct 22, 2026
  4. github-actions
    added the
    Task
    issue type on Oct 22, 2026
  5. github-actions
    removed
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Oct 22, 2026

  6. github-actions commented on Oct 26, 2026

    @github-actions
    github-actionsBot
    on Oct 26, 2026 – with GitHub Actions
    Contributor

    The triage sweep dispatched intake triage for this issue (attempt 1 of 2).

  7. melodic-automation-lanes
    added
    status: readyTriaged, unblocked, and fully specified; eligible to pick up.
    work-class: read-onlyAudits, research, reports. No repository mutation; tracker and queue writes only.
    status: confirmedA maintainer confirmed the item is real and in scope; ready to be worked.
    and removed
    status: readyTriaged, unblocked, and fully specified; eligible to pick up.
    on Oct 26, 2026

  8. melodic-automation-lanes commented on Oct 26, 2026

    @melodic-automation-lanes
    melodic-automation-lanesBot
    on Oct 26, 2026
    Contributor

    Not admitted to the issue lane: no trusted-human acceptance criteria: a listed human must put the confirmed - [ ] criteria in the body or post them as their own comment, or, on an issue a listed human filed, intake's newest refinement comment must hold them unedited. Admission takes the issue once each of those is met, when intake triages it again (.github/actions/resolve-config/admission-rules.json).

  9. melodic-automation-lanes commented on Oct 26, 2026

    @melodic-automation-lanes
    melodic-automation-lanesBot
    on Oct 26, 2026
    Contributor

    The latest triage verdict holds nothing: admission moved the issue to status: confirmed; its comment says why.

  10. melodic-automation-lanes
    added
    status: needs-decisionAwaiting a human or maintainer judgment call.
    on Oct 26, 2026

  11. melodic-automation-lanes commented on Oct 26, 2026

    @melodic-automation-lanes
    melodic-automation-lanesBot
    on Oct 26, 2026
    Contributor

    Refinement. A read-only run explored this repository, searched its issues and pull requests, and read the allowed docs to draft what follows. On an issue a listed human filed, it counts as the issue's spec once posted, while unedited; it is still model text, so check it.

    Intended outcome: A per-workflow breakdown of queued time on hosted ubuntu-24.04 for 2026-10-22 14:30Z-17:30Z attributes the backlog to the org runner limit or to one fanning-out workflow, and a follow-up fix issue is opened.

    Related work

    Open questions

    1. Can an unattended run query Actions run data for that window? The checkout holds no queue data, and the org-level data may need admin access.
      RECOMMENDED: Have a person or an admin-token job export created_at and started_at per workflow for 2026-10-22 14:30Z-17:30Z and attach it to the issue. Basis: judgment
    2. Should the deliverable be a report only, or also a fix (managed-fleet move or concurrency group)?
      RECOMMENDED: Report only; open a separate fix issue once the breakdown names the cause. Basis: judgment
    3. What is the org hosted-runner concurrency limit, and is moving jobs to the managed fleet acceptable to the owner?
      RECOMMENDED: The owner states the limit and approves or rejects the managed fleet before any move. Basis: judgment
    4. Should the superseded ci-status failure be handled here or left to the existing issue?
      RECOMMENDED: Leave it to #​9113 and keep this issue to queue attribution. Basis: 9113

    Sources read

    • /home/runner/work/_temp/refine-data/context.json: the issue asks which runs filled the hosted queue and includes no run data
    • /home/runner/work/claude-code-plugins/claude-code-plugins/.base/.github/workflows: 103 runs-on ubuntu-24.04 occurrences across 30 workflows (pr-unblock.yml 22, pr-require-checks.yml 12, intake-triage.yml 8), so many workflows could fan out

    Automated refinement by the lanes App. Answer with a comment whose first line is **Decision recorded.**.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    priority: mediumReal value, no hard deadline; normal backlog flow.status: confirmedA maintainer confirmed the item is real and in scope; ready to be worked.status: needs-decisionAwaiting a human or maintainer judgment call.work-class: read-onlyAudits, research, reports. No repository mutation; tracker and queue writes only.

    Type

    Task

    Fields

    No fields configured for Task.

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions