Follow-up to #1066, and strictly worse than the dwell reset that issue describes. #1066 says a busy agent's dwell restarts each tick. This is about the frontier itself resetting, which means outer rings can be unreachable indefinitely rather than merely delayed.
The mechanism
cmd_promote advances exactly one ring per invocation: the frontier, defined in scripts/canary-rollout.sh (~L921) as the first channel in ring order whose commit differs from the candidate:
if [ "$ch" = "next" ] || [ "$c" = "$cand" ]; then :; else frontier="$ch"; break; fi
The candidate is whatever <agent>/v<M>-next currently points at. So every autocut that advances next makes ring0 differ again, and the frontier snaps back to ring0 — regardless of how far the outer rings had progressed.
Observed
dev-lead, 2026-09-08. ring0 and ring1 had both been walked up to 90dcc01c, leaving stable as the only lagging ring:
v139-next 27b0e8a3 <- autocut moved it
v139-ring0 90dcc01c <- now differs from candidate => frontier resets HERE
v139-ring1 90dcc01c
v139-stable 40f6c232 <- the ring we actually needed to advance
promote reported gate=SOAKING for ring 'ring0' [next->ring0]. stable was three gated promotions away again, having been one away moments earlier.
Why this can be unbounded, not just slow
The cumulative dwell from next to stable for dev-lead is 4h + 8h + 12h = 24h (next->ring0 4h, ring0->ring1 8h, ring1->stable 12h). The canary schedule is cron: "33 */4 * * *" — every 4 hours. An agent whose watched paths change on most ticks therefore resets the frontier roughly every 4 hours, while needing 24 hours of uninterrupted progress to drain the pipeline.
For an active agent, stable is not merely delayed — it may be unreachable by normal promotion. dev-lead/v139-stable sat on 40f6c232 (2026-08-18) for three weeks while next advanced through eight releases (v139.4.0 → v139.11.0).
Second-order effect: it forces bad choices
Because the pipeline cannot drain, the only way to move stable is promote --override, repeatedly, walking each ring. That has two costs:
- It discards the soak evidence you were waiting for. On 2026-09-08 the fleet had 152 clean runs on
90dcc01c across three canary repos. Override-walking shipped 9e709367 — a commit with zero canary coverage — because each promote advances the target ring to the current candidate, not to the commit that was actually soaked.
- It normalises
--override, which is meant to be exceptional.
Acceptance criteria
Related
Follow-up to #1066, and strictly worse than the dwell reset that issue describes. #1066 says a busy agent's dwell restarts each tick. This is about the frontier itself resetting, which means outer rings can be unreachable indefinitely rather than merely delayed.
The mechanism
cmd_promoteadvances exactly one ring per invocation: the frontier, defined inscripts/canary-rollout.sh(~L921) as the first channel in ring order whose commit differs from the candidate:The candidate is whatever
<agent>/v<M>-nextcurrently points at. So every autocut that advancesnextmakesring0differ again, and the frontier snaps back toring0— regardless of how far the outer rings had progressed.Observed
dev-lead, 2026-09-08.ring0andring1had both been walked up to90dcc01c, leavingstableas the only lagging ring:promotereportedgate=SOAKING for ring 'ring0' [next->ring0].stablewas three gated promotions away again, having been one away moments earlier.Why this can be unbounded, not just slow
The cumulative dwell from
nexttostablefor dev-lead is 4h + 8h + 12h = 24h (next->ring04h,ring0->ring18h,ring1->stable12h). The canary schedule iscron: "33 */4 * * *"— every 4 hours. An agent whose watched paths change on most ticks therefore resets the frontier roughly every 4 hours, while needing 24 hours of uninterrupted progress to drain the pipeline.For an active agent,
stableis not merely delayed — it may be unreachable by normal promotion.dev-lead/v139-stablesat on40f6c232(2026-08-18) for three weeks whilenextadvanced through eight releases (v139.4.0→v139.11.0).Second-order effect: it forces bad choices
Because the pipeline cannot drain, the only way to move
stableispromote --override, repeatedly, walking each ring. That has two costs:90dcc01cacross three canary repos. Override-walking shipped9e709367— a commit with zero canary coverage — because each promote advances the target ring to the current candidate, not to the commit that was actually soaked.--override, which is meant to be exceptional.Acceptance criteria
nextmust not reset progress already made on outer rings. Options: pin the frontier to a specific candidate for the duration of its rollout; allow multiple rings to advance per run when each one's own gate passes; or track per-ring target versions rather than deriving everything from the single current candidate.--overrideand without walking the whole chain. Notecmd_rollbacklooks like this but is unusable for v-scoped channels — see the sibling issue on its major-scope bug.stablemust be reachable given the schedule interval, or the mismatch must be surfaced explicitly (e.g. "pipeline cannot drain at current cut cadence") rather than silently never completing.nextadvanced — outer-ring progress must not be discarded.Related
petry-projects/.github-private#1592— the delivery gap this whole family blocks