Repository navigation
[Outcome Report] Workflow health — 2026-10-13 #68198
Description
Executive read: Quality is not yet measurable because no items are accepted or rejected; pending volume is highest in Smoke Claude Routed, four workflows lack evaluation evidence, and stuck status cannot be established without prior-cycle history.
| Workflow | Status | Lifecycle health | References |
|---|---|---|---|
| Smoke Claude Routed | 🔵🟨🟨🔵 | 🟡 in flight | 🔵 #68165 · 🟨 #68177 · 🟨 #68178 · 🔵 #68160 |
| Cache directory setup | 🟨 | 🟡 in flight | 🟨 #68171 |
| Daily Model Inventory Checker | 🟨 | 🟡 in flight | 🟨 #68185 |
| Matt Pocock Skills Reviewer | 🟨 | 🟡 in flight | 🟨 #68183 |
| Smoke Aider | 🟨 | 🟡 in flight | 🟨 #68193 |
| Smoke Copilot Routed | 🟨🔵🔵🔵🔵 | 🟡 in flight | 🟨 #68191 · 🔵 #68164 · 🔵 #68179 · 🔵 #68175 · 🔵 #68161 |
| Smoke Copilot SDK | 🟨 | 🟡 in flight | 🟨 #68186 |
| Smoke Copilot SDK Routed | 🔵🟨🔵🔵 | 🟡 in flight | 🔵 #68166 · 🟨 #68180 · 🔵 #68176 · 🔵 #68163 |
| Smoke Kiro | 🟨 | 🟡 in flight | 🟨 #68190 |
| Agent Job Health Monitor | ⬜ | ⚪ underdefined* | ⬜ #68173 |
| Detection Analysis Report | ⬜ | ⚪ underdefined* | ⬜ #68169 |
| PR Sous Chef | ⬜ | ⚪ underdefined* | ⬜ 38318809720 |
| Smoke Call Workflow | ⬜ | ⚪ underdefined* | ⬜ 38325132653 |
Legend:
- Status: 🟩 accepted · 🟥 rejected · 🟨 pending · ⬜ unknown · 🔵 lifecycle closure.
- Lifecycle health: 🟢 resolving · 🟡 in flight · 🟠 aging · 🔴 stuck · ⚪ underdefined.
- References: one linked item per status emoji, in the same order as the Status column. Missing target references use the repository and run ID supplied by the evaluation.
- 🔵 extends the requested status legend to preserve lifecycle outcomes without relabeling them as accepted, rejected, or unknown.
- *Underdefined is provisional from current missing evidence or unsupported evaluators; persistence across cycles is unverified. Cache-memory read/write tools are unavailable, so history could neither be compared nor saved.
🔴 Action Items
- Evaluation coverage: Add a dedicated discussion evaluator with explicit acceptance/rejection criteria for Agent Job Health Monitor and Detection Analysis Report. Restore target references for PR Sous Chef (
missing_reference, create_issue) and Smoke Call Workflow (missing_reference, call_workflow). Reassess their provisional underdefined classifications after evidence is available. - Pending review: No item exceeds 48 hours pending; the oldest is Cache directory setup #68171, approximately 37 minutes. No stuck classification can be confirmed without history.
- Waste: All workflows tie at 0% waste; there are no highest-waste offenders and no prompt/configuration review triggered by the >25% threshold.
- Acceptance and zero-touch: No workflow has an accepted or rejected item. The summary's 0% acceptance is below the attention threshold, but its denominator is zero; it does not establish poor quality. Human-edit dependence and low zero-touch performance cannot yet be assessed.
- Ignored and fallback evidence: Neither threshold is triggered: ignored and existence-only fallback counts are both zero.
- History availability: Provide cache-memory read/write capability for subsequent collectors so pending/unknown persistence and rate trends can be tracked.
Detailed metrics, evidence quality, workflow counts, and trends
Outcome Scorecard — 2026-10-13
| Metric | Value | Status |
|---|---|---|
| Acceptance rate | 0% reported; denominator 0 | — not measurable; below 60% attention threshold numerically |
| Zero-touch rate | 0% reported; denominator 0 | — not measurable |
| Waste rate | 0% | 🟢 <10% |
| Median time to resolution | 1.42 minutes | — |
| Accepted | 0 / 23 | — |
| — strong evidence | 0 | merged, completed, approved |
| — medium evidence | 0 | engaged, retained |
| — weak evidence | 0 | legacy weak acceptance |
| Rejected | 0 | — |
| Ignored | 0 | expected effect not observed |
| Zero-touch | 0 / 0 | — |
| Pending | 10 | — |
| Unknown | 4 | insufficient evidence |
| Evaluation errors | 0 | verification failed |
| Lifecycle | 9 | bot closures, not rejection |
| Runs checked | 23 | — |
Median resolution is the supplied fleet metric (85 seconds); this cycle's terminal outcomes are lifecycle closures, not acceptance/rejection decisions.
Per-Workflow Breakdown
All waste rates are 0%; ties are sorted alphabetically. Acceptance and zero-touch are shown as — where their denominators are zero.
| Workflow | Accepted | Rejected | Ignored | Pending | Unknown | Errors | Lifecycle | Acceptance | Zero-touch |
|---|---|---|---|---|---|---|---|---|---|
| Agent Job Health Monitor | 0 | 0 | 0 | 0 | 1 | 0 | 0 | — | — |
| Cache directory setup | 0 | 0 | 0 | 1 | 0 | 0 | 0 | — | — |
| Daily Model Inventory Checker | 0 | 0 | 0 | 1 | 0 | 0 | 0 | — | — |
| Detection Analysis Report | 0 | 0 | 0 | 0 | 1 | 0 | 0 | — | — |
| Matt Pocock Skills Reviewer | 0 | 0 | 0 | 1 | 0 | 0 | 0 | — | — |
| PR Sous Chef | 0 | 0 | 0 | 0 | 1 | 0 | 0 | — | — |
| Smoke Aider | 0 | 0 | 0 | 1 | 0 | 0 | 0 | — | — |
| Smoke Call Workflow | 0 | 0 | 0 | 0 | 1 | 0 | 0 | — | — |
| Smoke Claude Routed | 0 | 0 | 0 | 2 | 0 | 0 | 2 | — | — |
| Smoke Copilot Routed | 0 | 0 | 0 | 1 | 0 | 0 | 4 | — | — |
| Smoke Copilot SDK | 0 | 0 | 0 | 1 | 0 | 0 | 0 | — | — |
| Smoke Copilot SDK Routed | 0 | 0 | 0 | 1 | 0 | 0 | 3 | — | — |
| Smoke Kiro | 0 | 0 | 0 | 1 | 0 | 0 | 0 | — | — |
Evidence Quality
- Unknown: two unsupported discussion evaluations and two missing target references; each affected workflow has 100% unknown outcomes in this cycle.
- Generic existence-only fallback: 0 / 23 (0%).
- Pending ages range from 176 to 2,216 seconds; none exceeds 172,800 seconds (48 hours).
- Status/reference validation: all 13 workflow rows have matching emoji/token counts and matching positional statuses, covering all 23 outcomes. Each workflow preserves evaluation-file order.
- Metrics and per-workflow counts are retained in this issue, but could not be persisted to cache-memory. Historical trends and multi-cycle lifecycle classifications are unavailable.
📊 Measured by Outcome Collector · codex · gpt60 · 53.5 AIC · ⌖ 52.9 AIC · ⊞ 15.1K
- expires on Oct 19, 2026, 4:21 PM UTC-08:00
Activity
github-actions commentedon Oct 19, 2026
This issue is being closed as outdated. A newer issue has been created: #70066
View newer issue
This action was performed automatically by the
Outcome Collectorworkflow.