[aw] Systemic: codex engine + gpt-5.3-codex recurring model_not_supported_error across 12+ workflows

[github-actions]

Systemic: codex engine + gpt-5.3-codex model recurring model_not_supported_error

Filed by: Agent Performance Analyzer (weekly report, 2026-09-12)

Problem

Across the last week (2026-09-06 → 2026-09-12), 19 of the last 25 "produced no safe outputs"
failure issues carry the identical model_not_supported_error + missing_safe_outputs failure
category, all traced to workflows using engine: {id: codex} with model: copilot/gpt-5.3-codex
or model: openai/gpt-5.3-codex.

Affected workflows observed failing (issue numbers in parens):

• Metrics Collector — 6 recurrences: [aw] Metrics Collector produced no safe outputs #58895, [aw] Metrics Collector produced no safe outputs #59105, [aw] Metrics Collector produced no safe outputs #59344, [aw] Metrics Collector produced no safe outputs #59611, [aw] Metrics Collector produced no safe outputs #59851, [aw] Metrics Collector produced no safe outputs #60157
• Daily Evals Feature Report — 3: [aw] Daily Evals Feature Report produced no safe outputs #59665, [aw] Daily Evals Feature Report produced no safe outputs #59918, [aw] Daily Evals Feature Report produced no safe outputs #60221
• Daily CLI Performance Agent — 3: [aw] Daily CLI Performance Agent produced no safe outputs #59762, [aw] Daily CLI Performance Agent produced no safe outputs #60000, [aw] Daily CLI Performance Agent produced no safe outputs #60290
• Daily Cache Strategy Analyzer — 2: [aw] Daily Cache Strategy Analyzer produced no safe outputs #60039, [aw] Daily Cache Strategy Analyzer produced no safe outputs #60319
• Daily Regulatory Report Generator — 2: [aw] Daily Regulatory Report Generator produced no safe outputs #60056, [aw] Daily Regulatory Report Generator produced no safe outputs #60333
• Daily Documentation Diagram — [aw] Daily Documentation Diagram produced no safe outputs #60373 (open)
• Daily Documentation Updater — [aw] Daily Documentation Updater produced no safe outputs #60392 (open)
• PureLock — [aw] PureLock produced no safe outputs #60405 (open)
• Auto-Triage Issues — [aw] Auto-Triage Issues produced no safe outputs #60406 (open)

All issues have auto-expired/closed as not_planned without the underlying config being fixed, so
the same failure recurs roughly daily across a rotating set of workflows.

Why this matters most right now

Metrics Collector — the workflow every meta-orchestrator (Agent Performance Analyzer, Campaign
Manager, Workflow Health Manager) depends on for fresh data — is itself hit by this bug. Its
metrics/latest.json snapshot has been stale since 2026-09-01 (11+ days) as a direct result,
degrading scoring quality across all three orchestrators for at least 3 consecutive weekly runs.

Root cause hypothesis

gpt-5.3-codex is listed as a valid model string for both the openai and github-copilot
providers in pkg/cli/data/models.json, so the model name itself is not invalid. The failure is
consistent with either:

1. Copilot/OpenAI account or org policy disabling gpt-5.3-codex specifically for the codex
   CLI engine (as opposed to other engines that reference the same model string), or
2. An engine/provider resolution gap in pkg/workflow/codex_engine.go
   (ResolveLLMProvider/codexModelID) for this particular model+engine combination.

Correction to prior analysis: earlier shared-alerts notes (2026-09-10/11) reclassified this
pattern as "transient model-availability/policy flakiness, not a hard config defect" based only on
lint-monster and daily-go-test-parallelizer showing mostly-successful recent runs. That does not
hold up against the fuller 2026-09-06→09-12 dataset, which shows an expanding, daily-recurring
pattern across 12+ distinct workflows, not two isolated cases.

Suggested fix

1. Verify whether gpt-5.3-codex is actually enabled under the codex engine's provider
   resolution path for this org/repo (check Copilot policy settings, and/or OpenAI API key
   entitlements if model-provider: openai is used).
2. Until resolved, fall back the most-frequently-failing workflows (Metrics Collector first, as it
   blocks downstream metrics) to a known-good alternative such as gpt-5.2-codex (also listed for
   both providers) via gh aw update.
3. Add a day-over-day failure-rate guard/alert so a policy-level model outage doesn't silently
   degrade Metrics Collector (and therefore all meta-orchestrator scoring) for 11+ days before
   being escalated.

Expected impact

Eliminates the dominant recurring failure category (76% of last 25 failure issues), restores
Metrics Collector's daily snapshot, and stops ~1 new auto-filed "no safe outputs" issue per day
across a rotating set of workflows.

Generated by ⚡ Agent Performance Analyzer - Meta-Orchestrator · copilot · auto · 93.5 AIC · ⌖ 8.22 AIC · ⊞ 18.2K · ◷

• expires on Sep 14, 2026, 4:55 AM UTC-08:00