fix(codex): hold a bound thread's account for its prompt cache (#4546)

devlog/_plan/260914_cost_guard_stabilization/000_unit.md

@@ -0,0 +1,120 @@
+# 260914 — Cost-guard stabilization for pooled Codex routing (#4546)
+
+## Where this starts
+
+#4546 reports that account-pool routing moved a **live** conversation between
+accounts once the pool got hot, discarding the account-isolated prompt-cache
+prefix on every hop. The reporter measured roughly 1.9 billion total tokens and
+323 million uncached tokens across 15,607 requests in about thirteen hours on
+five accounts, with a 7k-token turn arriving upstream as a 150k-token turn.
+
+Those are two different numbers and this unit keeps them apart. Total tokens,
+uncached throughput, billed API cost, and subscription quota drawdown are four
+separate quantities; only the second is directly attributable to a routing
+decision, and the Pro-plan quota-to-dollar conversion is not verifiable from the
+report. The defect is real regardless: uncached throughput is the thing routing
+controls, and routing multiplied it.
+
+## The shape of the defect
+
+The single-threshold rule is the visible half. `autoSwitchThreshold` (default 80)
+answers two unrelated questions with one number: *should a new session be placed
+here* and *should an existing session be evicted from here*. Those have opposite
+cost structures. Placing a new session on a cooler account costs nothing, because
+there is no warm prefix yet. Evicting a live session throws away a prefix that was
+paid for once and would otherwise be reused for the rest of the conversation.
+
+The invisible half is that nothing put a floor under the destination. The bound
+thread moved to whichever eligible account was **strictly cooler** — by any margin.
+Once every account sits in the 80–99% band the coolest one is still over the
+threshold, so the next turn moves again. Because the same predicate also
+short-circuits the 60-second re-score interval, a thread in that band is
+re-scored on *every request* rather than once a minute. That is the ping-pong.
+
+`pool.cacheAffinity` (#4292, merged 2026-09-12) already raises the eviction bar to
+genuine exhaustion, but it is opt-in and off by default, so no existing install is
+protected by it. And turning it on does not close the hole: a transient failure
+streak deletes the binding through a different code path that never consults the
+flag.
+
+## Objective
+
+Make the reported incident structurally impossible rather than less likely, in
+priority order, with each work phase independently revertible.
+
+The governing policy, stated once:
+
+> **A live binding is held for cache; a new session is placed for capacity; a
+> failure is handled at the scope where it actually occurred; and expensive work
+> is bounded before it is sent, not after it is billed.**
+
+## Roadmap
+
+| Doc | Work phase | Outcome |
+| --- | --- | --- |
+| `010_bound_binding_policy.md` | wp2 | Cache-first is the default for bound threads, and a move requires a destination with real headroom |
+| `020_backoff_preserves_binding.md` | wp3 | A transient streak routes around an account without surrendering ownership of the thread |
+| `030_move_reason_evidence.md` | wp3 | Every live-binding move carries a machine-readable reason |
+| `040_send_budget.md` | wp4 | One logical request has one total send budget across every retry layer |
+| `050_worker_isolation.md` | wp5 | Fan-out cannot consume the capacity an interactive session is bound to |
+| `060_quota_cache_domains.md` | wp6 | Credentials are grouped by observed quota and cache domain, not by string identity |
+| `070_delivery.md` | wp7 | Delivery, verification posture, and merge policy |
+
+wp2 and wp3 are the incident. wp4 through wp6 are the amplifiers that turn a
+routing mistake into a cost event; they ship after the incident is closed.
+
+## Relationship to #4581
+
+The L2 lane unit `devlog/_plan/260914_l2_pool_routing_cache/010_cache_safe_rebind.md`
+reached the headroom floor by a different route -- keep the threshold eviction rule,
+constrain the destination -- and landed on `dev` as #4581 while this unit was in
+flight. That analysis is correct and this work **builds on it** rather than beside
+it: `pickCacheSafeQuotaReplacement` is the shipped destination rule and both call
+sites here use it unchanged.
+
+What it deliberately left open, recorded in its own review, is this unit's scope: a
+below-threshold sibling still takes the thread once, so the prefix is lost one time
+before affinity goes sticky; cache affinity was still opt-in; and the transient path
+was untouched. A headroom floor alone still evicts a live session from an 85%
+account to a 5% account, which discards a warm prefix for a capacity preference the
+session never had. Holding the binding is the primary rule; the headroom floor is
+what protects the operator who explicitly opts back out.
+
+## Write scope
+
+Permitted: `src/codex/routing.ts`, `src/types/config.ts`, `src/config.ts`, the
+account-pool and session-affinity code, their tests under
+`tests/codex-integration/`, `docs-site/` configuration reference and its locales,
+`structure/` docs that own the affected invariants, and this unit.
+
+Excluded, owned by concurrent lanes: `src/providers/devin*`,
+`src/providers/antigravity*`, `src/server/responses/*`, `src/codex/catalog/*`,
+`src/adapters/cursor/*`, `gui/`.
+
+## Verification posture
+
+Local suite, typecheck, install and GUI build are **not run** for this unit by
+explicit instruction. Proof is hosted CI at the exact final head SHA and nothing
+else. Pull requests state that posture in their Verification section rather than
+implying a local green. Pushes use `--no-verify`.
+
+## Acceptance criteria
+
+1. With no `pool` key configured, a bound thread in the 80–99% band keeps its
+   account across repeated resolves, and the preview path agrees with resolve.
+2. `pool.cacheAffinity: false` restores the historical eviction rule, and under it
+   a bound thread still refuses to move to a destination without headroom.
+3. A transient failure streak routes the current request away from the account
+   without deleting the binding; once the streak clears the thread is served by
+   its original account again.
+4. Quota refusal, credential invalidation, generation bumps, pause, and TTL expiry
+   still release a binding, with their existing tests unchanged.
+5. Every live-binding move records a reason that names which of those causes fired.
+6. Hosted CI is green at the exact final head of each delivery branch.
+
+## What would make this fail
+
+Shipping the default flip without finding every test that encodes the old default,
+and calling a red CI run a flake. The blast radius is enumerated in `010`; it is
+not guesswork, and a surprised assertion is evidence the rule is wrong somewhere,
+not that the test is stale.

devlog/_plan/260914_cost_guard_stabilization/010_bound_binding_policy.md

@@ -0,0 +1,103 @@
+# 010 — wp2: a live binding is held for cache, not re-scored for capacity
+
+## Today
+
+`resolveCodexAccountForThreadDetailed` reuses a live binding, then calls
+`reevaluateAffinityQuota`. Under the `quota` strategy that helper scores the bound
+account and asks `mayRebindAffinityForQuota` (`src/codex/routing.ts:2235`), whose
+answer without `pool.cacheAffinity` is `usage >= autoSwitchThreshold`. When true it
+takes `pickLowerUsageAccount`, which returns any **strictly cooler** eligible
+account. `previewReusableAffinityAccount` (`:2207`) carries a second copy of the
+same rule and the suite asserts the two answer identically.
+
+Two independent defects fall out of that, and they need different fixes.
+
+**The threshold is the wrong question for a bound thread.** Crossing 80% says the
+account is getting busy. It does not say the account cannot serve this turn, and
+the cost of acting on it is the whole warmed prefix. `round-robin` and
+`fill-first` already keep bound threads sticky — rotation there is new-session-only
+by design. `quota` is the outlier.
+
+**Nothing constrains the destination.** With every account in the 80–99% band the
+coolest is still hot, so the thread is handed on again next turn. `mayRebind` is
+also the short circuit for the 60-second re-score interval, so in that band the
+thread is re-scored on every request.
+
+## The rules
+
+**R1 — cache-first is the default.** `pool.cacheAffinity` resolves to `true` when
+unset. A bound thread leaves only when its account genuinely cannot serve:
+unusable, paused, credential-invalid, generation-stale, TTL-expired, quota-refused,
+or known to be at 100%. An explicit `pool.cacheAffinity: false` restores the
+historical rule for operators who want capacity-first behaviour.
+
+**R2 — a move needs somewhere worth moving to.** Even under R1-off, a bound thread
+may only move to an account that has genuine quota headroom, the same bar
+`resetFirstAffinityReplacement` already applies for `reset-first` through
+`hasCodexQuotaHeadroom`. Headroom alone is not sufficient, because that predicate
+deliberately answers true for an account whose usage is **unknown** —
+unknown-means-selectable is right for an unbound request and wrong for a bound
+one, since trading a warm prefix for an unmeasured account is a guess. The
+candidate must clear both bars: headroom, and strictly lower usage than the bound
+account. `CODEX_UNKNOWN_USAGE_SCORE` is 101, so an unobserved account can never be
+strictly cooler than a known over-threshold score and the second bar excludes it
+without a special case.
+
+R2 is what makes the incident impossible for both settings of the flag. R1 is what
+makes the expensive case impossible without the operator having to know the flag
+exists.
+
+## Why the default flip is the right call and not just a preference
+
+Every comparable system reaches the same place. Upstream Codex has no pool at all:
+it pins the cache with a session-scoped `prompt_cache_key` and a turn-sticky
+`x-codex-turn-state` token that retries must replay, and its transport sets
+`retry_429: false` so a rate-limit answer is classified before anything moves.
+Claude Code treats the cache as the retry policy — a `Retry-After` under twenty
+seconds waits on the **same** model rather than switching. OpenClaw, which is the
+closest analogue because it does pool credentials, auto-pins an auth profile per
+session and rotates only on long-window limits, keeping same-key retry separate
+from rotation. Published proxy guidance for pooled ChatGPT accounts says the same
+thing in one line: pool for quota, pin the session, and do not expect a prefix
+warmed on one account to exist on another.
+
+The asymmetry that makes this safe: a session pinned to a busy account pays
+latency. A session moved off a warm account pays the entire prefix again, every
+turn, and the pool has no way to move the cache with it.
+
+## Where it changes
+
+- `mayRebindAffinityForQuota` — the flag read becomes `?? true`, expressed through
+  one resolver so the default lives in exactly one place.
+- `reevaluateAffinityQuota` and `previewReusableAffinityAccount` — both gain the R2
+  destination filter, together, because the suite pins them to agree.
+- `resetFirstAffinityReplacement` — already applies R2; it now shares the helper
+  instead of open-coding it.
+
+Release paths are deliberately untouched. `hasUnrecoveredCodexQuotaRefusal`
+(429/402) still outranks every affinity preference, generation checks still defeat
+a late-arriving failure from an account the thread already left, and an exhausted
+or unusable account still loses the binding. This narrows a *preference*; it never
+weakens a refusal.
+
+## Blast radius
+
+The default flip inverts tests that encode the old default. They are not stale —
+each one pinned real behaviour — so each is rewritten to state its intent
+explicitly with `pool: { cacheAffinity: false }`, and a default-on counterpart is
+added next to it. The enumeration is mechanical and complete before the edit; see
+`.tmp/research/a6-test-blast-radius.md` for the working list. The near-misses
+matter as much as the hits: unbound rotation, 429 refusal, cooldown, pause,
+failover streak and TTL tests must all keep passing untouched, and any of them
+changing is a signal the edit went too far.
+
+## Regression tests
+
+1. No `pool` key, bound thread, account crosses 80% while a 5% sibling exists: the
+   thread keeps its account across repeated resolves, and preview agrees.
+2. `pool.cacheAffinity: false`, same setup: the thread moves once, then stays.
+3. `pool.cacheAffinity: false`, every account in the 80–99% band: the thread does
+   not move at all, and does not move on any subsequent turn. This is the reported
+   ping-pong and it fails before R2.
+4. Known 100% usage on the bound account with a cool sibling: the thread still
+   leaves under both settings.

devlog/_plan/260914_cost_guard_stabilization/020_backoff_preserves_binding.md

@@ -0,0 +1,79 @@
+# 020 — wp3: a transient streak is a detour, not an eviction
+
+## Today
+
+`recordCodexUpstreamOutcome` handles a transient (non-429/402) failure by counting
+consecutive failures and, once `upstreamFailoverThreshold` (default 3) trips,
+doing three things: it writes an escalating `softAvoidUntil`, it deletes **this**
+thread's pin with `deleteThreadAffinitiesForAccount`, and then it clears **every**
+thread pinned to that account with `clearThreadAccountMapForAccount`
+(`src/codex/routing.ts:3009-3016`).
+
+The resolve path enforces the same conclusion independently: `failoverReady` is one
+of the gates that fails the reuse branch, so the next continue deletes the binding
+at `:2522` even if the recorder had left it alone. The preview path carries the
+same gates at `:2176-2189`. Fixing only the recorder would be a no-op.
+
+This is the hole that survives `pool.cacheAffinity`. The flag governs the quota
+preference and nothing else, so three 503s — a provider-wide overload that has
+nothing to do with this account — discard the binding and the warmed prefix
+exactly as an 80% threshold crossing used to. #4269 already showed how badly this
+misfires: a retryable 503 whose human-readable message happened to contain
+"reauthentication" was classified as an auth error. A failure's blast radius must
+come from its scope, not from its text or its count.
+
+## The rule
+
+Being temporarily unable to send is not the same as giving up ownership of the
+conversation. Separate the two:
+
+| Account state | This request | The binding |
+| --- | --- | --- |
+| Healthy | served by the bound account | held |
+| Transient streak / soft-avoid | served by an alternate | **held** |
+| Hard cooldown from quota refusal (429/402) | served by an alternate | released |
+| Unusable, paused, credential-invalid, generation-stale | released | released |
+| Known 100% usage | served by an alternate | released |
+
+The middle row is the change. The request detours; the thread keeps its home.
+When the streak clears — and the existing `preservedCooldownFields` design means
+`lastFailureStatus` survives exactly until the account serves again — the thread is
+served by its own warm account with no further action.
+
+## The bound on the hold
+
+A hold with no expiry is a different bug: an account that never recovers would keep
+a thread detouring forever while the real conversational cache accumulates
+somewhere else. The hold is therefore bounded. The affinity entry records when the
+detour started; if the bound account is still unusable when that window lapses, the
+binding is released normally and the thread rebinds through the ordinary path. A
+successful serve clears the marker.
+
+This keeps the failure modes ordered correctly: a blip costs nothing, a sustained
+outage converges to a real rebind, and neither one is decided by a message string.
+
+## Where it changes
+
+- `recordCodexUpstreamOutcome` transient branch — the two affinity clears become
+  conditional on the release policy rather than unconditional on the streak.
+- `resolveCodexAccountForThreadDetailed` — a reuse that fails **only** on
+  transient evidence takes the detour branch instead of the delete branch.
+- `previewReusableAffinityAccount` — same classification, so preview keeps agreeing
+  with resolve.
+
+The existing race guard stays exactly as it is: a late failure arriving from
+account A must never disturb a binding that has already moved to B, which is what
+the generation check and the pinned-account guard in
+`deleteThreadAffinitiesForAccount` exist for. Nothing here relaxes them.
+
+## Regression tests
+
+1. Three transient 5xx failures on the bound account: the next resolve returns a
+   different account **and** the binding still names the original.
+2. The account then serves successfully: the following resolve returns the original
+   account again.
+3. The streak persists past the hold window: the binding is released and the thread
+   rebinds to the account that can serve.
+4. A 429 on the bound account still releases the binding immediately, unchanged.
+5. A late transient failure from an account the thread already left does not touch
+   the current binding.

devlog/_plan/260914_cost_guard_stabilization/030_move_reason_evidence.md

@@ -0,0 +1,36 @@
+# 030 — wp3: every move says why
+
+## Today
+
+There is no account-move metric and no persisted move reason. `logCtx.affinity` is
+typed as `reused | new_bind | rebound | cleared` but never assigned, and
+`appendUsageEntry` would drop it. `src/codex/affinity-debug.ts` is an opt-in
+HMAC-tagged header diagnostic for account-switch **compatibility** failures, not a
+record of routing decisions. The only way to infer a move today is to read account
+labels across log lines, which is how #4546 had to be diagnosed in the first place.
+
+Cache accounting has a related gap. Missing cache information is correctly omitted
+rather than stored as zero on `OcxUsage`, and `cacheHitRate` is `null` when
+unobserved — but the bridged Responses, Chat and Anthropic paths always emit
+`cached_tokens: 0`, and Kiro always writes 0. A reader cannot distinguish "the
+provider reported no cache hit" from "the provider reported nothing", which is
+precisely the distinction needed to tell whether a routing change worked.
+
+## The rule
+
+A live-binding move is a decision the operator paid for, so it carries its reason:
+which cause fired (`soft-quota`, `quota-refusal`, `exhausted`, `transient-hold-expired`,
+`unusable`, `paused`, `generation`, `expired`, `detour`), and whether the binding
+was held or released. The reason rides the existing per-attempt record in
+`usage.jsonl` — the one surface that already has attempt granularity — so the GUI
+Logs attempt view and `ocx logs explain` can render it without a new store.
+
+Missing cache information stays `unknown`. A synthesized `cached_tokens: 0` on a
+bridged path is a reporting artifact and must not aggregate as a measured miss.
+
+## Scope for this unit
+
+wp3 lands the reason at the decision point and the record, because that is what
+makes the wp2 and wp3 rules auditable in the field rather than only in tests. The
+dashboard rendering and the amplification metric (sends per logical request) belong
+with wp4, where the send budget gives them a denominator that means something.