Skip to content

feat: advance the fallback chain after inference failures (#6) - #9

Merged
nabheet merged 2 commits into
mainfrom
feat/chain-advance-6
Oct 31, 2026
Merged

nabheet merged 2 commits into
mainfrom
feat/chain-advance-6

Conversation

@nabheet

@nabheet nabheet commented Oct 31, 2026

Copy link
Copy Markdown
Owner

Closes #6.

Problem

failover() picked with chain.find(...) over availability and cooldown only. The first chain entry stayed eligible forever, so every session.execution.failed moved the session to it again until maxAttempts ran out; later entries, including other providers, were never reached.

Fix

  • Per session, the plugin now remembers the refs already tried in this failure sequence and the ref the session is on (session.model.selected, session.step.started). Selection skips both.
  • A completed switch records the new ref immediately, rather than waiting for the matching model.selected event, which can arrive after the next failure.
  • attempts is consumed only when switchModel resolves. A rejected switch cools the model down and leaves the chain position and the budget alone, so the next failure picks the entry after it (previously the attempt was consumed before the switch).
  • A failed inference does not cool the model down. It advances this session's position only, so a sibling session can still use that model. Cooldown stays reserved for a failed switch, which is the separation the issue asked for.
  • session.execution.succeeded / .interrupted end the sequence, so the next failure starts at the top of the chain again; session.deleted drops the session's state.
  • event.data?.error — a malformed failed event no longer throws inside the loop, where the outer catch {} would silently end the subscription.

Tests

Five new cases: advance through the chain; never return to the model that just failed; fresh state after success; per-session isolation; a failed switch does not consume an attempt.

Proven against the old implementation rather than assumed:

Test Result with old index.js
advances to the next chain entry ✖
never returns to the model that just failed ✖
starts a fresh sequence after success ✖ (mutation check: removing the reset line fails it)
keeps attempt state per session ✔ (guards against over-resetting)
failed switch does not consume an attempt ✖

npm test → 13/13 pass. npm run lint → clean (biome + markdownlint).

Review

Adversarial pass found one blocker and fixed it: the selection originally added only refs the plugin had switched to, so with [one, two, three] and the session on one, the second failure returned to one. The failed model is now added to the tried set, and a regression test emits model.selected to reproduce exactly that path.

Also fixed from the review: event.data?.error, the README's "four methods" count, and the undocumented cooldown decision (now stated: an inference failure advances one session rather than freezing a model for everyone).

Scope

Only #6. The three per-session maps clear on success/interruption/deletion; they are not otherwise expired, so a long-lived server retains one small entry per live session, which matches how session state is expected to live. Not addressed here: busy is now redundant because the event loop is serial — left in place rather than widening the diff.

This repo publishes a patch on every push to main, so merging cuts a release.

`failover()` selected with `chain.find(...)` over availability and cooldown
only. The first chain entry stayed eligible forever, so every
`session.execution.failed` moved the session to it again until `maxAttempts`
ran out and later entries, including other providers, were never reached.

- Per session, remember the refs already tried in this failure sequence and
  the ref the session is on (`session.model.selected`, `session.step.started`);
  selection skips both.
- A completed switch records the new ref immediately instead of waiting for
  the matching `model.selected` event, which can arrive after the next failure.
- `attempts` is now consumed only when `switchModel` resolves. A rejected
  switch cools the model down and leaves the chain position and budget alone,
  so the next failure picks the entry after it.
- A failed inference does not cool the model down: it advances this session
  only, so a sibling session can still use the model. Cooldown stays reserved
  for a failed switch, per the issue's "keep them separate".
- A sequence ends on `session.execution.succeeded` / `.interrupted` (state
  reset, so the next failure starts at the top of the chain) and on
  `session.deleted` (state dropped).
- `event.data?.error`: a malformed failed event no longer throws inside the
  event loop where the outer catch would swallow the subscription.

Tests: five new cases (advance, no bounce-back to the failed model, reset after
success, per-session isolation, failed switch does not consume an attempt).
Three fail against the previous implementation, verified by reverting the
corresponding lines.
@nabheet
nabheet marked this pull request as ready for review October 31, 2026 04:03
@github-actions

Copy link
Copy Markdown

Beta prerelease published: 0.1.0-beta.22 — installable with npm i opencode-model-failover@beta.

@nabheet
nabheet merged commit cb8ef30 into main Oct 31, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Advance the fallback chain after inference failures

1 participant