Skip to content

Updated model fallback pages #1315

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
amanda-vapi merged 1 commit into from
Oct 13, 2026
Merged

Updated model fallback pages #1315

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 43 additions & 36 deletions fern/customization/model-fallback-plan.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,36 +1,44 @@
---
title: Model fallback configuration
subtitle: Configure a backup language model that activates automatically if your primary model fails
description: Configure fallbackModels in Vapi so a backup language model takes over automatically when your primary LLM fails. Covers which providers currently support model fallback and how it differs from voice and transcriber fallback.
description: Configure fallbackModels for OpenAI and Anthropic Bedrock models in the Dashboard or API, and compare model fallback with voice and transcriber fallback.
slug: customization/model-fallback-plan
---

## Overview

Model fallback lets a backup language model take over automatically if your primary model fails, so the call keeps going instead of ending in an error.

<Note>
Model fallback is currently supported for the [OpenAI](/providers/model/openai) and [Anthropic Bedrock](/providers/model/anthropic-bedrock) providers only. It is not yet a cross-provider fallback like [voice fallback](/voice-fallback-plan) or [transcriber fallback](/customization/transcriber-fallback-plan), which can fall back to a different provider entirely. Without `fallbackModels` configured, your call ends with an error if your chosen model provider fails.
</Note>
Model fallback lets a backup language model take over automatically if your primary model fails, so the call can keep going. You can configure fallback models for the [OpenAI](/providers/model/openai) and [Anthropic Bedrock](/providers/model/anthropic-bedrock) providers. Vapi models fall back automatically, so you don't configure fallback models for them.

## How it works

When a model failure occurs, Vapi will:

1. Detect the failure of the primary model.
2. If `fallbackModels` is configured, switch to the first fallback model in the list.
3. For OpenAI, continue through the rest of your specified list if subsequent failures occur. For Anthropic Bedrock, only one fallback model can be configured, so Vapi tries that single model.
4. Terminate the call only if every configured fallback has also failed.
2. If `fallbackModels` is configured, choose a fallback model.

Fallback models stay on the same provider as your primary model. To protect against a provider-wide outage, pair model fallback with [voice fallback](/voice-fallback-plan) and [transcriber fallback](/customization/transcriber-fallback-plan) on the other two pipeline components, which do support cross-provider fallback.

## Configure via Dashboard

You can configure a fallback model for Anthropic Bedrock in the Dashboard. To configure OpenAI fallback models, use the [API](#configure-via-api).

<Steps>
<Step title="Open the assistant">
Open the [Dashboard](https://dashboard.vapi.ai/). Select **Assistants**, then select an assistant.
</Step>
<Step title="Open the model settings">
Select the **Model** card, then choose an Anthropic Bedrock model.
</Step>
<Step title="Add a fallback model">
Expand **Advanced**. Under **Fallback model**, select **Add fallback model**, then choose a model.
</Step>
</Steps>

## Configure via API

Add `fallbackModels` directly to your assistant's `model` object. There is no separate `fallbackPlan` wrapper for models — unlike `voice.fallbackPlan.voices` and `transcriber.fallbackPlan.transcribers`, `fallbackModels` is a sibling property of `model.model` itself.
Add the `fallbackModels` field directly to your assistant's `model` object, next to `model.model`. Models don't use a `fallbackPlan` wrapper like `voice.fallbackPlan.voices` and `transcriber.fallbackPlan.transcribers`.

### OpenAI

Set `model.fallbackModels` to an ordered list of OpenAI model IDs. Vapi tries them in order if the primary model fails.
Set `model.fallbackModels` to an array of OpenAI model IDs. If the primary model fails, these models are tried, starting with the one expected to respond fastest. Configured fallback models might not be enabled for every account. See [`fallbackModels` for OpenAI](/api-reference/assistants/create#request.body.model.OpenAI.fallbackModels) in the API reference.

```json
{
Expand All @@ -43,55 +51,54 @@ Set `model.fallbackModels` to an ordered list of OpenAI model IDs. Vapi tries th
```

<Note>
You shouldn't need to set this unless you have a specific reason to — Vapi automatically finds the fastest fallback models that make sense for your primary model.
Most assistants don't need `fallbackModels`. The fastest fallback models that suit your primary model are found automatically.
</Note>

### Anthropic Bedrock

Set `model.fallbackModels` to at most one same-provider Claude model ID available through Bedrock.
Set `model.fallbackModels` to an array with one Claude model ID available through Bedrock. Bedrock accepts at most one fallback model. See [`fallbackModels` for Anthropic Bedrock](/api-reference/assistants/create#request.body.model.AnthropicBedrock.fallbackModels) in the API reference.

```json
{
"model": {
"provider": "anthropic-bedrock",
"model": "claude-sonnet-4-6",
"fallbackModels": "claude-haiku-4-5-20251001"
"fallbackModels": ["claude-haiku-4-5-20251001"]
}
}
```

<Note>
The fallback model is resolved using the call's Bedrock credential region (or `ANTHROPIC_BEDROCK_AWS_REGION`). A fallback name with no inference profile in that region is skipped and a warning is logged — Vapi never silently remaps it to a US or global profile. On Vapi EU, a fallback name without an EU inference profile is rejected at write time. `fallbackModels` cannot be combined with `thinking` in this release.
</Note>

## Other providers
## Verify the configuration

Vapi's other language model providers — including [Anthropic](/providers/model/anthropic) (non-Bedrock), [Google](/providers/model/gemini), [Groq](/providers/model/groq), [xAI](/providers/model/xai), [Mistral](/providers/model/mistral), [MiniMax](/providers/model/minimax), and [Custom LLM](/customization/custom-llm/fine-tuned-openai-models) — do not currently expose a `fallbackModels` field. If your primary model is on one of these providers and it fails, the call ends with an error unless you handle retries at the application layer.
Retrieve the assistant with [Get Assistant](/api-reference/assistants/get) and confirm that `model.fallbackModels` lists your fallback models.

## FAQ

<AccordionGroup>
<Accordion title="Can I fall back from one provider to a different provider, like voice and transcriber fallback do?">
Not yet. `fallbackModels` only lets you specify backup models from the same provider as your primary model. Cross-provider model fallback isn't currently supported — use [voice fallback](/voice-fallback-plan) and [transcriber fallback](/customization/transcriber-fallback-plan) to protect those two pipeline components against a provider-wide outage.
Not yet. `fallbackModels` only lets you specify backup models from the same provider as your primary model. Use [voice fallback](/voice-fallback-plan) and [transcriber fallback](/customization/transcriber-fallback-plan) to protect those two pipeline components against a provider-wide outage.
</Accordion>
<Accordion title="Does fallback affect pricing?">
You're billed normally for whichever model actually generates the response, whether that's your primary model or a fallback.
Each model is billed at its normal rate. If the primary model uses tokens before it fails, that usage can be included in the call's cost along with the fallback model's usage.
</Accordion>
<Accordion title="Why am I not seeing a fallbackModels option for my provider?">
`fallbackModels` is currently only available on `OpenAIModel` and `AnthropicBedrockModel`. It isn't exposed for other providers yet.
`fallbackModels` is currently only available on `OpenAIModel` and `AnthropicBedrockModel`.
</Accordion>
<Accordion title="Can I use model fallback with GPT-Live?">
No. GPT-Live doesn't support model fallbacks, and a GPT-Live model can't be a fallback model. See [GPT-Live configuration](/gpt-live/configuration).
</Accordion>
</AccordionGroup>

## Related

#### [Voice fallback configuration](/voice-fallback-plan)

Configure fallback voices that activate automatically when your primary voice provider fails.

#### [Transcriber fallback configuration](/customization/transcriber-fallback-plan)

Configure fallback transcribers so calls keep working if your primary speech-to-text provider fails.

#### [Model providers](/providers/model/overview)

Compare every language model provider Vapi supports.
<CardGroup cols={3}>
<Card title="Voice fallback configuration" href="/voice-fallback-plan">
Configure fallback voices that activate automatically when your primary voice provider fails.
</Card>
<Card title="Transcriber fallback configuration" href="/customization/transcriber-fallback-plan">
Configure fallback transcribers so calls keep working if your primary speech-to-text provider fails.
</Card>
<Card title="Model providers" href="/providers/model/overview">
Compare every language model provider Vapi supports.
</Card>
</CardGroup>
1 change: 0 additions & 1 deletion fern/docs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@

# yaml-language-server: $schema=https://schema.buildwithfern.dev/docs-yml.json

instances:
Expand Down
2 changes: 1 addition & 1 deletion fern/observability/monitoring-quickstart.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -523,7 +523,7 @@ curl -X POST "https://api.vapi.ai/monitoring/issue/e5f6a7b8-c9d0-1234-efab-56789
"title": "Add a fallback model",
"description": "Configure a faster fallback model (e.g., gpt-4o-mini) that activates when the primary model times out.",
"priority": "medium",
"docUrl": "/assistants/model-fallbacks"
"docUrl": "/customization/model-fallback-plan"
},
{
"title": "Enable retry logic",
Expand Down
12 changes: 6 additions & 6 deletions fern/voice-fallback-plan.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,14 @@
title: Voice fallback configuration
subtitle: Choose backup voices and troubleshoot failed fallback attempts
slug: voice-fallback-plan
description: Configure an ordered voice fallback plan, test backup voices, and troubleshoot calls that end before fallback activates.
description: Configure an ordered voice fallback plan in the Dashboard or API, test backup voices, and troubleshoot calls that end before fallback activates.
---

## Overview

Voice fallback lets you configure an ordered list of backup voices. When a voice error triggers fallback, the assistant tries the next voice in your plan so the call can continue.

Built-in fallback behavior without a configured plan depends on the selected voice. Configure `fallbackPlan.voices` to choose the backup voices and their order.
Built-in fallback behavior without a configured plan depends on the selected voice. Configure `voice.fallbackPlan.voices` to choose the backup voices and their order.

<Note>
A fallback plan does not guarantee recovery from every voice error or provider outage. Some failures can end a call before a backup voice starts.
Expand All @@ -19,7 +19,7 @@ Built-in fallback behavior without a configured plan depends on the selected voi

When a voice error triggers fallback, the assistant:

1. Tries the first voice in `fallbackPlan.voices`.
1. Tries the first voice in `voice.fallbackPlan.voices`.
2. Tries the next voice if a backup also fails and triggers fallback.
3. May end the call if no voice succeeds.

Expand All @@ -30,7 +30,7 @@ When a voice error triggers fallback, the assistant:
Open the [Dashboard](https://dashboard.vapi.ai/). Select **Assistants**, then select an assistant.
</Step>
<Step title="Open the fallback settings">
Select the **Advanced** tab. In the **Fallbacks** section, locate **Fallback Voices**.
Select the **Voice** card. Expand **Advanced**, then locate **Fallback Voices**.
</Step>
<Step title="Add a fallback voice">
Select **Add**, then choose a **Provider** and **Voice**. To enter a voice ID instead, select **Add Voice ID Manually**.
Expand Down Expand Up @@ -89,7 +89,7 @@ If a call ends instead of switching to a backup voice:
Confirm that the assistant or call configuration used for the call includes `voice.fallbackPlan.voices`. Check the order of the backup voices and verify each voice ID, model, and required provider credentials.
</Step>
<Step title="Test the backup as the primary voice">
Make a test call with a known working backup voice as the primary voice. If it produces audio, you can use that provider as the primary voice for new calls while you investigate why the original provider did not switch.
Make a test call with a known working backup voice as the primary voice. If it produces audio, use that provider as the primary voice for new calls. Then investigate why the original provider did not switch.
</Step>
<Step title="Share a failing call with support">
If the test call still fails or fallback does not start, share the call ID, `endedReason`, and primary and backup providers with [Vapi support](/support).
Expand Down Expand Up @@ -194,7 +194,7 @@ Each voice provider supports different configuration options. Expand the accordi

<AccordionGroup>
<Accordion title="How will pricing work?">
There is no change to the pricing of the voices. Your call will not incur any extra fees while using fallback voices, and you will be able to see the cost for each voice in your end-of-call report.
There is no change to the pricing of the voices. Your call will not incur any extra fees while using fallback voices, and you will be able to see the cost for each voice in your end-of-call report. To review a call's costs, see [Call Cost](/observability/logs/call-logs#call-cost) in call logs.
</Accordion>
<Accordion title="How many fallback voices can I configure?">
You can configure as many fallback voices as you need. However, we recommend 2-3 fallbacks from different providers for optimal reliability.
Expand Down
Loading