Skip to content

Document GPT-Live Voice Simulation support #1246

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dev-seth-vapi merged 1 commit into from
Sep 22, 2026
Merged

Document GPT-Live Voice Simulation support #1246

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion fern/gpt-live/design.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -263,6 +263,6 @@ Review a short set of conversations before expanding the prompt. Include a direc

Ask whether the assistant used what the caller had already said, delegated at the right moment, and returned naturally to unfinished work. Listen for whether its pace helps the caller make progress.

Keep a few representative calls as a baseline. Repeat them when either prompt changes. Use [post-call analysis](/gpt-live/configuration#review-and-monitor-calls) to track outcomes, while reviewing audio for timing and delivery. Check [testing limitations](/gpt-live/limitations#can-i-run-simulations-and-evals) before choosing an automated test method.
Keep a few representative calls as a baseline. Repeat them when either prompt changes. Use [post-call analysis](/gpt-live/configuration#review-and-monitor-calls) to track outcomes, while reviewing audio for timing and delivery. Use [Voice Simulations](/gpt-live/limitations#can-i-run-simulations-and-evals) to repeat scenarios with GPT-Live as the tester, target, or both. Choose Voice mode and a single assistant target.

For additional model-level examples, see OpenAI's [GPT-Live prompting guide](https://developers.openai.com/api/docs/guides/live-prompting). Use Vapi's fields when applying that guidance to your assistant.
20 changes: 18 additions & 2 deletions fern/gpt-live/limitations.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,9 +92,25 @@ If the caller corrects a date during a lookup, use the result that matches the u

## Can I run Simulations and Evals?

Voice Simulations are currently unsupported for GPT-Live.
Yes. **Voice Simulations support GPT-Live** as the assistant under test, the AI tester, or both. You can also pair a GPT-Live participant with a classic voice assistant.

Run voice scenarios through a [supported call connection](/gpt-live/configuration#connect-a-call). [Evals](/observability/evals-quickstart) can supplement testing of supported model decisions, but they don't establish GPT-Live's audio quality, timing, or interruption behavior.
| Simulation setup | Support |
| --- | --- |
| GPT-Live target with a classic AI tester | Supported in Voice mode |
| GPT-Live AI tester with a classic assistant target | Supported in Voice mode |
| GPT-Live tester and target | Supported in Voice mode |
| Chat mode with either participant using GPT-Live | Unsupported; choose Voice |
| Squad runs with a GPT-Live participant | Unsupported; choose a single assistant target |

To test your assistant:

1. Follow the [Simulations quickstart](/observability/simulations-quickstart), selecting your GPT-Live assistant as the target.
2. Configure the scenario, AI tester personality, and success criteria. If you use a GPT-Live tester, select a supported native OpenAI voice.
3. Run the suite in **Voice** mode, then review its transcript, evaluations, and available recording.

Existing GPT-Live organization access requirements still apply. Simulations handle the participants' audio formats, including mixed GPT-Live and classic runs; you don't need to build an audio bridge yourself. [Tool mocks](/observability/simulations-advanced#mock-tool-responses) are supported for GPT-Live simulation tools. Unmocked tools can perform real actions, so configure mocks or sandbox services before running scenarios that change external data.

Voice Simulations let you exercise speech, timing, interruptions, and tool behavior. Listen to the recording as well as checking evaluations: a passing criterion doesn't establish every aspect of conversation quality. [Evals](/observability/evals-quickstart) remain useful for supported model decisions, but don't replace testing the voice interaction.

## Recordings and post-call data

Expand Down
1 change: 1 addition & 0 deletions fern/gpt-live/overview.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ You also have more to design than the words alone. Set a voice, give the speaker
| HTTP live call control: end call, speaker context, and cold transfer | [Control an active call](/gpt-live/configuration#live-call-control) |
| Speaker prompts, reasoner settings, and personality packs | [Model settings](/gpt-live/configuration#model-settings) |
| 22 preset voices, with audio previews | [Choose a voice](/gpt-live/configuration#voices) |
| Voice Simulations with GPT-Live testers and targets | [Test conversations](/gpt-live/limitations#can-i-run-simulations-and-evals) |
| Transcripts, available recordings, and post-call analysis | [Review calls](/gpt-live/configuration#review-and-monitor-calls) |
| Structured outputs, scorecards, Boards, and Monitoring | [Track quality](/gpt-live/configuration#review-and-monitor-calls) |

Expand Down
Loading