From 6fdf1a4da208eefe77cfad9458a991ae460d9ff0 Mon Sep 17 00:00:00 2001 From: dev-seth-vapi Date: Tue, 22 Sep 2026 14:17:26 -0700 Subject: [PATCH] docs: document GPT-Live Voice Simulation support --- fern/gpt-live/design.mdx | 2 +- fern/gpt-live/limitations.mdx | 20 ++++++++++++++++++-- fern/gpt-live/overview.mdx | 1 + 3 files changed, 20 insertions(+), 3 deletions(-) diff --git a/fern/gpt-live/design.mdx b/fern/gpt-live/design.mdx index e54e77061..7ef28ad81 100644 --- a/fern/gpt-live/design.mdx +++ b/fern/gpt-live/design.mdx @@ -263,6 +263,6 @@ Review a short set of conversations before expanding the prompt. Include a direc Ask whether the assistant used what the caller had already said, delegated at the right moment, and returned naturally to unfinished work. Listen for whether its pace helps the caller make progress. -Keep a few representative calls as a baseline. Repeat them when either prompt changes. Use [post-call analysis](/gpt-live/configuration#review-and-monitor-calls) to track outcomes, while reviewing audio for timing and delivery. Check [testing limitations](/gpt-live/limitations#can-i-run-simulations-and-evals) before choosing an automated test method. +Keep a few representative calls as a baseline. Repeat them when either prompt changes. Use [post-call analysis](/gpt-live/configuration#review-and-monitor-calls) to track outcomes, while reviewing audio for timing and delivery. Use [Voice Simulations](/gpt-live/limitations#can-i-run-simulations-and-evals) to repeat scenarios with GPT-Live as the tester, target, or both. Choose Voice mode and a single assistant target. For additional model-level examples, see OpenAI's [GPT-Live prompting guide](https://developers.openai.com/api/docs/guides/live-prompting). Use Vapi's fields when applying that guidance to your assistant. diff --git a/fern/gpt-live/limitations.mdx b/fern/gpt-live/limitations.mdx index 42c58dc77..45cf5f9fd 100644 --- a/fern/gpt-live/limitations.mdx +++ b/fern/gpt-live/limitations.mdx @@ -92,9 +92,25 @@ If the caller corrects a date during a lookup, use the result that matches the u ## Can I run Simulations and Evals? -Voice Simulations are currently unsupported for GPT-Live. +Yes. **Voice Simulations support GPT-Live** as the assistant under test, the AI tester, or both. You can also pair a GPT-Live participant with a classic voice assistant. -Run voice scenarios through a [supported call connection](/gpt-live/configuration#connect-a-call). [Evals](/observability/evals-quickstart) can supplement testing of supported model decisions, but they don't establish GPT-Live's audio quality, timing, or interruption behavior. +| Simulation setup | Support | +| --- | --- | +| GPT-Live target with a classic AI tester | Supported in Voice mode | +| GPT-Live AI tester with a classic assistant target | Supported in Voice mode | +| GPT-Live tester and target | Supported in Voice mode | +| Chat mode with either participant using GPT-Live | Unsupported; choose Voice | +| Squad runs with a GPT-Live participant | Unsupported; choose a single assistant target | + +To test your assistant: + +1. Follow the [Simulations quickstart](/observability/simulations-quickstart), selecting your GPT-Live assistant as the target. +2. Configure the scenario, AI tester personality, and success criteria. If you use a GPT-Live tester, select a supported native OpenAI voice. +3. Run the suite in **Voice** mode, then review its transcript, evaluations, and available recording. + +Existing GPT-Live organization access requirements still apply. Simulations handle the participants' audio formats, including mixed GPT-Live and classic runs; you don't need to build an audio bridge yourself. [Tool mocks](/observability/simulations-advanced#mock-tool-responses) are supported for GPT-Live simulation tools. Unmocked tools can perform real actions, so configure mocks or sandbox services before running scenarios that change external data. + +Voice Simulations let you exercise speech, timing, interruptions, and tool behavior. Listen to the recording as well as checking evaluations: a passing criterion doesn't establish every aspect of conversation quality. [Evals](/observability/evals-quickstart) remain useful for supported model decisions, but don't replace testing the voice interaction. ## Recordings and post-call data diff --git a/fern/gpt-live/overview.mdx b/fern/gpt-live/overview.mdx index d3c0f75b9..e50c74fbd 100644 --- a/fern/gpt-live/overview.mdx +++ b/fern/gpt-live/overview.mdx @@ -62,6 +62,7 @@ You also have more to design than the words alone. Set a voice, give the speaker | HTTP live call control: end call, speaker context, and cold transfer | [Control an active call](/gpt-live/configuration#live-call-control) | | Speaker prompts, reasoner settings, and personality packs | [Model settings](/gpt-live/configuration#model-settings) | | 22 preset voices, with audio previews | [Choose a voice](/gpt-live/configuration#voices) | +| Voice Simulations with GPT-Live testers and targets | [Test conversations](/gpt-live/limitations#can-i-run-simulations-and-evals) | | Transcripts, available recordings, and post-call analysis | [Review calls](/gpt-live/configuration#review-and-monitor-calls) | | Structured outputs, scorecards, Boards, and Monitoring | [Track quality](/gpt-live/configuration#review-and-monitor-calls) |