Skip to content

Commit 2c18452

Browse files
dvsthdvsth
committed
docs: add GPT-Live architecture and implementation guides
1 parent 6318126 commit 2c18452

10 files changed

Lines changed: 893 additions & 0 deletions

File tree

‎fern/docs.yml‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -123,6 +123,25 @@ navigation:
123123
icon: fa-light fa-terminal
124124
path: cli/overview.mdx
125125

126+
- section: GPT-Live
127+
contents:
128+
- page: Architecture
129+
path: gpt-live/overview.mdx
130+
- page: Quickstart
131+
path: gpt-live/quickstart.mdx
132+
- page: Conversation design
133+
path: gpt-live/conversation-design.mdx
134+
- page: Speaker and reasoner
135+
path: gpt-live/prompts.mdx
136+
- page: Tools and business logic
137+
path: gpt-live/tools.mdx
138+
- page: Voices and languages
139+
path: gpt-live/voices.mdx
140+
- page: Configuration and compatibility
141+
path: gpt-live/configuration.mdx
142+
- page: Migration and testing
143+
path: gpt-live/testing.mdx
144+
126145
- section: Assistants
127146
collapsed: open-by-default
128147
contents:
@@ -733,6 +752,8 @@ navigation:
733752
collapsed: true
734753
icon: fa-light fa-link
735754
contents:
755+
- page: GPT-Live
756+
path: providers/gpt-live.mdx
736757
- section: Transcribers (Speech-to-text)
737758
path: providers/transcriber/overview.mdx
738759
icon: fa-light fa-microphone

‎fern/gpt-live/configuration.mdx‎

Lines changed: 145 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,145 @@
1+
---
2+
title: GPT-Live configuration and compatibility reference
3+
subtitle: Look up supported fields, transports, and feature restrictions
4+
description: Look up GPT-Live model settings, defaults, voice configuration, call transports, tools, greetings, recording behavior, and unsupported Vapi features.
5+
slug: gpt-live/configuration
6+
---
7+
8+
Configure GPT-Live through a Vapi assistant with `model.provider: "openai"`, `model.model: "gpt-live-1"`, and an OpenAI voice. The tables below describe this integration's supported behavior.
9+
10+
## Model and voice fields
11+
12+
These fields belong to the assistant configuration. See [Create Assistant](/api-reference/assistants/create) for the complete generated schema.
13+
14+
| Field | Values or behavior |
15+
| --- | --- |
16+
| `model.provider` | `openai` |
17+
| `model.model` | `gpt-live-1` |
18+
| `model.speaker.instructions` | Speaker prompt; see [instruction precedence](/gpt-live/prompts#understand-instruction-precedence). |
19+
| `model.speaker.personalityPacks` | Array of `eager-listener`, `idle-hummer`, `bouncy`, and `unhurried`. |
20+
| `model.reasoner.provider` | `openai`; defaults to OpenAI when omitted. |
21+
| `model.reasoner.model` | `gpt-5.6-sol`, `gpt-5.6-terra`, or `gpt-5.6-luna`; defaults to `gpt-5.6-terra`. |
22+
| `model.reasoner.reasoningEffort` | `none`, `low`, `medium`, `high`, `xhigh`, or `max`; defaults to `low`. |
23+
| `model.reasoner.instructions` | Custom reasoner prompt; omission uses Vapi's default. |
24+
| `model.tools` | Inline supported tool definitions. |
25+
| `model.toolIds` | IDs of saved tools to resolve for the assistant. |
26+
| `voice.provider` | `openai` |
27+
| `voice.voiceId` | One of the [22 supported voices](/gpt-live/voices). |
28+
29+
A custom prompt replaces the corresponding fallback prompt. An explicit empty string is preserved. See [Configure the speaker and reasoner](/gpt-live/prompts) before overriding defaults.
30+
31+
## Greetings and duration
32+
33+
Use a text greeting if the assistant should begin the conversation.
34+
35+
| Field or feature | Behavior |
36+
| --- | --- |
37+
| `firstMessageMode: "assistant-speaks-first"` | Speaks the configured text greeting. |
38+
| `firstMessageMode: "assistant-waits-for-user"` | Waits for the caller. |
39+
| `firstMessage` | Text greeting; without usable text, the assistant waits. |
40+
| `maxDurationSeconds` | Maximum call duration in seconds. |
41+
| Audio URL greeting | Unsupported; not played as a greeting. |
42+
| Generated-message first-message mode | Unsupported; the assistant waits for the caller. |
43+
44+
Generated speech may vary from the supplied greeting. A text greeting is not a guarantee of exact prerecorded playback.
45+
46+
## Call transports
47+
48+
Use the [Create Call endpoint](/api-reference/calls/create) or the dashboard's web-call interface.
49+
50+
| Transport | Support | Requirements |
51+
| --- | --- | --- |
52+
| WebRTC web call | Supported | Dashboard **Talk** or Vapi web-call interface. |
53+
| `vapi.websocket` | Supported with restrictions | Raw, mono PCM16 audio at 24 kHz. |
54+
| `vapi.sip` | Supported | Vapi SIP transport configuration. |
55+
| Native Twilio | Supported | A configured Twilio phone number and call transport. |
56+
| Native Telnyx or Vonage | Unsupported | Choose a supported transport. |
57+
58+
### WebSocket configuration
59+
60+
Send this body to `POST https://api.vapi.ai/call` with your Vapi private API key:
61+
62+
```json
63+
{
64+
"assistantId": "YOUR_ASSISTANT_ID",
65+
"transport": {
66+
"provider": "vapi.websocket",
67+
"audioFormat": {
68+
"format": "pcm_s16le",
69+
"container": "raw",
70+
"sampleRate": 24000
71+
}
72+
}
73+
}
74+
```
75+
76+
Connect to `transport.websocketCallUrl` in the response. Send and receive binary audio frames containing signed 16-bit little-endian samples at 24 kHz. Convert your microphone audio to that format before sending it.
77+
78+
The default WebSocket sample rate is not sufficient for GPT-Live; set `24000` explicitly. This audio requirement applies to the Vapi WebSocket transport, not to browser-negotiated WebRTC audio or carrier audio.
79+
80+
Close the WebSocket to end the call, or have the assistant invoke `endCall`. Incoming JSON call-control commands are not supported by this GPT-Live connection. See [WebSocket transport](/calls/websocket-transport) for the general transport, applying the restrictions above.
81+
82+
## Tool and transfer compatibility
83+
84+
The supported tool types are `function`, `apiRequest`, `endCall`, `dtmf`, and `transferCall`. All other tool types are rejected.
85+
86+
| Feature | Support or restriction |
87+
| --- | --- |
88+
| Function tools | Synchronous and asynchronous functions. |
89+
| API request tools | Configured HTTP requests. |
90+
| Outgoing DTMF | Supported on SIP using RTP DTMF. Use digits as a string, such as `"0011*#"`; native Twilio, web calls, and raw WebSocket calls do not support this tool. |
91+
| SIP INFO DTMF | Unsupported; use RTP DTMF on SIP. |
92+
| Incoming keypad collection | Rejected when `keypadInputPlan.enabled` is true. |
93+
| Blind transfer | Supported on native Twilio and Vapi SIP to configured phone or SIP destinations. |
94+
| Warm transfer or generated transfer summary | Rejected. |
95+
| Transfer fallback plans or SIP `bye` verb | Rejected. |
96+
| SIP transfer with extension dialing | Unsupported; use a direct destination. |
97+
| Squads and assistant handoffs | Unsupported. |
98+
99+
A configured transfer tool does not make every transport transfer-capable. Web calls and raw WebSocket calls do not gain phone transfer support by adding the tool. Carrier acceptance of a transfer does not confirm that the destination answered.
100+
101+
## Settings that do not apply
102+
103+
GPT-Live handles speech directly. Classic transcriber selection, text-to-speech controls, and start/stop-speaking plans do not configure this audio conversation.
104+
105+
General model fields such as `temperature` and `maxTokens` do not tune this integration. Use the speaker prompt for conversational behavior and `model.reasoner.reasoningEffort` for reasoning effort.
106+
107+
## Unsupported features and artifact behavior
108+
109+
Check these restrictions before moving an existing assistant:
110+
111+
| Feature | Behavior |
112+
| --- | --- |
113+
| Model fallbacks | Rejected. GPT-Live also cannot be a fallback model. |
114+
| Voice fallbacks | Rejected. |
115+
| Non-OpenAI or custom voice IDs | Rejected unless included in the supported Vapi voice catalog. |
116+
| Live message injection and call-control commands | Unsupported for this integration. |
117+
| Listen or monitor sockets | Unsupported. |
118+
| Reconnecting to an existing GPT-Live session | Unsupported. Start a new call and reconcile pending work in your service. |
119+
| Recording disabled in the assistant | No recording is created. |
120+
| Recording-consent plan configured | GPT-Live recording is disabled because this integration does not collect that consent. |
121+
122+
Call artifacts and analysis are separate from your service's authoritative operation records. A transcript cannot by itself establish that a booking succeeded. See [Migrate and test GPT-Live](/gpt-live/testing).
123+
124+
## Post-call analysis and observability
125+
126+
Completed GPT-Live calls use Vapi's shared post-call processing. Configure analysis and observability on the assistant and inspect the resulting call data after processing finishes.
127+
128+
| Capability | Configuration or result | Requirement |
129+
| --- | --- | --- |
130+
| Structured outputs | Attach IDs through `artifactPlan.structuredOutputIds`; read `call.artifact.structuredOutputs`. | Create the output definitions and retain the conversation data needed for extraction. |
131+
| Legacy call analysis | Configure `analysisPlan`. | Existing configurations remain supported; prefer structured outputs for new extraction and evaluation setups. |
132+
| Scorecards | Grade structured output values; read `call.artifact.scorecards`. | Configure a scorecard and its required structured outputs. |
133+
| Boards | Query call data and plot outcomes. | Choose metrics available on your calls; classic pipeline timing fields do not describe GPT-Live's architecture. |
134+
| Monitoring | Configure monitors, thresholds, and notifiers. | Retained call data is required; monitoring is unavailable with Zero Data Retention. |
135+
| End-of-call reports | Receive configured `end-of-call-report` server messages. | Configure the assistant's Server URL and requested server messages. |
136+
137+
See [Call analysis](/assistants/call-analysis), [Scorecards](/observability/scorecard-quickstart), and [Monitoring](/observability/monitoring-quickstart) for setup. These are post-call capabilities; they do not imply access to live listen sockets or message injection.
138+
139+
An extracted statement such as “booking confirmed” describes the conversation. Verify the booking in your service when measuring actual task success. Analysis results do not replace the authoritative operation record.
140+
141+
## Credentials and cost
142+
143+
If you bring your own OpenAI key, it needs access to GPT-Live and the selected reasoner. Organization access to the Vapi feature is a separate prerequisite.
144+
145+
Review Vapi call costs for your configuration. OpenAI's [model pricing](https://developers.openai.com/api/docs/models/gpt-live-1) describes direct provider charges; it is not a quote for your complete Vapi call. Reasoning, tools, and telephony can contribute additional cost.

‎fern/gpt-live/conversation-design.mdx‎

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
---
2+
title: Design GPT-Live conversations
3+
subtitle: Handle corrections, overlapping speech, and work in progress
4+
description: Design GPT-Live conversations around clarification, interruptions, pending tools, corrected requests, confirmed outcomes, and recoverable business actions.
5+
slug: gpt-live/conversation-design
6+
---
7+
8+
Design the conversation around the caller's current intent and the status of the task. The speaker can keep talking while work proceeds, so an acknowledgment must stay distinct from a confirmed result.
9+
10+
## Track conversation and action separately
11+
12+
Three events can happen at different times: the caller finishes speaking, the tool returns, and the caller hears the result. Treat each as evidence of its own outcome.
13+
14+
| What happens | What it establishes | What it does not establish |
15+
| --- | --- | --- |
16+
| Caller interrupts | New speech needs attention | A pending operation was cancelled |
17+
| Assistant says “I'll check” | It acknowledged the request | A lookup started or succeeded |
18+
| Tool reports available slots | The lookup returned availability | A slot is reserved |
19+
| Booking service confirms an operation | The booking succeeded | The caller heard the confirmation |
20+
| Connection closes | The conversation ended | External work was rolled back |
21+
22+
## Handle a correction during a lookup
23+
24+
Consider this exchange:
25+
26+
> **Caller:** What do you have on Thursday?
27+
> **Assistant:** I'll check.
28+
> **Caller:** Actually, Friday afternoon.
29+
30+
The corrected task is Friday afternoon. The speaker should acknowledge that change without reading out Thursday's results as the answer.
31+
32+
Your service should associate requests with the current task. For example, keep a task revision with the requested date and time zone. Before using a returned result or executing a booking, verify that it still matches the current request. This is an application design pattern, not a Vapi configuration field.
33+
34+
If the Thursday lookup cannot be cancelled, let it finish and mark its result as outdated. If an action already changed a record, report that state and use your service's change or cancellation procedure.
35+
36+
## Ask for the missing decision
37+
38+
Ask one short question that makes the next action possible. An appointment request may need a date, time zone, appointment type, or confirmation of a specific slot.
39+
40+
Prefer “Which time zone should I use?” to asking the caller to repeat the entire request. When a correction invalidates a previous confirmation, obtain a new confirmation for the revised action.
41+
42+
Short acknowledgments can keep a conversation understandable. Avoid instructions that require a filler phrase before every tool call; evaluate whether the assistant's timing is useful in real audio.
43+
44+
## Explain work in progress accurately
45+
46+
Use language that matches what the service knows:
47+
48+
| Task status | Example spoken response |
49+
| --- | --- |
50+
| Lookup pending | “I'm checking Friday afternoon.” |
51+
| Slots returned | “There are openings at two and four. Which works for you?” |
52+
| Booking pending | “I'm submitting the two o'clock appointment.” |
53+
| Booking confirmed | “Your two o'clock appointment is booked.” |
54+
| Outcome unknown | “I haven't confirmed whether that booking went through yet.” |
55+
| Cancellation confirmed | “The appointment has been cancelled.” |
56+
57+
The examples illustrate meaning, not guaranteed verbatim speech. Choose one useful update when work is slow instead of repeatedly announcing the same pending state.
58+
59+
## Treat cancellation as an operation
60+
61+
“Stop talking,” “stop checking,” and “cancel my appointment” have different effects. Ask a brief clarification when the intended action is unclear.
62+
63+
Put cancellation and duplicate prevention in the service that owns the action. A timeout or lost response can leave an operation's outcome unknown. Reconcile that outcome before retrying a write. Use an operation identifier so a retry cannot silently create a second booking.
64+
65+
## Choose where exact wording matters
66+
67+
GPT-Live generates speech and can paraphrase supplied text. Test language, pronunciation, and required phrases with audio; a matching text prompt does not establish verbatim delivery.
68+
69+
If your application requires an exact prerecorded opening or approval before any audio plays, check the [compatibility reference](/gpt-live/configuration) before choosing this integration. Do not assume that withholding a reasoner result pauses the speaker.
70+
71+
## Evaluate the whole exchange
72+
73+
Test corrections during speech, during a lookup, and immediately before a write. For each test, inspect the audio, tool results, and final record in your service.
74+
75+
OpenAI's [migration guide](https://developers.openai.com/api/docs/guides/live-migration) discusses the separation between interruption and backend cancellation. The [Vapi testing guide](/gpt-live/testing) turns these cases into a repeatable comparison.

0 commit comments

Comments
 (0)