Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions fern/calls/call-features.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,10 @@ curl 'https://api.vapi.ai/call'

```

<Note>
GPT-Live supports a subset of live call control: HTTP `end-call`, `append-context` for speaker commentary, thinking, or instructions, and cold `transfer` on supported phone connections. The `say`, `add-message`, and mute/unmute controls below aren't supported by GPT-Live. Use the [GPT-Live live call control guide](/gpt-live/configuration#live-call-control) for its request examples and limits.
</Note>

## Call Control Features

Once you have the `controlUrl`, you can use various control features during a live call. Here are all the available control options:
Expand Down
80 changes: 75 additions & 5 deletions fern/gpt-live/configuration.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Create an assistant that checks appointment availability and reads back the resu

For help deciding what to put in each prompt, read [Design your assistant](/gpt-live/design). For feature restrictions, use [Limitations and FAQs](/gpt-live/limitations).

**Jump to:** [Create an assistant](#create-an-assistant) · [Model settings](#model-settings) · [Personality](#personality-and-language) · [Voices](#voices) · [Tools](#connect-your-tools) · [Calls](#connect-a-call) · [Analysis](#review-and-monitor-calls)
**Jump to:** [Create an assistant](#create-an-assistant) · [Model settings](#model-settings) · [Personality](#personality-and-language) · [Voices](#voices) · [Tools](#connect-your-tools) · [Calls](#connect-a-call) · [Live control](#live-call-control) · [Analysis](#review-and-monitor-calls)

## Before you start

Expand Down Expand Up @@ -177,7 +177,7 @@ These fields belong to the assistant object. See [Create Assistant](/api-referen

## Prompt defaults and overrides

Set `model.speaker.instructions` for the spoken conversation and `model.reasoner.instructions` for task procedures.
Set `model.speaker.instructions` for the spoken conversation and `model.reasoner.instructions` for task procedures. See [where instructions belong](/gpt-live/design#where-should-an-instruction-go), the [paired prompt example](/gpt-live/design#example-split-a-scheduling-prompt), and [what context the reasoner receives](/gpt-live/design#what-context-does-the-reasoner-receive).

- Explicit speaker instructions take precedence over `model.systemPrompt` and system messages in `model.messages`. If you omit the speaker instructions, Vapi uses that legacy configuration.
- Omitting reasoner instructions uses Vapi's default reasoner prompt. Custom instructions replace that prompt entirely.
Expand Down Expand Up @@ -285,9 +285,25 @@ Return a result for each request using its actual `toolCallId`:

Replace the sample result with your service's data. If the lookup fails, return the failure so the assistant can explain it. See [Server authentication](/server-url/server-authentication) to protect the endpoint.

Supported tool types are `function`, `apiRequest`, `endCall`, `dtmf`, and `transferCall`. Give tools unique resolved function names. Save reusable tools and attach them through `model.toolIds`, or include definitions in `model.tools`.
Supported tool types are `function`, `apiRequest`, `mcp`, `endCall`, `dtmf`, and `transferCall`. Give tools unique resolved function names. Save reusable tools and attach them through `model.toolIds`, or include definitions in `model.tools`.

Use function or API request tools to connect external data sources. For phone actions, check [transfer and keypad support](/gpt-live/limitations#transfers-and-keypad-input).
Use function, API request, or MCP tools to connect external data sources. For phone actions, check [transfer and keypad support](/gpt-live/limitations#transfers-and-keypad-input).

### MCP tools

You can keep the MCP tools you already use with an existing Vapi assistant when migrating to GPT-Live. Attach the same saved tool IDs through `model.toolIds`, or keep your inline `type: "mcp"` definitions in `model.tools`. Existing server URLs, headers, credentials, and protocol settings are reused. See [MCP tools](/tools/mcp) for setup.

For example, add your saved MCP tool ID to the assistant's `model` configuration, preserving any other tool IDs:

```json
{
"toolIds": ["YOUR_EXISTING_MCP_TOOL_ID"]
}
```

At call setup, Vapi connects to the MCP server and discovers its tools. Those tools are available to the **reasoner**. The speaker delegates work to the reasoner, which calls the tool and returns the result for the conversation. Put tool-use procedures in `model.reasoner.instructions` and delegation guidance in `model.speaker.instructions`.

The discovered tool list stays fixed for that call. Each tool invocation opens a separate MCP connection; it doesn't rediscover the tool list. Start a new call to pick up changes to the server's available tools. GPT-Live doesn't play classic per-tool speech messages; use speaker instructions to guide how it talks while work runs.

### Slow and asynchronous requests

Expand Down Expand Up @@ -339,7 +355,61 @@ Connect to `transport.websocketCallUrl` in the response. Send and receive binary

Set `sampleRate` to `24000` for GPT-Live. Browser WebRTC and phone connections negotiate their own audio formats.

Close the WebSocket to end the call, or have the assistant invoke `endCall`. Incoming JSON call-control commands are not supported by this GPT-Live connection. See [WebSocket transport](/calls/websocket-transport) for the general transport, applying the restrictions above.
Close the WebSocket to end the call, have the assistant invoke `endCall`, or send an HTTP `end-call` request to the [live control URL](#live-call-control). Send control commands over HTTP, not as JSON frames on the audio WebSocket. See [WebSocket transport](/calls/websocket-transport) for the general transport, applying the restrictions above.

## Live call control

Your application can end an active GPT-Live call, append speaker context, or initiate a cold transfer. Send an HTTP POST to that call's `monitor.controlUrl`, returned in the call object. These commands aren't supported through audio WebSocket text frames or SDK data channels.

Set `CONTROL_URL` to the returned URL. Call control must be enabled in the assistant's `monitorPlan`: setting `controlEnabled` to `false` disables it. If `controlAuthenticationEnabled` is `true`, include `Authorization: Bearer YOUR_VAPI_PUBLIC_API_KEY` from the same organization on each request. The examples below assume control authentication is disabled. Keep the control URL private.

### End the call

```bash
curl --fail-with-body -X POST "$CONTROL_URL" \
-H "Content-Type: application/json" \
-d '{"type":"end-call"}'
```

### Append speaker context

Use `append-context` with a `kind` and nonblank `content` of up to 16,000 characters:

| Kind | Purpose | Example content |
| --- | --- | --- |
| `commentary` | Give the speaker information to convey in its own words | `Your appointment is confirmed for Tuesday at 2 PM.` |
| `thinking` | Add silent context for the speaker to use | `The caller has already verified their account.` |
| `instructions` | Guide the speaker's behavior or ask it to try saying something | `Tell the caller their order is ready, then ask whether they need anything else.` |

```bash
curl --fail-with-body -X POST "$CONTROL_URL" \
-H "Content-Type: application/json" \
-d '{"type":"append-context","kind":"instructions","content":"Tell the caller their order is ready, then ask whether they need anything else."}'
```

These messages go to the speaker. They don't add context to the reasoner's history or directly start reasoning or tool work. `thinking` doesn't request speech, but it isn't a separate private context store: the speaker can use that information in subsequent responses.

A successful append response means the context was submitted. It doesn't guarantee exact wording, immediate speech, or completed playback.

### Cold transfer

On native Twilio and Vapi SIP calls, send a phone or SIP destination. For example:

```bash
curl --fail-with-body -X POST "$CONTROL_URL" \
-H "Content-Type: application/json" \
-d '{"type":"transfer","destination":{"type":"number","number":"+14155550100","transferPlan":{"mode":"blind-transfer"}}}'
```

Replace the number with your transfer destination. For SIP, use a destination such as `{"type":"sip","sipUri":"sip:agent@example.com"}`. An application supplies the destination in this request; a reasoner using a `transferCall` tool selects from that tool's configured destinations.

Cold transfer isn't supported on browser or raw WebSocket calls. Don't include pre-transfer `content`, warm-transfer plans, generated summaries, or fallback plans. See [transfer restrictions](/gpt-live/limitations#transfers-and-keypad-input). Carrier acceptance doesn't establish that the destination answered.

### Responses and unsupported controls

Successful commands return HTTP `200` with `{"status":"ok"}`. Rejected requests return an error, including when the call isn't active or is already transferring. If a `502` response reports an unconfirmed outcome, don't automatically retry: the action may have reached the provider. A `503` response stating that no action was taken is safe to retry.

GPT-Live doesn't support `say`, `add-message`, or mute/unmute controls. Use `append-context` for speaker guidance. See [live control compatibility](/gpt-live/limitations#can-i-inject-commentary-or-control-speech-during-a-call).

## Review and monitor calls

Expand Down
82 changes: 80 additions & 2 deletions fern/gpt-live/design.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ The design task is to make those moments feel connected. Decide what the assista

This guide uses scheduling as an example. The same approach applies to support, intake, and other conversations that combine questions with actions.

**Jump to:** [Architecture](#speaker-and-reasoner-roles) · [Squads](#start-with-one-assistant) · [Conversation flow](#questions-and-changes-of-topic) · [Voice design](#speaking-style) · [Testing](#testing-conversations)
**Jump to:** [Architecture](#speaker-and-reasoner-roles) · [Prompt example](#example-split-a-scheduling-prompt) · [Context](#what-context-does-the-reasoner-receive) · [Squads](#start-with-one-assistant) · [Conversation flow](#questions-and-changes-of-topic) · [Voice design](#speaking-style) · [Testing](#testing-conversations)

## Speaker and reasoner roles

Expand All @@ -30,6 +30,84 @@ For example, the speaker needs to know that it can help find an appointment. The

This separation lets you keep the speaker's instructions focused. Put detailed business procedures in the reasoner prompt. Your services remain responsible for enforcing permissions and validating actions.

### Where should an instruction go?

Use `model.speaker.instructions` for **how to conduct the conversation and when to ask for help**. Use `model.reasoner.instructions` for **how to decide and act once asked**.

| Instruction | Where it belongs | Why |
| --- | --- | --- |
| “Ask one question at a time and speak briefly.” | Speaker | Controls the interaction the caller hears. |
| “Delegate availability checks and booking requests.” | Speaker | Tells it when to involve the reasoner. |
| “Use `lookupAvailability` with an explicit date and time zone.” | Reasoner | Defines how to use a tool correctly. |
| “Call `bookAppointment` only after the caller confirms a specific slot.” | Reasoner | Defines the condition for taking an action. |
| “Never tell the caller a booking succeeded before the tool confirms it.” | Both, tailored to each role | The reasoner must verify success; the speaker must describe it accurately. |

The prompts are separate. Instructions written only in the speaker prompt aren't automatically included in the reasoner prompt. Put rules that affect both conversation and actions in both prompts, using wording appropriate to each role. Keep detailed tool procedures in the reasoner prompt rather than copying the entire prompt into both fields.

### Example: split a scheduling prompt

Suppose the assistant can look up availability and book appointments. The example below assumes you have attached tools named `lookupAvailability` and `bookAppointment`. Replace those names and requirements with your actual tools. The [configuration walkthrough](/gpt-live/configuration#create-an-assistant) provides a lookup-only example if you haven't connected booking yet.

```text title="model.speaker.instructions"
You help callers find and book appointments. Speak warmly and briefly.
Ask one question at a time. Use details the caller has already supplied.

Collect an explicit date and time zone, then delegate availability checks.
Read back the returned options. Ask the caller to confirm one specific
slot before delegating a booking request.

While a lookup runs, you may ask about preferences that don't change
its inputs. If the caller changes the date or time zone, acknowledge
the correction and delegate the updated request. Don't present an old
lookup result as an answer to the new request.

Only say an appointment is booked after the reasoner returns a confirmed
booking. If information is missing or a tool fails, explain the issue
and ask for the next detail needed.
```

```text title="model.reasoner.instructions"
Help the speaker complete the caller's scheduling request using the
conversation transcript and the available tools.

For availability, require an explicit date and time zone before calling
lookupAvailability. If either is missing or ambiguous, return a concise
clarification for the speaker to ask. Don't invent missing arguments.

For booking, require the caller's explicit confirmation of a specific
slot returned by lookupAvailability. Call bookAppointment only when
those conditions and the tool's required arguments are satisfied.

Use the latest request in the supplied transcript. Before taking a new
action, check prior tool results so you don't repeat a completed booking.
Don't retry an uncertain booking outcome without checking its status.

Return the relevant date, time zone, slots, or confirmed booking details
for the speaker to explain. If a tool fails, report the failure and the
next useful step. Don't invent availability or claim success without a
successful tool result.
```

The speaker asks the caller for clarification; the reasoner returns what needs clarification. The reasoner doesn't speak to the caller directly. Your service should also enforce booking requirements: prompt instructions don't replace validation in the tool handler.

### What context does the reasoner receive?

**At each delegation, the reasoner receives the available spoken conversation transcript so far**, including caller and assistant turns. It also receives its own instructions, available tool definitions, and retained history from prior reasoner work and tool results within that call. It doesn't rely on a short summary or a few selected lines supplied by the speaker.

The speaker and reasoner still have different context:

| Context | Speaker | Reasoner |
| --- | --- | --- |
| Live audio | Listens and speaks continuously | Receives text, not the audio stream |
| Spoken conversation | Part of the live conversation | Receives the transcript available when delegation starts |
| Instructions | Speaker prompt and personality guidance | Reasoner prompt; the speaker prompt isn't automatically copied |
| Tool work | Receives results returned by the reasoner | Has tool definitions and retained reasoner/tool history |
| HTTP `append-context` | Receives the injected speaker context | Doesn't receive it directly in its history |

The transcript is a snapshot of what Vapi has received at that point. Speech that arrives while the reasoner is already working doesn't automatically refresh that delegation's transcript. A subsequent delegation receives the updated transcript. Completed work and tool results can inform later reasoning, but the reasoner isn't continuously listening alongside the speaker.

For example, if the caller says “Actually, Thursday” while a Friday lookup is running, the speaker can hear and acknowledge the correction. The ongoing lookup may still return Friday's availability. Delegate the changed request, match results to the date they answer, and don't assume an interruption cancels an action already sent to a tool.

### When to delegate

Give the speaker a clear reason to ask for help: “Delegate requests to find an appointment that fits the caller's preferences.” Give the reasoner the steps needed to produce that result.
Expand Down Expand Up @@ -148,7 +226,7 @@ explaining a process. Follow requests to speed up, slow down, or
explain more. Leave room for the caller to respond.
```

These are prompted behaviors, not precise speed or pause controls. Test the spoken result. [Programmatic speech control](/gpt-live/limitations#can-i-inject-commentary-or-control-speech-during-a-call) is currently unsupported in Vapi's GPT-Live integration.
These are prompted behaviors, not precise speed or pause controls. Test the spoken result. Your application can also send [live speaker guidance](/gpt-live/configuration#live-call-control) with `append-context`; it doesn't guarantee exact wording, speed, or pauses.

### Pacing

Expand Down
Loading
Loading