diff --git a/fern/customization/custom-llm/thegrid.mdx b/fern/customization/custom-llm/thegrid.mdx index 2bc804f14..38a359cc8 100644 --- a/fern/customization/custom-llm/thegrid.mdx +++ b/fern/customization/custom-llm/thegrid.mdx @@ -37,7 +37,7 @@ Send this body to `POST https://api.vapi.ai/assistant` with the same Vapi privat "model": { "provider": "custom-llm", "url": "https://api.thegrid.ai/v1", - "model": "text-standard", + "model": "text-prime", "metadataSendMode": "off", "maxTokens": 600, "messages": [ @@ -54,14 +54,36 @@ Setting `metadataSendMode` to `off` keeps Vapi call metadata out of requests to ## Choose an instrument -`GET https://api.thegrid.ai/v1/models` lists available instrument IDs. Check [instrument specifications](https://thegrid.ai/instruments) and [pricing](https://thegrid.ai/pricing) for current details. Examples include: +`GET https://api.thegrid.ai/v1/models` lists available instrument IDs. Check [instrument specifications](https://thegrid.ai/instruments) and [pricing](https://thegrid.ai/pricing) for current details. Current instruments: -| Instrument | Use it for | +### Capability tiers + +| Instrument | Description | +| --- | --- | +| `text-standard` | Price-optimized models with low latency, high throughput, and shorter maximum outputs | +| `text-prime` | Reliable models for everyday text generation, editing, and analysis across diverse workflows | +| `text-max` | Frontier models for deep reasoning, long context, and complex workflows | +| `agent-standard` | Price-optimized models for fast tool calls, simple agent loops, high-throughput automation, and orchestration | +| `agent-prime` | Reliable models for dependable agentic applications, multi-step tool use, and reasoning workflows | +| `agent-max` | Frontier models for autonomous research, deep multi-step tool chains, and complex long-horizon tasks | +| `code-standard` | Price-optimized models for rapid autocomplete, linting, high-frequency suggestions, and batch edits | +| `code-prime` | Reliable models for everyday software tasks, code completion, review, and standard debugging | +| `code-max` | Frontier models for complex research, architectural decisions, debugging, and multi-file development | + +### Latest-model aliases + +Each alias follows the newest model release from one lab, with fallbacks to previous versions. + +| Instrument | Description | | --- | --- | -| `text-standard` | General conversation. The usual starting point. | -| `text-prime`, `text-max` | More demanding questions; compare current speed and price. | -| `agent-standard`, `agent-max` | Multi-step tool use. | -| `code-standard`, `code-prime` | Assistants that read or write code. | +| `bytedance-pro-latest` | Latest ByteDance Pro model release with fallbacks to previous versions for reliability | +| `claude-opus-latest` | Latest Claude Opus model release with fallbacks to previous versions for reliability | +| `deepseek-pro-latest` | Latest DeepSeek Pro model release with fallbacks to previous versions for reliability | +| `gemini-pro-latest` | Latest Gemini Pro model release with fallbacks to previous versions for reliability | +| `glm-latest` | Latest GLM model release with fallbacks to previous versions for reliability | +| `gpt-sol-latest` | Latest OpenAI GPT Sol model release with fallbacks to previous versions for reliability | +| `kimi-latest` | Latest Kimi model release with fallbacks to previous versions for reliability | +| `minimax-latest` | Latest MiniMax model release with fallbacks to previous versions for reliability | ## Set `maxTokens` for spoken replies @@ -69,6 +91,20 @@ Vapi uses `250` for `maxTokens` when it is omitted and sends the value to The Gr The `600` value in the example is a starting point. Test your chosen instrument with representative spoken turns and raise the limit if replies are truncated. Keep the system prompt explicit about brevity. +## Check the other model settings + +In the [Vapi Dashboard](https://dashboard.vapi.ai), open your assistant, select the **Model** card, and choose **Custom LLM** in the panel that opens. Check each setting against The Grid's values. [`GET https://api.thegrid.ai/v1/models`](https://api.thegrid.ai/v1/models) returns the current numbers for every instrument. + +| Setting | What to set | +| --- | --- | +| **Model** | The exact instrument ID, such as `text-prime`. Don't use the display name. | +| **Custom LLM URL** | `https://api.thegrid.ai/v1` | +| **Max Tokens** | At or below the instrument's `max_completion_tokens`. Vapi's default of `250` suits short spoken replies. | +| **Request Timeout** | Above the instrument's `grid.ttft_max_seconds`, the longest time its providers can take to start streaming and still meet the specification. Vapi's default is 20 seconds. | +| **Metadata Send Mode** | **Off — send only messages**, to keep Vapi call metadata out of requests to The Grid. | + +Limits differ between instruments and change over time, so read them from [`GET https://api.thegrid.ai/v1/models`](https://api.thegrid.ai/v1/models) instead of copying numbers into your configuration. + ## Tool calling The Grid's API supports OpenAI-style function calling. To configure Vapi tools, see the [tool calling integration guide](/customization/tool-calling-integration). Test a tool call with your chosen instrument before relying on it.