Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 20 additions & 20 deletions docs/operations/token-cost-tracking.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ The token cost of tool metadata that MCP sends to clients via `tools/list`:

- Tool names and descriptions
- Parameter schemas (JSON Schema)
- Output schemas (`outputSchema`, JSON Schema) for tools that declare one
- Total overhead per tool and across all tools

**Exclusions:**
Expand All @@ -27,19 +28,19 @@ pnpm run measure-tokens
```
📊 MCP Server Token Cost Report
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Total Tokens: 14,068
Tool Count: 24
Average/Tool: 586
Total Tokens: 75,759
Tool Count: 73
Average/Tool: 1,038
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Per-Tool Breakdown:

┌─────────────────────────────┬────────┬─────────┐
│ Tool │ Tokens │ % Total │
├─────────────────────────────┼────────┼─────────┤
│ search_events │ 1192 │ 8.5% │
│ update_issue │ 1166 │ 8.3% │
│ search_docs │ 975 │ 6.9% │
│ get_alert_rule │ 2517 │ 3.3% │
│ find_releases │ 2033 │ 2.7% │
│ update_alert_rule │ 1975 │ 2.6% │
...
```

Expand All @@ -58,11 +59,11 @@ JSON format:

```json
{
"total_tokens": 14068,
"tool_count": 24,
"avg_tokens_per_tool": 586,
"total_tokens": 75759,
"tool_count": 73,
"avg_tokens_per_tool": 1038,
"tools": [
{"name": "search_events", "tokens": 1192, "percentage": 8.5},
{"name": "get_alert_rule", "tokens": 2517, "percentage": 3.3},
...
]
}
Expand All @@ -87,19 +88,18 @@ GitHub Actions workflow runs on every PR and push to main:

## Understanding the Results

- ~14,068 tokens total
- ~586 tokens/tool average
Run `pnpm run measure-tokens` for current numbers; the examples above are a
snapshot and go stale.

**Tool count limits:**

- **Target:** ≤20 tools (current best practice)
- **Maximum:** ≤25 tools (hard limit for AI agents)
**Tool count limits:** the gateway surface (`TOP_LEVEL_TOOL_NAMES`) targets ≤20
tools, hard limit 25. `EXPOSE_ALL_TOOLS` lists every tool and relies on harness
tool search.

**When to investigate:**

- Total tokens increase >10% without new tools
- Individual tool >1,000 tokens (indicates overly verbose descriptions)
- New tool adds >500 tokens (review description clarity)
- Total tokens increase >10% without new tools or new output schemas
- A tool sits well above the average (check the top of the per-tool table)
- A new tool, or a new output schema, adds far more than its neighbours

## Implementation Details

Expand All @@ -123,7 +123,7 @@ tsx measure-token-cost.ts --help # Show help
- Remove redundant examples
- Focus on unique, non-obvious details

**Simplify parameter schemas:**
**Simplify parameter and output schemas:**

- Use `.describe()` sparingly
- Avoid duplicate descriptions in nested schemas
Expand Down
6 changes: 6 additions & 0 deletions packages/mcp-core/scripts/measure-token-cost.ts
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ type ToolDefinition = {
title: string;
description: string | ((context: { experimentalMode: boolean }) => string);
inputSchema: Record<string, ZodTypeAny>;
outputSchema?: ZodTypeAny;
experimental?: boolean;
hideInExperimentalMode?: boolean;
deprecated?: { replacedBy: readonly string[] };
Expand Down Expand Up @@ -93,6 +94,11 @@ function formatToolsForMCP(tools: Record<string, ToolDefinition>) {
experimentalMode: false,
}),
inputSchema: jsonSchema,
// matches @modelcontextprotocol/server's
// standardSchemaToJsonSchema(schema, "output") (draft-2020-12)
...(tool.outputSchema && {
outputSchema: z.toJSONSchema(tool.outputSchema, { io: "output" }),
}),
...(tool.annotations && { annotations: tool.annotations }),
};
});
Expand Down
Loading