Skip to content

Add Anthropic tool search bridge for deferred tool definitions - #941

Open
phfarath wants to merge 1 commit into
evalstate:mainfrom
phfarath:feat/anthropic-tool-search
Open

Add Anthropic tool search bridge for deferred tool definitions#941
phfarath wants to merge 1 commit into
evalstate:mainfrom
phfarath:feat/anthropic-tool-search

Conversation

@phfarath

@phfarath phfarath commented Sep 4, 2026

Copy link
Copy Markdown

Summary

First slice of #766: a provider-bridged tool-search abstraction, implemented for Anthropic's native mechanism.

With many MCP tools, every definition is serialized into each request, costing tokens and degrading selection accuracy past ~30-50 tools. This adds a RequestParams.tool_search policy (off/auto/always, default off) and bridges it to Anthropic tool search: regular tool definitions are marked defer_loading: true and a server-side BM25 tool_search tool is appended, so the model retrieves definitions on demand. Search runs server-side; the agent only receives the final tool_use for the real tool, so the execution loop is unchanged (name resolution already tolerates definitions that were never loaded, and *_tool_result blocks — including tool_search_tool_result — are already persisted/replayed by the existing server-tool channel machinery).

Changes

  • RequestParams.tool_search: Literal[off,auto,always] (default off), excluded from provider payloads via BASE_EXCLUDE_FIELDS
  • AnthropicLLM: _defer_tool_definitions() marks tools defer_loading and appends the tool_search_tool_bm25_20251119 tool param; _resolve_tool_search_active() resolves the policy
  • Disabled on anthropic-vertex (beta allowlist), suppressed for structured tool_use requests, and disabled when a sampling tool_choice pins specific tools
  • auto defers above 16 tools
  • Docs: Anthropic provider page + generated request-params reference

Testing

  • Unit tests: tests/unit/fast_agent/llm/provider/anthropic/test_tool_search.py (policy resolution, defer marking, structured suppression, Vertex gating, threshold)
  • Local payload E2E: real app/agent/tool-runner pipeline with network stubbed at the SDK boundary — confirmed defer_loading: true + search tool present with always, absent by default
  • Live API smoke (haiku): API accepted the payload; model searched server-side, called deferred tools by name; turn-2 history replay with tool_search_result blocks worked; control run unchanged
  • format --check, lint, typecheck clean on changed files; llm suite 1693 passed (6 pre-existing failures reproduced on clean baseline)

Follow-ups (out of scope here)

  • Bridge defer_loading for provider-managed remote MCP on Anthropic (currently OpenAI Responses only)
  • Generic client-side search tool fallback for providers without native support

Addresses #766.


Add RequestParams.tool_search policy (off/auto/always, default off) and
bridge it to Anthropic's native tool search: regular tool definitions are
marked defer_loading and a server-side BM25 tool_search tool is exposed so
the model retrieves definitions on demand. Disabled on Vertex, suppressed
for structured tool_use requests and sampling tool_choice pins.

Addresses evalstate#766.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant