Add Anthropic tool search bridge for deferred tool definitions - #941
Open
phfarath wants to merge 1 commit into
Open
Add Anthropic tool search bridge for deferred tool definitions#941phfarath wants to merge 1 commit into
phfarath wants to merge 1 commit into
Conversation
Add RequestParams.tool_search policy (off/auto/always, default off) and bridge it to Anthropic's native tool search: regular tool definitions are marked defer_loading and a server-side BM25 tool_search tool is exposed so the model retrieves definitions on demand. Disabled on Vertex, suppressed for structured tool_use requests and sampling tool_choice pins. Addresses evalstate#766.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
First slice of #766: a provider-bridged tool-search abstraction, implemented for Anthropic's native mechanism.
With many MCP tools, every definition is serialized into each request, costing tokens and degrading selection accuracy past ~30-50 tools. This adds a
RequestParams.tool_searchpolicy (off/auto/always, defaultoff) and bridges it to Anthropic tool search: regular tool definitions are markeddefer_loading: trueand a server-side BM25tool_searchtool is appended, so the model retrieves definitions on demand. Search runs server-side; the agent only receives the finaltool_usefor the real tool, so the execution loop is unchanged (name resolution already tolerates definitions that were never loaded, and*_tool_resultblocks — includingtool_search_tool_result— are already persisted/replayed by the existing server-tool channel machinery).Changes
RequestParams.tool_search: Literal[off,auto,always](defaultoff), excluded from provider payloads viaBASE_EXCLUDE_FIELDSAnthropicLLM:_defer_tool_definitions()marks toolsdefer_loadingand appends thetool_search_tool_bm25_20251119tool param;_resolve_tool_search_active()resolves the policyanthropic-vertex(beta allowlist), suppressed for structuredtool_userequests, and disabled when a samplingtool_choicepins specific toolsautodefers above 16 toolsTesting
tests/unit/fast_agent/llm/provider/anthropic/test_tool_search.py(policy resolution, defer marking, structured suppression, Vertex gating, threshold)defer_loading: true+ search tool present withalways, absent by defaulttool_search_resultblocks worked; control run unchangedformat --check,lint,typecheckclean on changed files; llm suite 1693 passed (6 pre-existing failures reproduced on clean baseline)Follow-ups (out of scope here)
defer_loadingfor provider-managed remote MCP on Anthropic (currently OpenAI Responses only)Addresses #766.