Tool-def deferral part 3: proxy fallback for agents without native deferral

[iamtoruk]

Split out of #614 per its scope update: parts 1 (deferral-gap detection, optimize.ts) and 2 (config plans, act/plans.ts) shipped in #631. This tracks the remaining design work: a proxy that injects tool-def deferral for agents with no native support. Design owner per the 2026-07-16 thread: Pavan (AVSRPA1KR).

[AVSRPA1KR]

This is the remaining request-path piece the epic (#602) reserved for its own review: a local, opt-in proxy that gives tool-def deferral to Claude Code-compatible agents that have no native support.

Who needs it

Parts 1–2 help agents with native deferral. This is for the ones without it — Cline, Roo Code, Windsurf, Continue, and Copilot when pointed at a Claude model over the Anthropic API all load every MCP tool def upfront on every turn. Pointing them at this proxy gives them the same deferral Claude Code has natively, using the GA API mechanism (defer_loading + a tool search tool), with the prefix untouched so prompt caching is preserved.

Scope is deliberately Anthropic-only. Other providers/wire formats are explicitly out of scope for this issue.

Epic Class-C evidence gate (#602)

Gate	Answer
Pre-turn classification accuracy	Not applicable. Deferral needs no classification — no model choice, no per-turn prediction. That gate belongs to routing, which stays firmly out of scope here.
Cache-hit impact	Positive by documented design (deferred defs leave the prefix; discovered tools append inline, prefix byte-stable). Encoded as a test: the cached prefix of consecutive requests through the proxy is asserted byte-identical — a proxy that perturbs the prefix would turn every turn into a full-price cache re-write, so that's the failure mode I most wanted pinned.
Auth mode	Pure pass-through. Authorization / x-api-key forwarded untouched; no credentials at rest, none in logs (redaction at the logger level, /key|token|secret|auth/i); request/response bodies never logged. Binds 127.0.0.1 by default.

Design

Pass-through first. Forwards byte-faithfully by default. Only a POST /v1/messages it fully understands is transformed; anything else — unknown path, unsupported model, malformed body, or any thrown error mid-transform — degrades to forwarding the original bytes. Never a request-dropper. SSE streams pipe through unbuffered.

Request path. On a qualifying request (supported model, enough MCP tool defs, not already deferring): inject the server-side tool_search_tool_regex_20251119 tool (kept non-deferred), set defer_loading:true on the MCP tool defs, keep ≥1 non-deferred tool, and relocate any cache_control off deferred tools onto a non-deferred one (a deferred tool carrying cache_control is a 400 — encoded as a test).

Response path — the core competency. Discovered tools come back as tool_reference / tool_search_tool_result blocks that a non-native client may drop or mangle on replay. The proxy keeps per-conversation bookkeeping and repairs replayed history — re-inserting dropped blocks only on positive anchor evidence (never speculatively), and stripping client-invented tool_results for srvtoolu_ ids. This is the whole reason a naive proxy is unsafe: openai/codex#23839 (identifiers dropped on replay after deferred discovery) is the reference failure mode this is built to prevent.

Model gating. Sonnet 4.5+ / Opus 4.5+ / Haiku 4.5 (verify the exact list at implementation time per #605). Unknown or older models pass through byte-identically.

Bonus interaction with part 1. A Claude Code user behind this proxy can safely set ENABLE_TOOL_SEARCH=true and keep native deferral, because the proxy forwards tool_reference — resolving the silent-off state that non-first-party base URLs otherwise cause (the exact "unknown proxy" case part 1's defer-enable refuses to auto-fix).

Packaging & lifecycle

A separately installed package, not a core subcommand — the core's zero-proxy guarantee stays auditable by directory listing. Request-path code imports nothing from src/; the only core dependency is the act journal, used by install / uninstall / status, which mirror guard's conventions (journaled, surgical config edit of ANTHROPIC_BASE_URL, uninstall restores byte-identical settings). Fail-open: proxy down + base URL reverted = client works normally. Zero new npm dependencies (node:http/crypto only).

What it never does

No model rewriting, no param changes, no per-turn classification, no prompt mutation beyond relocating tool defs, no telemetry, no network calls except forwarding to the provider the client already targets.

Non-goals

• Routing — explicitly not this, not ever in this issue.
• Other providers / wire formats — Anthropic Messages only.
• Client-side search executor — server-side only; a client-side variant is a possible later follow-up and needs no protocol change, but not in scope here.
• Nothing on by default — install is an explicit, journaled, reversible opt-in.

Measurement

act report (#606) integration: capture a baseline (tool-def tokens/session, cache-write tokens/session, cache-hit rate over the trailing window) and report realized deltas post-apply — and say so plainly if deferral ever hurt hit rate for that user.

One honest note carried over from part 2: the defer-* plans don't capture baselines yet — act report currently returns "not measurable: no baseline captured at apply time" for them (verified locally). It's a small, self-contained fix (needsConfigBaseline + a captureBaseline case + a report row), and arguably it should land before this so measurement exists before more surface. Happy to take that first.

Acceptance

• A user on a non-native agent (Cline/Windsurf/Continue/Copilot in Claude mode) installs the proxy, runs a real session with all tools available and correct responses; the tool_reference round-trip is verified in tests.
• Cached prefix of consecutive requests is byte-identical.
• Unsupported models, non-/v1/messages paths, malformed bodies, transform errors all pass through byte-identically.
• A client that strips discovery blocks on replay is repaired; the conversation continues.
• No auth material in any log line.
• install → journal entry; uninstall → byte-identical settings restored; proxy down + reverted config → client works normally.
• Zero behaviour change to existing commands; core src/ untouched but for the ActionKind union (defer-proxy-install / -uninstall).

[AVSRPA1KR]

@iamtoruk two things I'd like your steer on before I start building, since both shape the approach:

1. Where should it live? The epic's Class-C rule allows a request-path proxy in-repo (as a separate opt-in package) after review, but a companion package outside the repo is also reasonable given it's request-path code. Do you have a preference? Happy either way.

2. act report baselines first? As noted in the measurement section, the part-2 defer-* plans don't capture baselines yet, so act report can't measure them. It's a small, self-contained fix. Do you want me to pick it up first and then move to proxy?