feat: passthrough OpenAI tool_search family; flatten-tool-search-family flag#183
feat: passthrough OpenAI tool_search family; flatten-tool-search-family flag#183yyyr-p wants to merge 6 commits into
Conversation
…ough
Floway 502'd on any /v1/responses request that carried a member of the
OpenAI Responses API tool_search feature family (gpt-5.4+): the store
layer's stageInputItem threw a TypeError for the unknown
`additional_tools` input item type, and the surrounding protocol had no
declared shape for the family's other artifacts either. Fix at the
gateway boundary so passthrough is loss-free regardless of any later
flatten policy.
Protocol (packages/protocols/src/responses/index.ts):
- Collect the new input item variants: `additional_tools` (Codex's
turn-scoped tool declaration), `program` and `program_output`
(Programmatic Tool Calling output items echoed back into input on
manual conversation replay).
- Extend `ResponsesHostedToolType` with `programmatic_tool_calling`.
- Declare `defer_loading?: boolean` and `allowed_callers?: string[]`
on `ResponsesFunctionTool` / `ResponsesCustomTool`, and `caller?`
on `ResponsesFunctionToolCallItem` / `ResponsesOutputFunctionCall`
(set to `{type: 'program'}` on PTC-driven calls).
- Extend `ResponsesOutputItem` with the two PTC output-item permissive
variants so the streamed lifecycle survives without a native
canonicalizer entry.
Store (packages/gateway/src/data-plane/chat/responses/items/):
- format.ts: register `program: 'pg'` and `program_output: 'pgo'`
prefixes so upstream-generated PTC output items can be persisted
(and stitched via `previous_response_id`).
- store.ts: skip-persist `additional_tools` in `stageInputItem` —
it is a turn-scoped developer directive the client re-declares
every request, with no cross-turn reader. Mirrors
`compaction_trigger`.
- format_test.ts: pin the new prefixes and the
`additional_tools`-throws-in-prefix-lookup regression.
Refs:
https://developers.openai.com/api/docs/guides/tools-tool-search
https://developers.openai.com/api/docs/guides/tools-programmatic-tool-calling
Introduce three protocol-level helpers used by both the gateway
interceptors and the responses-via-* translators to convert a Responses
`tools[]` between the gpt-5.4+ tool_search family shape and a legacy
tools[]-only shape:
- `unpackNamespaceTools(tools)`: expands hosted `type: 'namespace'`
container entries into their nested tools, prefixing each sub-tool
`name` with `<namespace>__`. Preserves the grouping semantic on
targets that can't carry a `namespace` container. Constraint:
sub-tool names must not contain `__`.
- `unprefixNamespaceToolCall(name)`: inverse of the name-prefixing.
Splits on the LAST delimiter so a namespace name that itself
contains `__` round-trips.
- `flattenToolSearchFamilyTools(tools)`: one-pass desugar — drop
`tool_search` and `programmatic_tool_calling` entries, unpack
namespaces (via `unpackNamespaceTools`), strip `defer_loading` and
`allowed_callers` fields on remaining function/custom tools.
`NAMESPACE_NAME_DELIMITER = '__'` is exported for callers that need to
know the delimiter (e.g. events translators). `__` is the only value
that satisfies the Chat Completions function-name regex
`^[a-zA-Z0-9_-]{1,64}$` and is uncommon enough in real sub-tool names
to keep the split unambiguous.
Ref: https://developers.openai.com/api/docs/guides/tools-tool-search
`translateResponsesTools` (responses-via-chat-completions) and `translateTools` (responses-via-messages) narrowed the input tools[] to `function` and `custom` entries and let everything else fall through without action — including `type: 'namespace'` container tools, whose nested sub-tools then disappeared silently. The drop dates back to when no client actually used `namespace`. Codex now packs real functional tools (`spawn_agent`, `send_message`, `followup_task`, …) inside a `collaboration` namespace and ships them via `additional_tools`; routing such a request through a claude-code or ollama candidate quietly lost every namespaced sub-tool, breaking the client's tool-call loop. Fix both translators to run `flattenToolSearchFamilyTools` at the top of the tools loop before narrowing. Namespace containers now expand into flat sub-tools (sub-tool names prefixed `<namespace>__`, which the response-side unprefixer reverses); the tool_search family scaffolding (`tool_search`, `programmatic_tool_calling` entries; `defer_loading` and `allowed_callers` fields) is stripped in the same pass since no target protocol supports it either. Leaf hosted tools (`web_search`, `image_generation`) still fall through — no sub-tools to expand and no faithful bridge onto Chat / Messages.
Register a new operator-toggleable flag that desugars the gpt-5.4+
tool_search family (additional_tools + namespace + defer_loading +
tool_search + programmatic_tool_calling + allowed_callers) into a
legacy tools[]-only shape before dispatch. Consumed by the pair of
request/response interceptors added in a follow-up commit.
Provider defaults (exhaustive FlagDefaults record forces each provider
to declare a value):
- codex / copilot / azure / custom: false — these upstreams either
speak native OpenAI Responses (Codex backend; Azure OpenAI) or
proxy it in a way that inherits gpt-5.4+ tool_search support
(Copilot); Custom is assumed OpenAI-compatible and operators
override on a case-by-case basis for legacy proxies.
- ollama / claude-code: true — Ollama has its own protocol and
claude-code translates through Anthropic Messages; without the
desugar the translator would either drop the family or throw on
the additional_tools input item.
The two hard-required upstream populations (family fully supported vs
family fully absent) map cleanly onto the flag off vs flag on states;
splitting the six operations into orthogonal flags would create
unreachable parameter combinations, so they share one flag.
…tors
Wire the `flatten-tool-search-family` flag into the responses
interceptor chain with a pair of complementary interceptors.
Request-side `withFlattenToolSearchFamily`:
- move every `additional_tools` input item's `tools[]` into top-level
`payload.tools[]` and remove the item from `payload.input`;
- apply `flattenToolSearchFamilyTools` on the merged tools[] to drop
`tool_search` / `programmatic_tool_calling` entries, unpack
`namespace` containers (sub-tool names prefixed `<namespace>__`),
and strip `defer_loading` / `allowed_callers` fields.
Response-side `withUnprefixNamespaceToolCalls`:
- stream-rewrite `function_call` and `custom_tool_call` output items
(both on `response.output_item.added/.done` and inside
`response.completed/.incomplete/.failed` envelopes) to strip the
`<namespace>__` prefix off `name`; on `custom_tool_call`, the
stripped prefix is moved into the `namespace` field.
- Since dispatchResponses is the terminal of the responses
interceptor chain, the same interceptor handles both native
Responses upstreams and the translated Chat/Messages paths — by
the time it runs the events are already Responses-shaped.
Ordering:
- flatten runs BEFORE demote-developer-to-system so the demote step
only sees message items with `role: 'developer'`, never the
additional_tools directive that also carries that role.
- unprefix runs adjacent to (and just outside) canonicalize-output-
items, so id/encrypted_content pinning finishes before name
rewriting on the same stream.
… test Drives responsesAttempt.generate against a mocked chat-completions candidate with the full tool_search family payload (additional_tools + namespace + tool_search + programmatic_tool_calling + defer_loading + allowed_callers). Verifies three properties end-to-end: 1. Flag on — outbound Chat Completions body captured by the mock is fully desugared: no additional_tools item in input; tools[] flat with sub-tool names prefixed `<namespace>__`; tool_search and programmatic_tool_calling entries dropped; defer_loading and allowed_callers fields stripped. 2. Flag on — mocked upstream returns a Chat Completions tool_call named `collaboration__spawn_agent`; the client-facing Responses event stream carries a function_call output item with the bare name `spawn_agent` (proves withUnprefixNamespaceToolCalls ran on the translated stream). 3. Flag off — translator throws `Invalid input item type 'additional_tools'` (rendered as 400 one layer up in serve.ts). Proves both that Layer 1 store passthrough is intact (no 502 at ingress) and that the fold interceptor is what removes additional_tools from input on flag=on. The test lives at responsesAttempt.generate — one layer below serve's error rendering — so the flag-off case asserts on the raw throw. This matches the semantic being tested (fold vs no fold) without adding a harness for serve's registry mock. Verified live against a Custom upstream fronting menci's Floway → real OpenAI gpt-4o via /chat/completions before writing this fixture; the fixture captures the same wire contract without needing a live upstream.
Known limitations of the flatten-tool-search-family interceptorsThe 1.
|
1 participant
Summary
Floway 502'd on any
/v1/responsesrequest whoseinput[]carried atype: 'additional_tools'item — the gpt-5.4+ OpenAI Responses API tool_search feature family (additional_toolsinput items,type: 'namespace'tool containers,type: 'tool_search'/programmatic_tool_callinghosted tools,defer_loading/allowed_callersfunction-tool fields,program/program_outputoutput items) was entirely unknown to the protocol / store / translate layers.stageInputItemthrew a TypeError for the unknown item type; further latent gaps existed for the family's other members that would surface as clients start using them.The change lands in two layers:
Layer 1 — unconditional passthrough (no flag): make Floway an opaque gateway for the whole family, per the repo's
Upstream Models And Field Values Are Opaquedesign principle. Protocol union collects the new input/output item variants, hosted-tool discriminator, function/custom tool fields, andfunction_call.caller. Store registersprogram: 'pg'/program_output: 'pgo'prefixes so PTC output items survive snapshot storage, and skip-persistsadditional_tools(turn-scoped developer directive re-declared verbatim by the client on every request — mirrorscompaction_trigger).Layer 2 — opt-in flatten to legacy shape (
flatten-tool-search-familyflag): for upstreams that do not implement the family (translated Chat Completions / Messages targets; Responses endpoints stuck below gpt-5.4), one interceptor rewrites the request in a single pass — moveadditional_tools[].toolsinto top-levelpayload.tools[]; unpacknamespacecontainers (sub-tool names prefixed ` __ `); droptool_search/programmatic_tool_callingentries; stripdefer_loading/allowed_callers. A response-side interceptor scans the downstream event stream and reverses the name-prefixing onfunction_call(bare name) andcustom_tool_call(bare name +namespacefield) output items so clients match against their originally-declared tool registry.Provider defaults:
falsefor OpenAI-native (codex / copilot / azure / custom);truefor translated targets (ollama / claude-code) that need the desugar to make the request survive translation.Also fixes a pre-existing silent-drop bug in `responses-via-chat-completions` and `responses-via-messages` translators — their `translateResponsesTools` narrowed to `function` / `custom` and let `namespace` container tools fall through, silently losing the nested sub-tools. Both now apply the same shared `flattenToolSearchFamilyTools` helper unconditionally so sub-tools survive translation with the same `__` prefix that the response-side interceptor reverses.
Structured as 6 commits following the layering:
fix(responses): accept OpenAI tool_search family artifacts on passthroughfeat(protocols): tool_search family desugar helpersfix(translate): unpack namespace sub-tools instead of silently droppingfeat(flags): flatten-tool-search-family for legacy upstreamsfeat(responses): flatten-tool-search-family request/response interceptorstest(responses): add fixture-based tool_search family e2e integration testRefs:
Test plan