feat(responses): support Codex image_gen through the image shim#172
Open
Menci wants to merge 20 commits into
Open
feat(responses): support Codex image_gen through the image shim#172Menci wants to merge 20 commits into
Menci wants to merge 20 commits into
Conversation
Copilot's /responses endpoint rejects Codex's deferred-tool namespace
{ type: "namespace", name: "image_gen", tools: [...] } with a 400
"User-defined namespace 'image_gen' collides with an existing tool
namespace", the same way it already rejects the hosted
{ type: "image_generation" } tool. Extend the existing strip so both
image-generation shapes are dropped on the Copilot Responses boundary,
while every other namespace and tool is preserved. tool_choice cleanup
now also drops a choice that names the removed image_gen namespace.
Evidence: caozhiyuan/copilot-api#312 and commit e260303a.
Add tests for dropping the image_gen namespace tool while preserving unrelated namespaces, for clearing a forced image_gen namespace tool_choice, and for keeping a tool_choice that names a surviving namespace. Update TRANSLATION.md's Copilot Responses boundary bullet.
Owner
Author
|
Claude Code agent update: production Azure
I am extending this PR so the gateway image shim consumes the Codex namespace before the Copilot fallback strip, canonicalizes alternate image declarations to one replacement tool, preserves/restores the original namespace/choice shape downstream, and follows deterministic last-declaration precedence. The existing Copilot strip remains the disabled-shim safety net. |
Cover exact namespace recognition, namespace-only dispatch and echo, forced namespace tool choice restoration, tool-name collision handling, mixed declaration ordering, duplicate hosted last-wins behavior, and preservation of unrelated namespaces.
Menci
changed the title
Model deferred-tool namespaces explicitly and let server-tool adapters match full client declarations and forced choices. The image adapter consumes only the exact Codex image_gen/imagegen namespace, collapses mixed image declarations to one collision-safe function, selects the last declaration for config and canonical echo, and restores the original tool choice. Web search retains first-declaration precedence. The Copilot boundary strip remains the fallback when the image shim flag is disabled.
Capture the forced namespace choice at the first upstream boundary before the multi-turn shim intentionally demotes it to auto, and prove that disabling the shim leaves image_gen available for the provider-specific fallback.
Describe the image shim as the primary image_gen compatibility path and the Copilot strip as its disabled-shim fallback. Record the Azure production probe semantics for reserved namespaces and duplicate hosted last-wins behavior, and cite Microsoft's pinned native Responses header examples.
Make the namespace round-trip test mirror the real multi-turn request: the first forced shim call is demoted to auto before the terminal turn, whose upstream envelope therefore echoes auto. The synthesized client envelope must still restore the original forced namespace choice.
The multi-turn shim changes a forced replacement choice to auto after the first intercepted call so the orchestrator can finish. Restore the exact pre-rewrite client choice in synthesized response echoes even when the final upstream turn therefore echoes auto rather than the injected function choice.
Select the canonical declaration while scanning, then construct the single replacement function after deduplication. This removes repeated throwaway function objects for duplicate declarations and eliminates the non-null index assertion without changing first/last precedence.
Responses request JSON is canonicalized but not runtime-schema validated. Guard the namespace's nested tools array before checking its exact Codex shape so a malformed namespace remains unrecognized instead of throwing inside the shim.
Verify the hosted image_generation choice follows the same replacement and post-demotion restoration path as the Codex namespace choice, and assert that a collision-resolved replacement carries neither namespace metadata nor a mangled imagegen name.
Runtime request bodies can contain non-object entries despite the TypeScript wire type. Require the sole namespace member to be a non-null object before checking the exact imagegen function discriminants.
Remove the stale claim that other Responses upstreams accept both image tool shapes. The gateway shim is the normal compatibility path; this provider boundary remains the Copilot-only disabled-shim fallback.
Model an upstream whose final Responses envelope omits tool_choice after the shim captured and rewrote a forced image_gen namespace choice. The synthesized client envelope must still restore that exact original choice.
Consult the pre-rewrite server-tool choice before falling back to the terminal upstream envelope. This preserves a forced hosted or namespace choice even when the upstream omits tool_choice from the final response.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Codex clients can advertise image generation through a deferred-tool namespace instead of the hosted Responses tool:
{ "type": "namespace", "name": "image_gen", "tools": [{ "type": "function", "name": "imagegen" }] }Azure and Copilot reject this namespace before inference because
image_genis reserved. A production Azuregpt-5.4+gpt-image-2probe, using the documentedx-ms-oai-image-generation-deploymentheader, also confirmed:image_generation+ namespace is rejected in either orderExtend the Responses image-generation shim to recognize only the exact Codex namespace shape as an alternate image capability. The shim consumes all recognized hosted/namespace declarations before the provider boundary and replaces them with one collision-resolved plain function tool. The replacement carries no namespace metadata and does not expose a mangled
imagegenname.Calls are executed through the existing image backend and return the normal
image_generation_calllifecycle. Synthesized Responses envelopes restore the selected original declaration and exact original forcedtool_choice. Multiple declarations use deterministic last-recognized precedence for configuration and canonical echo, matching Azure's observed duplicate-hosted behavior. Unrelated namespaces remain untouched.The Copilot provider strip remains only as a fallback when the image-generation shim is explicitly disabled.
The protocol types now model namespace tools, nested functions, and named namespace choices explicitly.
Sources:
Test plan