Skip to content

Add OpenAI Responses API support v2 - #7008

Open
dimitarproynov wants to merge 1 commit into
spring-projects:mainfrom
dimitarproynov:dimitarproynov/implement-responses-api-support-v2
Open

dimitarproynov wants to merge 1 commit into
spring-projects:mainfrom
dimitarproynov:dimitarproynov/implement-responses-api-support-v2

Conversation

@dimitarproynov

Copy link
Copy Markdown
Contributor

Teach OpenAiChatModel to talk to /v1/responses as well as /v1/chat/completions, which is the only way to combine reasoning with tool calling on GPT-5.4 and later. One ChatModel, one OpenAiChatOptions and one auto-configuration serve both endpoints, so nothing above the model - ChatClient, advisors, chat memory, tool calling, observability

  • has to know which one a request went to.

OpenAiChatModel keeps the options merging, the observations and the common builder, and forwards to a package-private OpenAiChatApi: OpenAiChatCompletionsApi, the existing mapping moved out of the model unchanged, and OpenAiResponsesApi. Both are wired up front over the same pair of OpenAI clients and each request picks one, so a ChatClient call that overrides the model or the endpoint at runtime is answered by the same bean.

spring.ai.openai.chat.api selects the endpoint and defaults to auto: the Responses API from GPT-5.4 onwards, or when the request uses a setting only it provides, and Chat Completions otherwise. Anything that does not parse as a GPT version - a Microsoft Foundry deployment name, a third-party model - stays on Chat Completions, the endpoint every OpenAI-compatible provider implements, and GitHub Models is refused the Responses API at startup. OpenAiChatApiSelection holds that rule together with the warnings: an option the chosen endpoint cannot honour is logged once per JVM and dropped rather than rejected, because the same options bean is routinely used against both.

The Responses side is unchanged in substance. A reply is an ordered list of typed items rather than one message, and a reasoning model returns its chain of thought as an opaque encrypted blob that must be handed back verbatim to keep its train of thought across a tool call. Both are carried by the MessagePart list on AssistantMessage, one part per output item and in order, so a reasoning item still precedes the function calls it produced. Only function_call items become ToolCalls, so ToolCallingAdvisor never looks for a local callback named web_search, and reasoning is replayed only when its payload came from this endpoint. The stateless mode is the only one implemented: every request sends the whole Prompt and store=false, which is also what makes the encrypted reasoning come back at all.

Also add reasoningSummary, maxToolCalls, hostedTools, include and truncation to OpenAiChatOptions and OpenAiChatProperties, map safetyIdentifier on both endpoints - the property was documented but never sent - and fold the Responses material into the OpenAI chat reference documentation.

Full build passed as well as integration tests

Teach OpenAiChatModel to talk to /v1/responses as well as
/v1/chat/completions, which is the only way to combine reasoning with
tool calling on GPT-5.4 and later. One ChatModel, one OpenAiChatOptions
and one auto-configuration serve both endpoints, so nothing above the
model - ChatClient, advisors, chat memory, tool calling, observability
- has to know which one a request went to.

OpenAiChatModel keeps the options merging, the observations and the
common builder, and forwards to a package-private OpenAiChatApi:
OpenAiChatCompletionsApi, the existing mapping moved out of the model
unchanged, and OpenAiResponsesApi. Both are wired up front over the
same pair of OpenAI clients and each request picks one, so a
ChatClient call that overrides the model or the endpoint at runtime is
answered by the same bean.

spring.ai.openai.chat.api selects the endpoint and defaults to auto:
the Responses API from GPT-5.4 onwards, or when the request uses a
setting only it provides, and Chat Completions otherwise. Anything
that does not parse as a GPT version - a Microsoft Foundry deployment
name, a third-party model - stays on Chat Completions, the endpoint
every OpenAI-compatible provider implements, and GitHub Models is
refused the Responses API at startup. OpenAiChatApiSelection holds
that rule together with the warnings: an option the chosen endpoint
cannot honour is logged once per JVM and dropped rather than
rejected, because the same options bean is routinely used against
both.

The Responses side is unchanged in substance. A reply is an ordered
list of typed items rather than one message, and a reasoning model
returns its chain of thought as an opaque encrypted blob that must be
handed back verbatim to keep its train of thought across a tool call.
Both are carried by the MessagePart list on AssistantMessage, one part
per output item and in order, so a reasoning item still precedes the
function calls it produced. Only function_call items become ToolCalls,
so ToolCallingAdvisor never looks for a local callback named
web_search, and reasoning is replayed only when its payload came from
this endpoint. The stateless mode is the only one implemented: every
request sends the whole Prompt and store=false, which is also what
makes the encrypted reasoning come back at all.

Also add reasoningSummary, maxToolCalls, hostedTools, include and
truncation to OpenAiChatOptions and OpenAiChatProperties, map
safetyIdentifier on both endpoints - the property was documented but
never sent - and fold the Responses material into the OpenAI chat
reference documentation.

Signed-off-by: Dimitar Proynov <dimitar.proynov@broadcom.com>
* @author Dimitar Proynov
* @since 2.1.0
*/
final class OpenAiChatApiSelection {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to verify the Api selection for the OpenAI compatible providers as well. I haven't explored the selection logic yet; but want to call this out here.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See method isResponsesModel

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request openai

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants