Skip to content

Add OpenAI Responses API support - #7006

Open
dimitarproynov wants to merge 2 commits into
spring-projects:mainfrom
dimitarproynov:dimitarproynov/implement-responses-api-support
Open

dimitarproynov wants to merge 2 commits into
spring-projects:mainfrom
dimitarproynov:dimitarproynov/implement-responses-api-support

Conversation

@dimitarproynov

Copy link
Copy Markdown
Contributor

Add OpenAiResponsesChatModel, a second OpenAI ChatModel talking to /v1/responses, which is the only way to combine reasoning with tool calling on GPT-5.4 and later. The model is stateless: every call sends the whole Prompt, so ChatMemory, advisors and RAG keep working as they do with OpenAiChatModel, and the server-side continuation modes stay out of scope.

A Responses reply is an ordered list of typed items rather than one message, and a reasoning model returns its chain of thought as an opaque encrypted blob that must be handed back verbatim to keep its train of thought across a tool call. Both are carried by the MessagePart list on AssistantMessage, one part per output item and in order, so a reasoning item still precedes the function calls it produced. A reasoning item becomes a ReasoningPart whose OpaquePayload holds the encrypted content, a message item a TextPart, a function call a ToolCallPart, a generated image a MediaPart, and every tool OpenAI ran itself an UnknownPart keeping the item verbatim. The item id, the item status and the message phase travel in the part attributes, since an item cannot be rebuilt without them.

Only function_call items become ToolCalls, so ToolCallingAdvisor never looks for a local callback named web_search. Reasoning is replayed only when its payload came from this provider, because the API rejects unsigned reasoning; a foreign or payload-less reasoning part is skipped with a debug log rather than sent and refused.

Streaming stamps every part with its output index through StreamingParts and lets MessageAggregator rebuild the transcript. A completing item is emitted emptied of whatever already arrived as deltas, so its payload and attributes reach the aggregated part without the text being shown twice, and every chunk carries the response id the aggregator groups indexed parts by.

Also add OpenAiResponsesChatOptions and HostedTool with support for the tools OpenAI executes server-side, structured output, image and PDF input, observability, an auto-configuration selecting the endpoint through spring.ai.openai.chat.api, and a reference documentation page.

Full build and all OpenAI integration tests pass.

Add OpenAiResponsesChatModel, a second OpenAI ChatModel talking to
/v1/responses, which is the only way to combine reasoning with tool
calling on GPT-5.4 and later. The model is stateless: every call sends
the whole Prompt, so ChatMemory, advisors and RAG keep working as they
do with OpenAiChatModel, and the server-side continuation modes stay
out of scope.

A Responses reply is an ordered list of typed items rather than one
message, and a reasoning model returns its chain of thought as an
opaque encrypted blob that must be handed back verbatim to keep its
train of thought across a tool call. Both are carried by the
MessagePart list on AssistantMessage, one part per output item and in
order, so a reasoning item still precedes the function calls it
produced. A reasoning item becomes a ReasoningPart whose OpaquePayload
holds the encrypted content, a message item a TextPart, a function
call a ToolCallPart, a generated image a MediaPart, and every tool
OpenAI ran itself an UnknownPart keeping the item verbatim. The item
id, the item status and the message phase travel in the part
attributes, since an item cannot be rebuilt without them.

Only function_call items become ToolCalls, so ToolCallingAdvisor never
looks for a local callback named web_search. Reasoning is replayed
only when its payload came from this provider, because the API rejects
unsigned reasoning; a foreign or payload-less reasoning part is
skipped with a debug log rather than sent and refused.

Streaming stamps every part with its output index through
StreamingParts and lets MessageAggregator rebuild the transcript. A
completing item is emitted emptied of whatever already arrived as
deltas, so its payload and attributes reach the aggregated part
without the text being shown twice, and every chunk carries the
response id the aggregator groups indexed parts by.

Also add OpenAiResponsesChatOptions and HostedTool with support for
the tools OpenAI executes server-side, structured output, image and
PDF input, observability, an auto-configuration selecting the endpoint
through spring.ai.openai.chat.api, and a reference documentation page.

Signed-off-by: Dimitar Proynov <dimitar.proynov@broadcom.com>
Signed-off-by: Dimitar Proynov <dimitar.proynov@broadcom.com>
* @author Dimitar Proynov
* @since 2.1.0
*/
public sealed interface HostedTool {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we rename HostedTool to something like OpenAIManagedTool or BuiltInTool (what OpenAI Responses refer this as) to add more clarity?


private RequestOptions buildRequestOptions(Prompt prompt) {
OpenAiResponsesChatOptions chatOptions = (OpenAiResponsesChatOptions) prompt.getOptions();
Assert.state(chatOptions != null, "Prompt options must be OpenAiResponsesChatOptions type");

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the assertion needs to happen before casting prompt.getOptions in the line before

* @param connectorId an OpenAI-managed connector id, or {@code null}
* @param allowedTools restricts the callable tools, or {@code null} for all of them
* @param headers headers to send to the MCP server, e.g. authorization
* @param requireApproval {@code always} or {@code never}, or {@code null} for the

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This contradicts with the out-of-scope statement above on requireApproval. Depending on the allowed values, we can introduce assertion at the toTool() method as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request openai

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants