Skip to content

Make replaying reasoning content opt-out - #6986

Open
chengwudi1 wants to merge 1 commit into
spring-projects:mainfrom
chengwudi1:fix-replay-reasoning-content
Open

chengwudi1 wants to merge 1 commit into
spring-projects:mainfrom
chengwudi1:fix-replay-reasoning-content

Conversation

@chengwudi1

Copy link
Copy Markdown

Fixes #6968

Problem

reasoning_content is echoed back on assistant messages so that
OpenAI-compatible reasoning endpoints which require it keep working.
DeepSeek's thinking mode answers HTTP 400 when an assistant turn that
carried reasoning arrives without it (#6296), which is why the replay was
added in the first place.

Endpoints in the opposite position return the property but reject it on
the way back. Groq answers a follow-up request built from such a history
with:

400: 'messages.6' : for 'role:assistant' ...
property 'reasoning_content' is unsupported

The first model invocation succeeds; it is the second one — after
ToolCallingAdvisor has run the tool and rebuilt the request from the
conversation history — that fails. Since the two requirements are
mutually exclusive, no unconditional behavior can satisfy both.

Change

The replay becomes configurable rather than unconditional:

  • OpenAiChatOptions.replayReasoningContent defaults to null, which
    keeps today's behavior: the property is sent whenever it is present.
    Setting it to false stops the replay. The assistant turn itself is
    still sent as before — only the reasoning_content property is dropped.
  • Exposed as spring.ai.openai.chat.replay-reasoning-content.

Nothing changes for OpenAI itself (the property is only added when the
assistant message carries reasoning content) and nothing changes for
DeepSeek (the default stays on).

Tests

  • OpenAiChatModelTests.reasoningContentNotReplayedWhenOptedOut — builds
    a request from an assistant message carrying reasoning with the opt-out
    set, asserts the property is absent, and asserts the assistant content is
    still sent. It fails without the guard in createRequest (verified by
    removing the guard locally: the property reappears).
  • OpenAiChatOptions builder coverage: both mutate() and
    combineWith(...) carry the new option.
  • OpenAiChatPropertiesTests.chatReplayReasoningContentTest — the property
    binds, reaches toOptions(), and
    chatReplayReasoningContentDefaultsToReplaying pins that it stays unset
    by default.

Verification

./mvnw -pl models/spring-ai-openai -Dmaven.build.cache.enabled=false test
  Tests run: 228, Failures: 0, Errors: 0, Skipped: 0

./mvnw -pl auto-configurations/models/spring-ai-autoconfigure-model-openai \
    -Dmaven.build.cache.enabled=false test
  Tests run: 32, Failures: 0, Errors: 0, Skipped: 0

checkstyle, javaformat and enforcer are clean.

(Both commands are run from the repository root. The build cache is
disabled because it otherwise skips the surefire execution and the build
reports success without running any test.)

Documentation

One row added to the OpenAI chat property table in
spring-ai-docs/.../api/chat/openai-chat.adoc.

reasoning_content is echoed back on an assistant message so that
OpenAI-compatible reasoning endpoints which require it keep working:
DeepSeek's thinking mode returns HTTP 400 when an assistant turn that
carried reasoning arrives without it. Other endpoints return the
property but reject it on the way back, so Groq answers a follow-up
request built from such a history with

    400: 'messages.6' : for 'role:assistant' ...
    property 'reasoning_content' is unsupported

Both cannot be satisfied at once, so the replay becomes configurable:
OpenAiChatOptions.replayReasoningContent defaults to null, which keeps
sending the property whenever it is present, and a value of false
stops it. The property is also exposed as
spring.ai.openai.chat.replay-reasoning-content.

Closes spring-projects#6968

Signed-off-by: chengwudi1 <chengwudi1@users.noreply.github.com>
@chengwudi1
chengwudi1 force-pushed the fix-replay-reasoning-content branch from 37cff71 to c9db89f Compare September 14, 2026 04:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Groq GPT-OSS tool calling fails on subsequent request because reasoning_content is replayed by ToolCallingAdvisor

2 participants