Skip to content

Hold Spring AI at 2.0.1: openai-java 4.66+ no longer closes a cancelled stream - #48

Merged
codeministry merged 1 commit into
mainfrom
chore/spring-ai-2.0.2
Oct 26, 2026
Merged

codeministry merged 1 commit into
mainfrom
chore/spring-ai-2.0.2

Conversation

@codeministry

Copy link
Copy Markdown
Owner

Why Spring AI stays at 2.0.1

Spring AI 2.0.2 breaks stopping a streaming chat turn: four tests fail (ChatStreamTest.stopCancelsTheModelCallAndKeepsThePartialAnswerMarkedStopped, both stop-on-delete tests in ChatConversationTest, ChatTurnDeadlineTest.aTurnPastItsDeadlineIsStoppedAndFreesItsThread). In each case the model stub never sees its connection closed: the turn ends on our side, but the model call keeps running to the end.

Root cause: an upstream regression in openai-java 4.66.0

  • Spring AI 2.0.2 bumps the OpenAI SDK from 4.49.0 to 4.80.0 (spring-projects/spring-ai@351c272, #7170). Spring AI's own streaming code did not change in a way that matters: OpenAiChatModel.internalStream still does sink.onDispose(response::close).
  • openai-java 4.66.0 (openai/openai-java#1027, commit 1441ba1) reordered StreamResponse.close() in StreamHandler.kt: the transport response is now closed before the BufferedReader. Before, reader.close() ran first and waited on the reader's lock until the in-flight readLine() on the stream-handler thread returned, so the OkHttp body was closed with no read in progress.
  • We cancel from another thread (takeUntilOther on the stop signal or the deadline), while the stream-handler thread is blocked in a socket read. Closing the OkHttp body concurrently makes ChunkedSource.close() → discard() → Okio AsyncTimeout.enter() throw IllegalStateException: Unbalanced enter/exit. That skips noNewExchanges()/responseBodyComplete(), so the connection is neither released nor closed, and the exception is swallowed by the whenComplete in AsyncStreamResponse.close().

Evidence

  • Spring AI 2.0.2 held fixed, only com.openai:openai-java-core forced: bisected over every release from 4.49.0 to 4.80.0. 4.65.0 green, 4.66.1 red (#1027 is tagged in v4.66.0, which never reached Maven Central; 4.66.1 is the first published release with it); the same four tests flip, the other 16 stay green. Still red on today's 4.92.0, and StreamHandler.kt on openai-java main is unchanged since #1027.
  • A throwaway probe wrapping the SDK's HttpClient logged the close() calls: with 4.65.0 the reader closes, then the response, and the stub sees the reset; with 4.80.0 and 4.92.0 response.close() throws IllegalStateException: Unbalanced enter/exit and the stub writes every remaining chunk.
  • Plain OkHttp 4.12.0, no SDK: closing a ResponseBody while another thread is blocked reading it throws the same exception and leaves the connection in the pool; closing the reader first, or Call.cancel(), closes the socket at once.
  • Cancelling on the stream-handler thread itself (e.g. take(3) inside onNext) still closes the connection with 4.80.0. A stop or a deadline cannot cancel on that thread: it is blocked waiting for the next chunk.

The app code is not at fault: disposing the Flux from ChatModel.stream() is Spring AI's way to cancel a call, and the tests state user-visible guarantees (stop ends the model call, deadline frees the thread, delete stops the streaming turn), so neither is changed.

What this PR changes

No code or test changes.

Verification

  • ./gradlew -p backend check green on this branch.
  • renovate-config-validator renovate.json: valid.

Spring AI 2.0.2 upgrades openai-java from 4.49.0 to 4.80.0. Since
openai-java 4.66.0 (openai/openai-java#1027) StreamResponse.close()
closes the transport before the reader. Cancelling a stream from another
thread, which a stop or the turn deadline does, then closes the OkHttp
body under an in-flight read: Okio throws "Unbalanced enter/exit", the
connection is never released or closed, and the model call runs to its
end. Bisected with Spring AI 2.0.2 held fixed: openai-java 4.65.0 passes,
4.66.1 fails the same four chat tests, and 4.92.0 still fails.

Note the reason next to springAi, and move Spring AI out of the backend
group so a held version no longer turns that group red.
@codeministry codeministry self-assigned this Oct 26, 2026
@codeministry
codeministry merged commit 10375a8 into main Oct 26, 2026
6 checks passed
@codeministry
codeministry deleted the chore/spring-ai-2.0.2 branch October 26, 2026 17:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant