Repository navigation
Hold Spring AI at 2.0.1: openai-java 4.66+ no longer closes a cancelled stream - #48
Merged
Merged
Conversation
Spring AI 2.0.2 upgrades openai-java from 4.49.0 to 4.80.0. Since openai-java 4.66.0 (openai/openai-java#1027) StreamResponse.close() closes the transport before the reader. Cancelling a stream from another thread, which a stop or the turn deadline does, then closes the OkHttp body under an in-flight read: Okio throws "Unbalanced enter/exit", the connection is never released or closed, and the model call runs to its end. Bisected with Spring AI 2.0.2 held fixed: openai-java 4.65.0 passes, 4.66.1 fails the same four chat tests, and 4.92.0 still fails. Note the reason next to springAi, and move Spring AI out of the backend group so a held version no longer turns that group red.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why Spring AI stays at 2.0.1
Spring AI 2.0.2 breaks stopping a streaming chat turn: four tests fail (
ChatStreamTest.stopCancelsTheModelCallAndKeepsThePartialAnswerMarkedStopped, both stop-on-delete tests inChatConversationTest,ChatTurnDeadlineTest.aTurnPastItsDeadlineIsStoppedAndFreesItsThread). In each case the model stub never sees its connection closed: the turn ends on our side, but the model call keeps running to the end.Root cause: an upstream regression in openai-java 4.66.0
OpenAiChatModel.internalStreamstill doessink.onDispose(response::close).StreamResponse.close()inStreamHandler.kt: the transport response is now closed before theBufferedReader. Before,reader.close()ran first and waited on the reader's lock until the in-flightreadLine()on the stream-handler thread returned, so the OkHttp body was closed with no read in progress.takeUntilOtheron the stop signal or the deadline), while the stream-handler thread is blocked in a socket read. Closing the OkHttp body concurrently makesChunkedSource.close()→discard()→ OkioAsyncTimeout.enter()throwIllegalStateException: Unbalanced enter/exit. That skipsnoNewExchanges()/responseBodyComplete(), so the connection is neither released nor closed, and the exception is swallowed by thewhenCompleteinAsyncStreamResponse.close().Evidence
com.openai:openai-java-coreforced: bisected over every release from 4.49.0 to 4.80.0. 4.65.0 green, 4.66.1 red (#1027 is tagged in v4.66.0, which never reached Maven Central; 4.66.1 is the first published release with it); the same four tests flip, the other 16 stay green. Still red on today's 4.92.0, andStreamHandler.kton openai-javamainis unchanged since #1027.HttpClientlogged theclose()calls: with 4.65.0 the reader closes, then the response, and the stub sees the reset; with 4.80.0 and 4.92.0response.close()throwsIllegalStateException: Unbalanced enter/exitand the stub writes every remaining chunk.ResponseBodywhile another thread is blocked reading it throws the same exception and leaves the connection in the pool; closing the reader first, orCall.cancel(), closes the socket at once.take(3)insideonNext) still closes the connection with 4.80.0. A stop or a deadline cannot cancel on that thread: it is blocked waiting for the next chunk.The app code is not at fault: disposing the
FluxfromChatModel.stream()is Spring AI's way to cancel a call, and the tests state user-visible guarantees (stop ends the model call, deadline frees the thread, delete stops the streaming turn), so neither is changed.What this PR changes
gradle/libs.versions.toml: a comment next tospringAisaying why it is held at 2.0.1 and when to retry.renovate.json: Spring AI gets its own pull request instead of riding in "backend (minor and patch)". Held inside the group, it turns the whole group red every week (as in Update backend (minor and patch) #40 and chore(deps): Renovate updates of 2026-10-26 combined #47); on its own, its CI result says whether the next release fixed it.No code or test changes.
Verification
./gradlew -p backend checkgreen on this branch.renovate-config-validator renovate.json: valid.