Repository navigation
Python: [Bug]: Ollama and Gemini chat clients leave the SDK stream open when the consumer stops early #9143
Description
Description
OllamaChatClient and GeminiChatClient iterate the SDK stream with a bare async for inside their _stream() generators. When the consumer stops early (breaks out of the loop, exits async with stream, or calls stream.close()), the SDK stream is never closed, so its HTTP response stays open until the abandoned async generator is garbage collected.
This is the same problem #8762 reported for OpenAIChatCompletionClient, fixed in #8773 for that client only.
Expected: once the ResponseStream is closed, the SDK stream is closed too.
Actual: the SDK stream is still open right after the close returns.
The repro is offline: it replaces each SDK stream with an async generator that records when it is closed.
Code Sample
import asyncio
from unittest.mock import AsyncMock, MagicMock
from agent_framework import Message
from agent_framework_gemini import GeminiChatClient
from agent_framework_ollama import OllamaChatClient
from google.genai import types
from ollama._types import ChatResponse
from ollama._types import Message as OllamaMessage
closed = {}
def sdk_stream(name, chunks):
# Stand-in for the SDK stream, which keeps the HTTP response open until it is closed.
async def gen():
try:
for chunk in chunks:
yield chunk
finally:
closed[name] = True
closed[name] = False
return gen()
async def stop_early(name, client):
stream = client._inner_get_response(messages=[Message(role="user", contents=["hi"])], options={}, stream=True)
async with stream:
async for _ in stream:
break # the consumer stops after the first update
print(f"{name}: SDK stream closed after the consumer stopped = {closed[name]}")
async def main():
ollama_sdk = MagicMock()
ollama_sdk.chat = AsyncMock(
return_value=sdk_stream("ollama", [ChatResponse(message=OllamaMessage(role="assistant", content="hello"), model="m")])
)
await stop_early("ollama", OllamaChatClient(client=ollama_sdk, model="m"))
genai = MagicMock()
genai._api_client.vertexai = False
genai._api_client._http_options.base_url = "https://generativelanguage.googleapis.com/"
chunk = types.GenerateContentResponse(
candidates=[types.Candidate(content=types.Content(role="model", parts=[types.Part(text="hello")]))]
)
genai.aio.models.generate_content_stream = AsyncMock(return_value=sdk_stream("gemini", [chunk]))
await stop_early("gemini", GeminiChatClient(client=genai, model="gemini-2.5-flash"))
asyncio.run(main())Error Messages / Stack Traces
Actual:
ollama: SDK stream closed after the consumer stopped = False
gemini: SDK stream closed after the consumer stopped = False
Expected:
ollama: SDK stream closed after the consumer stopped = True
gemini: SDK stream closed after the consumer stopped = TruePackage Versions
agent-framework-core: 1.20.0, agent-framework-ollama: 1.0.0b261002, agent-framework-gemini: 1.0.0b261002 (main @ 245d428)
Python Version
Python 3.13
Additional Context
The repro calls _inner_get_response directly. Through the default function-invocation layer, the close is not forwarded to the provider stream yet; that part is tracked in #9099 (#9100). The two fixes are complementary.
AnthropicClient uses the same bare async for pattern. I left it out of this report because its unit tests mock messages.create with plain async generators, so the fix needs those mocks updated as well. Happy to follow up separately.
I have a fix with tests ready and will open a PR.
Metadata
Metadata
Labels
Type
Projects
- StatusShow more project fieldsDone
Activity
github-actions commentedon Oct 7, 2026
🤖 Automated triage reproduction notes (agent-authored — trust but verify)
Agent analysis
Repro:
python/packages/ollama/agent_framework_ollama/_chat_client.py::OllamaChatClient._inner_get_responseandpython/packages/gemini/agent_framework_gemini/_chat_client.py::GeminiChatClient._inner_get_responseleave their SDK async iterables open when a consumer exits the framework stream after one update. Minimal repro: substitute an SDK async generator with a closure-recordingfinally, consume one update insideasync with stream, then assert the closure flag immediately after context exit.python/packages/ollama/tests/test_ollama_chat_client.py, python/packages/gemini/tests/test_gemini_client.pyagent-framework-ollama 1.0.0b261002; agent-framework-gemini 1.0.0b261002agent-framework-ollama 1.0.0b261002; agent-framework-gemini 1.0.0b261002