Skip to content

Python: [Bug]: Ollama and Gemini chat clients leave the SDK stream open when the consumer stops early #9143

Closed

Description

@ankit2235

Description

OllamaChatClient and GeminiChatClient iterate the SDK stream with a bare async for inside their _stream() generators. When the consumer stops early (breaks out of the loop, exits async with stream, or calls stream.close()), the SDK stream is never closed, so its HTTP response stays open until the abandoned async generator is garbage collected.

This is the same problem #8762 reported for OpenAIChatCompletionClient, fixed in #8773 for that client only.

Expected: once the ResponseStream is closed, the SDK stream is closed too.
Actual: the SDK stream is still open right after the close returns.

The repro is offline: it replaces each SDK stream with an async generator that records when it is closed.

Code Sample

import asyncio
from unittest.mock import AsyncMock, MagicMock

from agent_framework import Message
from agent_framework_gemini import GeminiChatClient
from agent_framework_ollama import OllamaChatClient
from google.genai import types
from ollama._types import ChatResponse
from ollama._types import Message as OllamaMessage

closed = {}


def sdk_stream(name, chunks):
    # Stand-in for the SDK stream, which keeps the HTTP response open until it is closed.
    async def gen():
        try:
            for chunk in chunks:
                yield chunk
        finally:
            closed[name] = True

    closed[name] = False
    return gen()


async def stop_early(name, client):
    stream = client._inner_get_response(messages=[Message(role="user", contents=["hi"])], options={}, stream=True)
    async with stream:
        async for _ in stream:
            break  # the consumer stops after the first update
    print(f"{name}: SDK stream closed after the consumer stopped = {closed[name]}")


async def main():
    ollama_sdk = MagicMock()
    ollama_sdk.chat = AsyncMock(
        return_value=sdk_stream("ollama", [ChatResponse(message=OllamaMessage(role="assistant", content="hello"), model="m")])
    )
    await stop_early("ollama", OllamaChatClient(client=ollama_sdk, model="m"))

    genai = MagicMock()
    genai._api_client.vertexai = False
    genai._api_client._http_options.base_url = "https://generativelanguage.googleapis.com/"
    chunk = types.GenerateContentResponse(
        candidates=[types.Candidate(content=types.Content(role="model", parts=[types.Part(text="hello")]))]
    )
    genai.aio.models.generate_content_stream = AsyncMock(return_value=sdk_stream("gemini", [chunk]))
    await stop_early("gemini", GeminiChatClient(client=genai, model="gemini-2.5-flash"))


asyncio.run(main())

Error Messages / Stack Traces

Actual:
ollama: SDK stream closed after the consumer stopped = False
gemini: SDK stream closed after the consumer stopped = False

Expected:
ollama: SDK stream closed after the consumer stopped = True
gemini: SDK stream closed after the consumer stopped = True

Package Versions

agent-framework-core: 1.20.0, agent-framework-ollama: 1.0.0b261002, agent-framework-gemini: 1.0.0b261002 (main @ 245d428)

Python Version

Python 3.13

Additional Context

The repro calls _inner_get_response directly. Through the default function-invocation layer, the close is not forwarded to the provider stream yet; that part is tracked in #9099 (#9100). The two fixes are complementary.

AnthropicClient uses the same bare async for pattern. I left it out of this report because its unit tests mock messages.create with plain async generators, so the fix needs those mocks updated as well. Happy to follow up separately.

I have a fix with tests ready and will open a PR.

Activity

  1. added
    pythonUsage: [Issues, PRs], Target: Python
    triageUsage: [Issues], Target: All issues that still need to be triaged
    on Oct 7, 2026
  2. added
    reproducedUsage: [Issues], Target: all issues that can be reproduced by the triage workflow
    on Oct 7, 2026
  3. github-actions commented on Oct 7, 2026

    @github-actions
    Contributor

    🤖 Automated triage reproduction notes (agent-authored — trust but verify)

    Agent analysis

    Repro: python/packages/ollama/agent_framework_ollama/_chat_client.py::OllamaChatClient._inner_get_response and python/packages/gemini/agent_framework_gemini/_chat_client.py::GeminiChatClient._inner_get_response leave their SDK async iterables open when a consumer exits the framework stream after one update. Minimal repro: substitute an SDK async generator with a closure-recording finally, consume one update inside async with stream, then assert the closure flag immediately after context exit.

    • Failing test: python/packages/ollama/tests/test_ollama_chat_client.py, python/packages/gemini/tests/test_gemini_client.py
    • Files examined: python/packages/ollama/agent_framework_ollama/_chat_client.py, python/packages/gemini/agent_framework_gemini/_chat_client.py, python/packages/ollama/tests/test_ollama_chat_client.py, python/packages/gemini/tests/test_gemini_client.py
    • Tests run: test_streaming_closes_sdk_stream_when_consumer_stops_early (Ollama and Gemini), test_cmc_streaming, test_streaming_yields_text_chunks
    • Reported version: agent-framework-ollama 1.0.0b261002; agent-framework-gemini 1.0.0b261002
    • Current version: agent-framework-ollama 1.0.0b261002; agent-framework-gemini 1.0.0b261002
  4. added
    agentsUsage: [Issues, PRs], Target: Single agent
    and removed
    triageUsage: [Issues], Target: All issues that still need to be triaged
    on Oct 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

agentsUsage: [Issues, PRs], Target: Single agentpythonUsage: [Issues, PRs], Target: PythonreproducedUsage: [Issues], Target: all issues that can be reproduced by the triage workflow

Type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions