Repository navigation
Python: streaming Responses parser drops the final image of image_generation_call #8422
Description
Package: agent-framework-openai (affects agent-framework-foundry, which inherits the parser)
Versions checked: 1.10.0, 1.13.0, 1.14.3 (latest on PyPI) and main @ 6c3c58a4b2d8834ad3dce3d0b01ad10f7edaf352 — all affected
File: python/packages/openai/agent_framework_openai/_chat_client.py, _parse_chunk_from_openai
Summary
When the hosted image_generation tool is used with stream=True and partial_images is not
requested (the default, 0), the generated image never reaches the AgentResponse. The final
render is delivered by the Responses API on response.output_item.done as a completed
image_generation_call item whose result holds the base64 image, but the streaming parser has no
branch for that item. It only creates image_generation_tool_call / image_generation_tool_result
contents from response.image_generation_call.partial_image events, i.e. from progressive preview
frames. The non-streaming parser handles the completed item correctly, so streaming and
non-streaming are not at parity.
Consequences with the default tool configuration:
AgentResponse.messagescontains no image content (onlytext_reasoning/text).HistoryProviders persist a message with no reference to the image.- Any consumer built on the documented content types silently loses the image; the only way to get
it is to dig intoresponse.raw_representation(which is also what the Foundry docs sample does).
Reproduction
import asyncio
from agent_framework import Agent
from agent_framework_foundry import FoundryChatClient
from azure.identity import AzureCliCredential
async def main():
client = FoundryChatClient(
project_endpoint="<project endpoint>",
model="gpt-5", # same on gpt-5.4, gpt-5.6-terra
credential=AzureCliCredential(),
default_headers={"x-ms-oai-image-generation-deployment": "gpt-image-2"},
)
agent = Agent(
client=client,
instructions="You are a helpful assistant.",
tools=[client.get_image_generation_tool(model="gpt-image-2")], # no partial_images
)
stream = agent.run("generate an image of a black cat with yellow eyes", stream=True)
async for update in stream:
ev = update.raw_representation.raw_representation
if getattr(ev, "type", None) == "response.output_item.done" and ev.item.type == "image_generation_call":
print("provider sent final image, base64 length:", len(ev.item.result)) # ~2,000,000
final = await stream.get_final_response()
print([c.type for m in final.messages for c in m.contents])
# observed: ['text_reasoning', 'text']
# expected: [..., 'image_generation_tool_call', 'image_generation_tool_result', 'text']
asyncio.run(main())Raw SSE sequence observed from Foundry (identical on every model, no partial frames):
response.output_item.added item.type=image_generation_call status=in_progress
response.image_generation_call.in_progress
response.image_generation_call.generating
response.image_generation_call.completed
response.output_item.done item.type=image_generation_call status=completed result_len=2401164
response.completed output=[image_generation_call(result_len=2401164), message]
Passing partial_images=1 makes the image appear (via the partial_image handler), which confirms
the diagnosis, but it persists a preview frame rather than the final render and depends on the
service emitting previews.
Code
Non-streaming path maps the completed item (works):
agent-framework/python/packages/openai/agent_framework_openai/_chat_client.py
Lines 2985 to 3007 in 6c3c58a
Streaming path only maps partial_image events:
agent-framework/python/packages/openai/agent_framework_openai/_chat_client.py
Lines 3441 to 3471 in 6c3c58a
Streaming response.output_item.done handles reasoning, mcp_call, web_search_call,
file_search_call, shell items, custom_tool_call, tool_search_call — but not
image_generation_call:
agent-framework/python/packages/openai/agent_framework_openai/_chat_client.py
Lines 3568 to 3623 in 6c3c58a
response.output_item.added lists ImageGenerationCall in its comment of handled types but has no
case for it either (line 3295).
Note the drop is silent: because the outer match does hit case "response.output_item.done", the
image item never reaches the case _: fallback, so not even the "Unparsed event of type ..." debug
line is emitted for it. With agent_framework.openai at DEBUG the only trace is the earlier
output_item.added event being reported as unparsed:
_chat_client.py:_parse_chunk_from_openai - Unparsed event of type: response.output_item.added: ResponseOutputItemAddedEvent(item=ImageGenerationCall(id='ig_...', result=None, status='in_progress', ...
_chat_client.py:_parse_chunk_from_openai - Unparsed event of type: response.image_generation_call.in_progress: ...
_chat_client.py:_parse_chunk_from_openai - Unparsed event of type: response.image_generation_call.generating: ...
_chat_client.py:_parse_chunk_from_openai - Unparsed event of type: response.image_generation_call.completed: ...
(nothing for response.output_item.done, which carries the 2.5 MB result)
Origin
- The streaming image handling was introduced by PR Python: Add tool call/result content types and update connectors and samples #2971 "Add tool call/result content types and
update connectors and samples" (merged 2026-01-08,3f7ea350dc), in
python/packages/core/agent_framework/openai/_responses_client.py. It added thepartial_image
handler and the accompanying sampleopenai_responses_client_streaming_image_generation.py, which
setspartial_images, so the final-item case was never exercised. - PR Python: [BREAKING] Python: Provider-leading client design & OpenAI package extraction #4818 (2026-03-25,
5e056b672e) moved the parser toagent_framework_openai/_chat_client.py
unchanged in this respect. - Streaming support itself came in PR Python: OpenAI Responses Image Generation Stream Support, Sample and Unit Tests #1853 (merged 2025-11-11), "with partial images" — the
partial_imagehandler and its sample were written around preview frames from the start. - Bisecting every commit of
_chat_client.pysince then: no revision has ever handled
image_generation_callonresponse.output_item.done. This is a gap since the feature was added,
not a regression from a specific release. It surfaced for us in the second week of September 2026
because the Foundry Responses endpoint used to emit onepartial_imageframe even without
partial_images(which the handler above picked up) and stopped doing so; the same code, same
dependency versions and same deployments that returned images up to 2026-09-08 return none now.
The documented contract (OpenAI image generation guide:partial_images0 → only the final image,
delivered on the completed item) was never handled by the streaming parser.
Also verified: the stored response (responses.retrieve) returns the image item with
"result": null, and no include value restores it, so the streamed output_item.done event is
the only moment the bytes are available. Dropping it there loses the image for good.
Proposed fix
Add an image_generation_call branch to the response.output_item.done handler mirroring the
non-streaming mapping (call + result content, result as a data: URI). Patch against main:
case "response.output_item.done":
done_item = event.item
- if getattr(done_item, "type", None) == "reasoning":
+ if getattr(done_item, "type", None) == "image_generation_call":
+ image_result = getattr(done_item, "result", None)
+ if image_result:
+ image_id = getattr(done_item, "id", None)
+ contents.append(
+ Content.from_image_generation_tool_call(
+ image_id=image_id, raw_representation=done_item
+ )
+ )
+ contents.append(
+ Content.from_image_generation_tool_result(
+ image_id=image_id,
+ outputs=Content.from_uri(
+ uri=f"data:{detect_media_type_from_base64(data_str=image_result) or 'image/png'}"
+ f";base64,{image_result}",
+ raw_representation=image_result,
+ ),
+ raw_representation=done_item,
+ )
+ )
+ elif getattr(done_item, "type", None) == "reasoning":
encrypted_content = getattr(done_item, "encrypted_content", None)When partial_images is also set, consumers already keep the last result per image_id, so the
final render simply supersedes the preview frames.
Activity
CristinaStn commentedon Sep 16, 2026
Proof from a real run (added after filing)
agent_framework.openailogger at DEBUG during one streamed image request. The parser reports theoutput_item.addedimage item and the three progress events as unparsed; theoutput_item.doneevent that carries the result produces no line at all (it matches the done case and falls through):
The same run as persisted by
CosmosHistoryProvider(agent-framework-azure-cosmos): the assistantmessage holds only the reasoning item and an empty text — nothing references the image that was
generated (encrypted reasoning payload and Cosmos system fields trimmed):
{ "session_id": "87e32ef8-82cc-4ed7-a3c6-7846b863f6e0", "source_id": "azure_cosmos_history", "message": { "type": "message", "role": "assistant", "contents": [ { "type": "text_reasoning", "protected_data": "gAAAAABqqmm4dSw8XJEny1_h2nwGHGTXKDnq...", "id": "rs_0fe397a81f463db1006aaa69b60c0c8195b8d16b66df336335", "additional_properties": {} }, { "type": "text", "text": "", "additional_properties": {} } ], "author_name": "GeneralAssistant", "additional_properties": {} } }Expected:
image_generation_tool_callandimage_generation_tool_resultcontents forig_0fe397a81f463db1006aaa6b74afac819599a01d90a2e99ababetween the reasoning and the text, as thenon-streaming parser would produce.
anishmehta24 commentedon Sep 16, 2026
Thanks for the thorough report, the diagnosis matches what I see in the code. I've put together a fix that adds the missing
image_generation_callbranch to the streamingresponse.output_item.donehandler; rather than duplicating the mapping, it factors the existing non-streaming logic into a small shared helper so both paths emit identicalimage_generation_tool_call/image_generation_tool_resultcontents (same mime detection anddata:URI). It includes unit tests for the done event with and without a result, plus a parity test comparing streaming and non-streaming output. Thepartial_imagepath is untouched, so withpartial_images > 0the final render simply follows the previews under the sameimage_id. PR incoming shortly.agooll commentedon Sep 16, 2026
Thanks for the detailed diagnosis and proposed patch.
Before I start working on this, are you planning to submit the fix yourself? If not, and the maintainers agree with the proposed direction, I’d be happy to add focused regression tests for the streaming image-generation path and implement the smallest compatible fix.
CristinaStn commentedon Sep 16, 2026
No, I'm not planning to submit a PR myself — please go ahead. Note that Anish Mehta (@anishmehta24) mentioned above that a fix with tests is already in progress, so it may be worth coordinating with them rather than duplicating the work.
anishmehta24 commentedon Sep 16, 2026
agooll the PR is already up: #8424 (streaming done handler + shared helper, three tests incl. a streaming/non-streaming parity check). Happy to take review feedback there if you spot anything missing.
fix(openai): emit image_generation_call result from streaming output_…