Repository navigation
Python: [Feature]: Explicit append/replace semantics for streamed content merging #8910
Description
Description
Problem
When streamed updates are aggregated (ChatResponse.from_updates, AgentResponse, history providers), core has to guess whether an incoming chunk is a delta to append or a snapshot that replaces what came before. For code-interpreter inputs and outputs, _merge_content_item_lists infers this with prefix comparison. Text is always appended through Content.__add__. This inference is the root cause of:
- Python: [Bug]: Streamed code_interpreter code is corrupted when a delta is a prefix of the accumulated code #8903: deltas that happen to be a prefix of the accumulated text are dropped, and the final snapshot is then concatenated.
- Python: [Bug]: Streaming response aggregation is O(n^2) in the number of text chunks #8908: folding with repeated
__add__is O(n^2) in the number of chunks. - Earlier, Python: CosmosHistoryProvider Code interpreter tool calls are saved chunk by chunk #5793 / Python: coalesce code interpreter history chunks #5801: chunks were persisted one by one until the coalescing heuristic was added.
Providers already know which kind each event is (for example Responses *.delta vs *.done), but that knowledge is lost before core merges.
Proposal
- Let providers mark each mergeable content with an explicit merge mode (
append|replace), as a first-class field or a reserved metadata key in line with ADR 0044 ([BREAKING] Python: Consolidate content identity and protocol metadata #8886). - Core merges by that mode only: append concatenates, replace overwrites, and there are no prefix heuristics. Unmarked content keeps today's append behavior.
- Aggregation collects each run of mergeable chunks and builds one
Contentat the end ("".join(...), one flattened raw-representation list, one props merge), which makes it linear. - Add a property-based contract test for every mergeable content type: split a known value into random chunks, optionally followed by a snapshot, and assert it reassembles exactly for both
from_updatesand streamingget_final_response().
Expected behavior
Aggregated content equals what the service produced, regardless of how the text happens to be chunked, and aggregation cost grows linearly.
Alternatives considered
Improving the prefix heuristic. Rejected, because any heuristic misfires on repetitive content such as code.
Code Sample
# Provider side (e.g. OpenAI Responses client)
Content.from_code_interpreter_tool_call(call_id=cid, inputs=[Content.from_text(event.delta)],
merge="append") # *.delta
Content.from_code_interpreter_tool_call(call_id=cid, inputs=[Content.from_text(event.code)],
merge="replace") # *.done
# Core: no inference
merged = existing.replace_with(incoming) if incoming.merge == "replace" else existing.append(incoming)Language/SDK
Python
Metadata
Metadata
Labels
Type
Projects
- StatusShow more project fieldsDone
Activity