Repository navigation
Python: [Bug]: Streaming response aggregation is O(n^2) in the number of text chunks #8908
Description
Description
_coalesce_text_content (agent_framework/_types.py) folds streamed text chunks with first_new_content += content. Each Content.__add__ builds a new text string, merges additional_properties, and rebuilds raw_representation via self_list + other_list, so aggregating n chunks costs O(n^2).
Measured with ChatResponse.from_updates (one text chunk per update, each with a small raw_representation):
| chunks | 2,000 | 4,000 | 8,000 | 16,000 |
|---|---|---|---|---|
| time | 0.014 s | 0.036 s | 0.134 s | 0.532 s |
Doubling the input roughly quadruples the time. Long reasoning or tool-heavy streams, and code paths that re-aggregate per iteration, pay this repeatedly.
Suggested fix: collect each run of mergeable chunks and build one Content at the end, using "".join(texts), a single flattened raw-representation list, and one props merge.
Code Sample
import time
from agent_framework import Content, ChatResponse, ChatResponseUpdate
for n in (2000, 4000, 8000, 16000):
ups = [ChatResponseUpdate(role="assistant", message_id="m",
contents=[Content.from_text("tok ", raw_representation={"i": i})])
for i in range(n)]
t = time.perf_counter(); ChatResponse.from_updates(ups)
print(n, round(time.perf_counter() - t, 3))Error Messages / Stack Traces
Package Versions
agent-framework-core 1.19.0 (main @ 853c456)
Python Version
Python 3.11
Additional Context
Found during a code review of main @ 853c456 (2026-09-30); the repro above was run against that commit.
Metadata
Metadata
Labels
Type
Projects
- Status
Activity
github-actions commentedon Sep 30, 2026
🤖 Automated triage reproduction notes (agent-authored — trust but verify)
Agent analysis
The bug is in
python/packages/core/agent_framework/_types.py::_coalesce_text_contentaround line 2300, where consecutive text chunks trigger repeatedContentaddition and copying of the accumulated raw-representation list. Minimal repro: create 100ChatResponseUpdateinstances containing text and non-nullraw_representation, instrument_combine_raw_representations, and callChatResponse.from_updates; 4,951 elements are processed instead of linear work.python/packages/core/tests/core/test_types.py::test_chat_response_from_updates_coalesces_raw_representations_in_linear_work1.19.01.19.0he-yufeng commentedon Sep 30, 2026
Verified on current main (7634356). The mechanism is as described:
_coalesce_text_contentfolds chunks withfirst_new_content += content, and everyContent.__add__rebuilds the text string, re-mergesadditional_properties, and re-concatenatesraw_representationlists, so n chunks cost O(n^2).Repro on main with the same script from the issue:
Doubling the input roughly quadruples the time, matching the reported table.
Two boundaries the fix has to preserve, both living in the current
+=path: theAdditionItemMismatchsplit fortext_reasoningchunks with conflicting ids, and the_MODEL_OUTPUT_KIND_KEYsplit for text chunks. A join-based fast path needs to keep both segment boundaries intact, build the text with one"".joinper segment, flattenraw_representationin a single list concat, and mergeadditional_propertiesonce per segment instead of once per chunk.Happy to open a PR along those lines if the direction sounds right.
he-yufeng commentedon Oct 1, 2026
Opened draft #8953 with the one-pass fold discussed above. Both split boundaries are preserved (the
model_output_kindsplit for text, the id and reasoning-text/summary splits for text_reasoning), and one extra semantic turned up while reproducing the old path: the fold's initialdeepcopydiscards the run head'sraw_representationvia_SHALLOW_COPY_FIELDS, so the new fold deliberately skips the head when flattening raws. An oracle test folds the same mixed streams with the old repeated-+= implementation and asserts identical output. Same-machine benchmark: 16,000 chunks 0.302s -> 0.011s.Python: fix(core): fold streamed text runs in one pass instead of rep…
Python: fix(core): fold streamed text runs in one pass instead of rep…