Skip to content

Python: make streamed text coalescing linear - #9058

Closed
Suhas Aggarwal (suhasagg) wants to merge 2 commits into
microsoft:mainfrom
suhasagg:fix-8908-linear-stream-aggregation
Closed

Suhas Aggarwal (suhasagg) wants to merge 2 commits into
microsoft:mainfrom
suhasagg:fix-8908-linear-stream-aggregation

Conversation

@suhasagg

Copy link
Copy Markdown
Contributor

Motivation & Context

Streaming responses may contain a large number of consecutive text chunks. _coalesce_text_content() previously merged those chunks incrementally using Content.__add__.

Each addition reconstructs the accumulated text and associated metadata. As the accumulated content grows, repeatedly merging it causes increasingly large copies and can result in quadratic behavior for long streaming responses.

This change makes ordinary text-stream coalescing linear by collecting compatible consecutive text chunks and aggregating them in a single pass.

The change is intentionally scoped to text content. text_reasoning continues to use the existing merge behavior because it has additional semantic boundaries involving IDs, protected data, and AdditionItemMismatch.

Description & Review Guide

  • What are the major changes?

    • Collect compatible consecutive text content into runs instead of repeatedly applying Content.__add__.
    • Join text fragments once using "".join(...).
    • Preserve _MODEL_OUTPUT_KIND_KEY boundaries so different logical output kinds are not incorrectly coalesced.
    • Preserve existing additional_properties precedence, where earlier chunks win for duplicate keys.
    • Preserve annotation ordering and existing raw_representation behavior.
    • Keep the existing text_reasoning coalescing path unchanged.
    • Add a regression test that instruments Content.__add__ and verifies that coalescing 100 text chunks performs zero repeated Content.__add__ calls. The previous implementation performed 99.
  • What is the impact of these changes?

    • Removes the repeated growing-content merge from ordinary streamed text aggregation.
    • Makes the text aggregation work scale linearly with the amount of streamed content.
    • Does not change the public API or intended coalescing semantics.
    • Existing core type tests continue to pass.
  • What do you want reviewers to focus on?

    • Semantic equivalence of the optimized text path, particularly property precedence, annotations, raw representations, and _MODEL_OUTPUT_KIND_KEY boundaries.
    • Whether keeping text_reasoning on the existing merge path is the appropriate scope for this fix.
    • The regression test as a deterministic guard against reintroducing repeated Content.__add__ aggregation.

Related Issue

Fixes #8908

Contribution Checklist

  • The code builds clean without any errors or warnings
  • All unit tests pass, and I have added new tests where possible
  • The PR follows the [Contribution Guidelines](https://github.com/microsoft/agent-framework/blob/main/CONTRIBUTING.md)
  • This PR is linked to an issue and there is no other open PR for this issue (see Related Issue above).
  • This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.

Validation

Full core type test suite:

377 passed

Static checks:

ruff check: All checks passed!
ruff format --check: 2 files already formatted
git diff --check: clean

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Coalesced annotations now unexpectedly alias the first input chunk.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)
What changed in this PR

Optimizes streamed text coalescing from quadratic to linear complexity.

Changes:

  • Aggregates compatible text chunks in one pass.
  • Preserves output-kind boundaries and merge semantics.
  • Adds a regression test preventing repeated Content.__add__ calls.
File Description
python/​packages/​core/​agent_framework/​_types.py Implements linear text coalescing.
python/​packages/​core/​tests/​core/​test_types.py Tests the optimized aggregation path.

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

Comment thread python/packages/core/agent_framework/_types.py Outdated
@eavanvalkenburg

Copy link
Copy Markdown
Member

Suhas Aggarwal (@suhasagg) Thanks for the contribution. This implements the same change as #8953, so we'll continue the work there and close this duplicate.

This branch was previously deployed

1 inactive deployment
github-app-auth — 3e8194ad Deployed Oct 5, 2026 by suhasagg via add_label #24440
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python: [Bug]: Streaming response aggregation is O(n^2) in the number of text chunks

3 participants