Repository navigation
Python: [Feature]: Support Responses API server-side compaction (context_management) with store=false #8890
Description
Description
The Responses API supports server-side compaction via context_management=[{"type": "compaction", "compact_threshold": N}], and it works with store=false. When the context crosses the threshold the service emits an encrypted compaction output item. For stateless (input-array) chaining, the caller appends that item to the next request's input and may drop the items that came before it.
Today OpenAIChatClient and FoundryChatClient (which inherits from RawOpenAIChatClient) only half support this:
- Request side works.
context_managementpasses through_prepare_optionstoresponses.create(...), so the service does compact. - The compaction item is dropped. Neither the non-streaming parser nor the
response.output_item.donestreaming handler knows thecompactionitem type, so it is logged as unparsed and never reachesChatResponse/ chat history. - Nothing is replayed. With
store=false, the next turn resends the full uncompacted history without the compaction item. The service has to compact again, and no tokens are saved. context_managementis not declared onOpenAIChatOptions/FoundryChatOptions.
Expected behavior
- A
compactionoutput item (streaming and non-streaming) is preserved as content in the response and in history, including itsencrypted_content. - With
store=false, the next request replays the latest compaction item as an input item and omits the messages that came before it (keeping system/developer messages). This is what makes compaction actually save tokens in stateless mode. - With
previous_response_id/ conversation continuation, the compaction item is not re-sent (the service already has it) and no pruning happens, as the docs require. context_managementis available as a typed option.
Alternatives considered
The framework's compaction_strategy does client-side compaction, which is a different feature and doesn't benefit from the service's encrypted compaction state.
Code Sample
from agent_framework import Agent, InMemoryHistoryProvider
from agent_framework_foundry import FoundryChatClient
agent = Agent(
client=FoundryChatClient(),
context_providers=[InMemoryHistoryProvider()],
default_options={
"store": False,
"context_management": [{"type": "compaction", "compact_threshold": 200000}],
},
)
session = agent.create_session()
# Long-running conversation: once the service compacts, later turns should send only
# the latest compaction item plus newer messages instead of the full history.
while True:
await agent.run(get_next_user_input(), session=session)Language/SDK
Python
Metadata
Metadata
Type
Projects
- Status
Activity
eavanvalkenburg commentedon Oct 15, 2026
/plan
github-actions commentedon Oct 15, 2026
Summary
The issue reports that
OpenAIChatClient(andFoundryChatClient, which wraps it) don't support the Responses API's server-side compaction feature end-to-end. Server-side compaction is a service feature: once a conversation gets long, OpenAI's Responses API can automatically "compact" (summarize) older turns server-side and hand back a specialcompactionitem. When the app sendsstore=false(it doesn't ask OpenAI to retain the conversation), the app is responsible for re-sending thatcompactionitem on the next turn instead of the full raw history, which is what actually saves tokens/cost.Investigation
I read the relevant client code in
python/packages/openai/agent_framework_openai/_chat_client.pyand confirmed each claim in the issue against current (not any PR's) code:context_managementisn't a declared option onOpenAIChatOptions(confirmed — no mention of it anywhere in the option TypedDicts), but because these option objects aren't runtime-validated, any extra key in the options dict still flows through_prepare_optionsinto the raw request sent to the API. So callers can pass it today, just without IDE/type-checker support._parse_response_from_openai), the bigmatch item.type:block has cases formessage,reasoning,function_call,mcp_call,image_generation_call, etc., but no case for"compaction"— it falls through to the catch-allcase _:which just logs"Unparsed output of type"and discards it. The streaming handler forresponse.output_item.donehas the same gap (cases forreasoning,message,mcp_call, search/shell/computer/image items — nocompactioncase)._prepare_message_for_openai(which turns a storedMessage/Contentback into a Responses API input item) and the coreContenttype (ContentTypeliteral inagent_framework/_types.py), there's no"compaction"content type at all, so there's nothing to even attach the item to, let alone replay or use to prune older history.context_managementdoesn't appear onOpenAIChatOptions(used directly byOpenAIChatClient, and aliased asFoundryChatOptionsforFoundryChatClient).I checked the OpenAI Python SDK's types directly (
response_compaction_item.py,response_output_item.py,response_create_params.pyinopenai/openai-python) and confirmed the SDK already modelsResponseCompactionItem(withid,encrypted_content,type: "compaction") as part of theResponseOutputItemunion, andcontext_managementas a typedResponseCreateParamsBasefield — so the SDK already exposes everything needed; the gap is entirely in this framework's client code.There's already a closely related pattern in this same file for OpenAI's encrypted reasoning items (
_prepare_reasoning_items_for_openai,protected_data, theinclude=["reasoning.encrypted_content"]opt-in, and the stateless-replay validation in_validate_reasoning_groups_for_stateless_replay). Server-side compaction is conceptually the same shape of problem (an opaque, encrypted, server-issued item that must be preserved and selectively replayed in stateless mode), so the framework already has the pattern and plumbing to extend — this isn't a totally new kind of integration.I noted PR #8891 is open and linked to this issue, but per the workflow rules I did not inspect its diff/approach; the plan below is derived independently from the current repository code and the SDK's type definitions.
Recommendation
Code changes are warranted. This is a real framework gap, not an application-specific scenario: anyone using
OpenAIChatClientorFoundryChatClientwith the Responses API in stateless (store=false) mode on a long-running conversation hits this — the compaction item is silently dropped, so the service has to recompact every turn and no tokens are ever saved, defeating the point of the feature. There's no existing workaround available to callers since this is internal request/response translation logic they don't control. Client-sidecompaction_strategy(mentioned as "alternatives considered" in the issue) is a different, already-supported feature and doesn't reduce redundant encrypted state round-trips the service-side feature avoids.The scope is proportionate to the benefit: it needs a new content type plus handling in a small, well-isolated set of methods that already have an analogous pattern (reasoning items), not a redesign.
Proposed plan
compactioncontent type. ExtendContentTypeinagent_framework/_types.pywith a"compaction"value and add aContentconstructor (e.g.Content.from_compaction(id=..., protected_data=...)), mirroring howfrom_text_reasoningcarriesprotected_data/additional_propertiesfor encrypted payloads._chat_client.py, add acase "compaction":branch to the non-streaming_parse_response_from_openaimatch block, and a corresponding branch in the streamingresponse.output_item.donehandler, each converting the SDK'sResponseCompactionItem/event item into the newContent, preservingencrypted_contentandid._prepare_message_for_openaito serialize a stored compactionContentback into a{"type": "compaction", "id": ..., "encrypted_content": ...}input item, similar to how reasoning items are reconstructed._prepare_messages_for_openai/_prepare_options, whenrequest_uses_service_side_storageisFalse(noprevious_response_id/conversation), scan for the latest compaction content in the message history; if present, send only that item plus later messages (keeping system/developer messages), instead of the full prior history. When continuation (previous_response_id/conversation) is used, skip this entirely — per the docs, the service already has the item and no pruning should happen.context_managementtoOpenAIChatOptions(whichFoundryChatOptionsalready aliases) with a type matching the SDK'sContextManagemententries (type,compact_threshold).compactionitem (includingencrypted_contentround-trip); serialization back to a request input item; pruning behavior withstore=false(only trailing compaction + newer messages sent); no pruning/no resend whenprevious_response_id/conversationis set; and a check thatcontext_managementis now a recognized, typed option. Existing reasoning-item and stateless-replay tests should keep passing unchanged, since this only adds a new, previously-unhandled item type.python/packages/openai/AGENTS.md(andfoundrypackage docs if relevant) to describe the new compaction replay/pruning behavior, following the existing documentation style used for the encrypted-reasoning-item handling.This is a proposal only — no code has been changed. You'd know this is resolved once a long
store=falseconversation usingcontext_managementcompaction shows the compaction item surviving inChatResponse/history and subsequent requests shrinking to roughly system messages + latest compaction item + new turns, instead of resending the full growing history.