Repository navigation
Python: perf(foundry-hosting): FoundryAgentSessionStore rebuilds FoundryStateStore (credential + metadata round-trip) on every request #8280
Description
Describe the bug / perf issue
FoundryAgentSessionStore (the default agent-session SessionStore for hosted MAF agents) rebuilds its backing FoundryStateStore on every get/set/delete. Each _get_store() call goes through FoundryStateStore.get_or_create("agent_sessions", user_isolation=True), which:
- constructs a fresh credential (empty token cache → a new managed-identity token fetch), and
- issues an
agent_sessionsmetadata round-trip (GET/POST state_stores) before the actual item operation.
Because the hosting infra calls set() in the finally of every Responses request, this redundant credential + metadata work lands on the critical path of every request.
Component
python / foundry_hosting — FoundryAgentSessionStore (_state_store.py).
Expected behavior
The agent-session scope ("agent_sessions", user_isolation=True) is identical for every request, so the backing store should be resolved once and reused for the process lifetime. Only the real item GET/PUT/DELETE should remain on the hot path — not a per-request credential build + metadata round-trip.
Measured impact
500 cold + 500 warm streaming requests per agent, concurrency 50, private/VNet Foundry project, gpt-4o-mini, 0 errors / 0 throttled. Same-conditions A/B, baseline vs cached Responses agent (ms, p50):
| Metric | Baseline | Cached | Δ |
|---|---|---|---|
| WARM TTLB p50 | 7,736 | 7,464 | −272 ms (−3.5%) |
| WARM TTFB p50 | 5,680 | 5,683 | ~flat |
The win is the elimination of the per-request credential + metadata round-trip; the remaining warm latency is the model first-token floor (~5.7 s TTFB), unaffected by this change.
Proposed fix
Cache one per-event-loop, per-scope FoundryStateStore for the agent-session scope (guarded by a per-loop lock with double-checked init; keyed by the running loop via a WeakKeyDictionary so a closed loop's store is GC'd, and keyed by scope so subclasses overriding DEFAULT_ROOT_SCOPE are isolated). Item get/set/delete semantics stay byte-for-byte identical; checkpoint / function-approval stores are left untouched (not on the per-request hot path).
Fix implemented in PR #8178 (closed pending this tracking issue).
Metadata
Metadata
Labels
Type
Projects
- Status
Activity
[-]perf(foundry-hosting): FoundryAgentSessionStore rebuilds FoundryStateStore (credential + metadata round-trip) on every request[/-][+]Python: perf(foundry-hosting): FoundryAgentSessionStore rebuilds FoundryStateStore (credential + metadata round-trip) on every request[/+]github-actions commentedon Sep 14, 2026
🤖 Automated triage reproduction notes (agent-authored — trust but verify)
Agent analysis
python/packages/foundry_hosting/agent_framework_foundry_hosting/_state_store.py::FoundryAgentSessionStore._get_storeresolves a new backing store whenever set/get/delete is called, including session persistence from_responses.py’s request-finalization path. Minimal repro: mockFoundryStateStore.get_or_create, call set/get/delete sequentialy oneFoundryAgentSessionStore, and observe an await count of 3 rather than 1.python/packages/foundry_hosting/tests/test_state_store.py::test_agent_session_store_reuses_resolved_store1.0.0b260910