Repository navigation
Python: add AgentBudget for autonomous loop resource limits - #9164
Mark Licata (marklicata) wants to merge 9 commits into
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
🟡 Changes recommended
Wall-clock limits do not interrupt in-flight work, and token fallback and streaming enforcement lack tests.
2 open findings
What changed in this PR
Adds opt-in token and wall-clock budgets for Python autonomous agent loops.
Changes:
- Introduces and exports
AgentBudget. - Enforces budgets and records loop exit reasons.
- Adds non-streaming budget tests.
| File | Description |
|---|---|
python/packages/core/agent_framework/_harness/_loop.py |
Implements budget validation and enforcement. |
python/packages/core/agent_framework/__init__.py |
Exports AgentBudget at runtime. |
python/packages/core/agent_framework/__init__.pyi |
Exports the typed public API. |
python/packages/core/tests/core/test_harness_loop.py |
Adds budget behavior and validation tests. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.
…allback Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Convert AgentBudget from a dataclass to a total=False TypedDict so it can be used for typing without importing it, and plain dicts work as budgets. Validation moves into AgentLoopMiddleware.__init__ with the same ValueError messages. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
max_tokens charges the wrapped agent's total_token_count per iteration (falling back to input + output). The with_judge judge call is never charged. Sub-agent usage counts only when it is reported in the agent's response usage_details; as_tool and background agents currently return text only, so their usage is not included. Not configurable. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sub-agent usage now rolls into the parent's response usage_details, streaming and non-streaming, so AgentBudget.max_tokens includes it. - Agent.as_tool reports the sub-agent's usage_details through a private FunctionInvocationContext metadata key (also before the approval/user-input raises). The function invocation loop moves it onto the function_result and folds it into the parent response exactly once (Phase 1/2/3 non-streaming, a usage content on the tool update when streaming), including the error-result, middleware-termination and user-input exception paths. Nested A->B->C counts each layer once. - BackgroundAgentsProvider stores each finished run's usage on BackgroundTaskInfo (usage_details/usage_reported, serialized) and charges it once, to the parent run that first observes the completion: through the observing tool's context, or a run-scoped chat middleware on the run's first model response when before_run observes it. - invoke_agent spans keep reporting each agent's own usage: the response-usage fallback subtracts delegated usage. - Documented gaps: failed/cancelled sub-agent runs, background tasks never observed or abandoned at release, and custom tools calling agent.run directly are not counted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Build real UsageDetails values instead of plain dicts, type the run helpers with Agent[Any], and give the no-chat-telemetry observability client its own _inner_get_response matching BaseChatClient's signature instead of reassigning MockBaseChatClient's methods. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| if cap_fired: | ||
| final_result.additional_properties.setdefault("loop_exit_reason", "iteration_cap_reached") | ||
| if budget_exit is not None: | ||
| final_result.additional_properties.setdefault("loop_exit_reason", budget_exit) |
There was a problem hiding this comment.
When the budget fires, setdefault preserves any stale loop_exit_reason supplied by an inner provider or nested loop, so callers can see the wrong reason even though this middleware stopped for the budget. Please assign the framework-owned reason unconditionally, and do the same for the cap paths, matching #9162's stale-reason behavior. Add a regression starting with a pre-existing reason.


Motivation & Context
Autonomous loops need explicit resource limits. Without them, a confused or looping agent can burn tokens indefinitely or hold a session open on a slow external tool, and neither is safe for production. This adds
AgentBudgetso operators can boundAgentLoopMiddlewareby token spend and wall-clock time.This is the Python half of the work previously proposed in #8994, split per language at the reviewers' request. The .NET counterpart is #9165.
Description & Review Guide
What are the major changes?
AgentBudgetdataclass:max_tokens: int | Noneandmax_duration: float | None, both optional with no defaultsAgentLoopMiddleware: newbudget: AgentBudget | Noneargument. Budget checks run beforeshould_continue, so an expensive judge isn't called once a budget is exhausted.total_token_countper iteration fromUsageDetails, falling back toinput + outputtime.monotonic()at loop start, checked per iterationadditional_properties["loop_exit_reason"]:"token_budget_exceeded"/"time_budget_exceeded"on budget stops (non-streaming only),"iteration_cap_reached"whenmax_iterationsfiresWhat is the impact of these changes?
Opt-in; loops without a budget behave as today.
Sub-agent usage
Sub-agent tokens count toward
max_tokens. This is not configurable.Agent.as_toolrolls the sub-agent'susage_detailsinto the parent's responseusage_details, in both streaming and non-streaming runs. This also holds when the sub-agent stops on an approval or user-input request. Nested A→B→C counts each layer once.BackgroundAgentsProvidercharges each finished task's usage once, to the parent run that first observes the completion.invoke_agenttelemetry spans still report each agent's own usage only.release_session, aren't counted. Custom tools that callagent.rundirectly aren't counted.What do you want reviewers to focus on?
should_continuetotal_token_countfallback when providers omit itadditional_propertiesRelated Issue
Part of #8862, together with #9165 for .NET. The issue is closed manually once both PRs have merged.
Contribution Checklist
🤖 Generated with Claude Code