Skip to content

Python: add AgentBudget for autonomous loop resource limits - #9164

Open
Mark Licata (marklicata) wants to merge 9 commits into
microsoft:mainfrom
marklicata:agent-budget-python
Open

Mark Licata (marklicata) wants to merge 9 commits into
microsoft:mainfrom
marklicata:agent-budget-python

Conversation

@marklicata

@marklicata Mark Licata (marklicata) commented Oct 7, 2026 •

Copy link
Copy Markdown

Motivation & Context

Autonomous loops need explicit resource limits. Without them, a confused or looping agent can burn tokens indefinitely or hold a session open on a slow external tool, and neither is safe for production. This adds AgentBudget so operators can bound AgentLoopMiddleware by token spend and wall-clock time.

This is the Python half of the work previously proposed in #8994, split per language at the reviewers' request. The .NET counterpart is #9165.

Description & Review Guide

What are the major changes?

  • AgentBudget dataclass: max_tokens: int | None and max_duration: float | None, both optional with no defaults
  • AgentLoopMiddleware: new budget: AgentBudget | None argument. Budget checks run before should_continue, so an expensive judge isn't called once a budget is exhausted.
  • Token tracking: cumulative total_token_count per iteration from UsageDetails, falling back to input + output
  • Time tracking: time.monotonic() at loop start, checked per iteration
  • Exit reason stamped in additional_properties["loop_exit_reason"]: "token_budget_exceeded" / "time_budget_exceeded" on budget stops (non-streaming only), "iteration_cap_reached" when max_iterations fires

What is the impact of these changes?

Opt-in; loops without a budget behave as today.

Sub-agent usage

Sub-agent tokens count toward max_tokens. This is not configurable.

  • Agent.as_tool rolls the sub-agent's usage_details into the parent's response usage_details, in both streaming and non-streaming runs. This also holds when the sub-agent stops on an approval or user-input request. Nested A→B→C counts each layer once.
  • BackgroundAgentsProvider charges each finished task's usage once, to the parent run that first observes the completion.
  • invoke_agent telemetry spans still report each agent's own usage only.
  • Known gaps: failed or cancelled sub-agent runs produce no response, so their tokens aren't counted. Background tasks that are never observed, or are abandoned at release_session, aren't counted. Custom tools that call agent.run directly aren't counted.

What do you want reviewers to focus on?

  • Check ordering: after the pending-approval escape hatch, before should_continue
  • The total_token_count fallback when providers omit it
  • Streaming limitation: the budget stops the loop but can't stamp additional_properties

Related Issue

Part of #8862, together with #9165 for .NET. The issue is closed manually once both PRs have merged.

Contribution Checklist

  • The code builds clean without any errors or warnings
  • All unit tests pass, and I have added new tests where possible
  • The PR follows the Contribution Guidelines
  • This PR is linked to an issue and there is no other open PR for this issue (see Related Issue above).
  • This is not a breaking change.

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Wall-clock limits do not interrupt in-flight work, and token fallback and streaming enforcement lack tests.

2 open findings
What changed in this PR

Adds opt-in token and wall-clock budgets for Python autonomous agent loops.

Changes:

  • Introduces and exports AgentBudget.
  • Enforces budgets and records loop exit reasons.
  • Adds non-streaming budget tests.
File Description
python/​packages/​core/​agent_framework/​_harness/​_loop.py Implements budget validation and enforcement.
python/​packages/​core/​agent_framework/​__init__.py Exports AgentBudget at runtime.
python/​packages/​core/​agent_framework/​__init__.pyi Exports the typed public API.
python/​packages/​core/​tests/​core/​test_harness_loop.py Adds budget behavior and validation tests.

🧠 Review effort: Balanced


💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

Comment thread python/packages/core/agent_framework/_harness/_loop.py Outdated
Comment thread python/packages/core/agent_framework/_harness/_loop.py
…allback

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread python/packages/core/agent_framework/_harness/_loop.py Outdated
Comment thread python/packages/core/agent_framework/_harness/_loop.py Outdated
Convert AgentBudget from a dataclass to a total=False TypedDict so it can be
used for typing without importing it, and plain dicts work as budgets.
Validation moves into AgentLoopMiddleware.__init__ with the same ValueError
messages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
max_tokens charges the wrapped agent's total_token_count per iteration
(falling back to input + output). The with_judge judge call is never
charged. Sub-agent usage counts only when it is reported in the agent's
response usage_details; as_tool and background agents currently return
text only, so their usage is not included. Not configurable.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@agent-framework-automation agent-framework-automation Bot added the documentation Usage: [Issues, PRs], Target: documentation in the code base and learn docs label Oct 13, 2026
Sub-agent usage now rolls into the parent's response usage_details, streaming
and non-streaming, so AgentBudget.max_tokens includes it.

- Agent.as_tool reports the sub-agent's usage_details through a private
  FunctionInvocationContext metadata key (also before the approval/user-input
  raises). The function invocation loop moves it onto the function_result and
  folds it into the parent response exactly once (Phase 1/2/3 non-streaming,
  a usage content on the tool update when streaming), including the
  error-result, middleware-termination and user-input exception paths.
  Nested A->B->C counts each layer once.
- BackgroundAgentsProvider stores each finished run's usage on
  BackgroundTaskInfo (usage_details/usage_reported, serialized) and charges it
  once, to the parent run that first observes the completion: through the
  observing tool's context, or a run-scoped chat middleware on the run's first
  model response when before_run observes it.
- invoke_agent spans keep reporting each agent's own usage: the
  response-usage fallback subtracts delegated usage.
- Documented gaps: failed/cancelled sub-agent runs, background tasks never
  observed or abandoned at release, and custom tools calling agent.run
  directly are not counted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Build real UsageDetails values instead of plain dicts, type the run
helpers with Agent[Any], and give the no-chat-telemetry observability
client its own _inner_get_response matching BaseChatClient's signature
instead of reassigning MockBaseChatClient's methods.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
if cap_fired:
final_result.additional_properties.setdefault("loop_exit_reason", "iteration_cap_reached")
if budget_exit is not None:
final_result.additional_properties.setdefault("loop_exit_reason", budget_exit)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When the budget fires, setdefault preserves any stale loop_exit_reason supplied by an inner provider or nested loop, so callers can see the wrong reason even though this middleware stopped for the budget. Please assign the framework-owned reason unconditionally, and do the same for the cap paths, matching #9162's stale-reason behavior. Add a regression starting with a pre-existing reason.

This branch was successfully deployed

1 active deployment
github-app-auth — 98fb54c7 Deployed Oct 13, 2026 by marklicata via add_label #25208
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Usage: [Issues, PRs], Target: documentation in the code base and learn docs python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants