Skip to content

Python: Keep span-only tool metadata off invocation metrics - #9567

Closed
Alexander Kaplan (tokaplan) wants to merge 1 commit into
microsoft:mainfrom
tokaplan:fix/python-tool-metric-sensitive-dimensions
Closed

Alexander Kaplan (tokaplan) wants to merge 1 commit into
microsoft:mainfrom
tokaplan:fix/python-tool-metric-sensitive-dimensions

Conversation

@tokaplan

Copy link
Copy Markdown
Member

Motivation & Context

PR #9361 detached tool-call payload attributes from function-invocation metrics, but the metric snapshot still copies every base tool-span attribute. As a result, gen_ai.tool.description (which may contain sensitive information) and the per-invocation gen_ai.tool.call.id remain histogram dimensions.

This follow-up keeps span telemetry unchanged while preventing sensitive and high-cardinality span-only metadata from flowing into metrics.

Description & Review Guide

  • What are the major changes? Build function-duration metric attributes from an explicit allowlist containing operation name, tool name, tool type, and the framework function name. Continue adding error.type only on failures.
  • What is the impact of these changes? Tool descriptions and call IDs remain available on execute_tool spans but no longer become metric dimensions. Existing low-cardinality dimensions and failure classification remain unchanged.
  • What do you want reviewers to focus on? Confirm that the allowlist preserves useful aggregation dimensions while preventing future span-only attributes from entering the histogram implicitly.

Related Issue

N/A - telemetry correctness and cardinality follow-up to #9361.

AI Assistance

  • No material AI assistance was used.
  • This is an AI-assisted contribution. I reviewed, understood, and verified all submitted content and accept responsibility for it.

AI assistance details: Copilot assisted with implementation, regression tests, and the PR description.

Contribution Checklist

  • The code builds clean without any errors or warnings
  • All unit tests pass, and I have added new tests where possible
  • The PR follows the Contribution Guidelines
  • This PR links to an agreed issue with no competing open PR, or the Related Issue section documents a trivial-change or repository-automation exception.
  • This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after the language prefix) — a workflow keeps the label and title prefix in sync automatically.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f572c8f1-a5c8-4eb0-b06c-1a15ba38d1c3
Copilot AI balanced review requested due to automatic review settings October 20, 2026 20:37
@agent-framework-automation agent-framework-automation Bot added the python Usage: [Issues, PRs], Target: Python label Oct 20, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The focused change preserves aggregation dimensions and error classification, with regression coverage for excluded metadata.

No open issues 🎉

What changed in this PR

This follow-up to #9361 keeps span-only tool metadata out of Python function-duration metrics without changing span telemetry.

Changes:

  • Explicitly allowlists four metric dimensions while preserving failure classification.
  • Extends regression assertions to exclude tool descriptions and call IDs from metrics.
File Description
python/​packages/​core/​tests/​core/​test_tools.py Verifies metric exclusions and preserved span attributes.
python/​packages/​core/​agent_framework/​_tools.py Replaces copied span attributes with an explicit metric allowlist.

Review effort: Balanced

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed a29302a7b1377ec8e5aa98be473722616980a106. The allowlist retains the four aggregation dimensions and keeps error.type on failures, while leaving call IDs and descriptions on spans. I found no correctness blocker in the diff.

Offline validation on Python 3.13.16 with OpenTelemetry SDK 1.45.1: all 151 tests in test_tools.py pass, and Ruff 0.17.0 lint/format checks pass for both changed files. The four affected regression cases fail against the base implementation because the call ID is still a metric attribute. I also exercised the actual SDK with two invocations having distinct call IDs and descriptions: the head produces one histogram series with count 2, and both spans retain their respective metadata. This was a focused review; I did not run the full core suite or type-checking matrix.

One nonblocking coverage suggestion: preserve that SDK aggregation case as a test using InMemoryMetricReader. The existing histogram mocks verify individual attributes, while a real reader also verifies that changing call IDs/descriptions does not split the metric into multiple time series.

@eavanvalkenburg

Copy link
Copy Markdown
Member

Thank you for identifying and implementing this telemetry follow-up. We opened #9592 as the maintained replacement; it uses the same explicit allowlist approach, adds the explicit dict[str, str] annotation required by the current Pyright check, strengthens the regression assertions around the complete metric attribute shape, and is rebased on the latest main. Closing this as a duplicate so review can continue there. Thanks again for the contribution.

This branch was successfully deployed

1 active deployment
github-app-auth — a29302a7 Deployed Oct 20, 2026 by tokaplan via team_check #7118
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants