Repository navigation
Python: Keep span-only tool metadata off invocation metrics - #9567
Alexander Kaplan (tokaplan) wants to merge 1 commit into
Conversation
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f572c8f1-a5c8-4eb0-b06c-1a15ba38d1c3
There was a problem hiding this comment.
🟢 Approval recommended
The focused change preserves aggregation dimensions and error classification, with regression coverage for excluded metadata.
No open issues 🎉
What changed in this PR
This follow-up to #9361 keeps span-only tool metadata out of Python function-duration metrics without changing span telemetry.
Changes:
- Explicitly allowlists four metric dimensions while preserving failure classification.
- Extends regression assertions to exclude tool descriptions and call IDs from metrics.
| File | Description |
|---|---|
| python/packages/core/tests/core/test_tools.py | Verifies metric exclusions and preserved span attributes. |
| python/packages/core/agent_framework/_tools.py | Replaces copied span attributes with an explicit metric allowlist. |
Review effort: Balanced
Bicheng (Kenneth) (BichengWang)
left a comment
There was a problem hiding this comment.
Reviewed a29302a7b1377ec8e5aa98be473722616980a106. The allowlist retains the four aggregation dimensions and keeps error.type on failures, while leaving call IDs and descriptions on spans. I found no correctness blocker in the diff.
Offline validation on Python 3.13.16 with OpenTelemetry SDK 1.45.1: all 151 tests in test_tools.py pass, and Ruff 0.17.0 lint/format checks pass for both changed files. The four affected regression cases fail against the base implementation because the call ID is still a metric attribute. I also exercised the actual SDK with two invocations having distinct call IDs and descriptions: the head produces one histogram series with count 2, and both spans retain their respective metadata. This was a focused review; I did not run the full core suite or type-checking matrix.
One nonblocking coverage suggestion: preserve that SDK aggregation case as a test using InMemoryMetricReader. The existing histogram mocks verify individual attributes, while a real reader also verifies that changing call IDs/descriptions does not split the metric into multiple time series.
|
Thank you for identifying and implementing this telemetry follow-up. We opened #9592 as the maintained replacement; it uses the same explicit allowlist approach, adds the explicit dict[str, str] annotation required by the current Pyright check, strengthens the regression assertions around the complete metric attribute shape, and is rebased on the latest main. Closing this as a duplicate so review can continue there. Thanks again for the contribution. |
Motivation & Context
PR #9361 detached tool-call payload attributes from function-invocation metrics, but the metric snapshot still copies every base tool-span attribute. As a result,
gen_ai.tool.description(which may contain sensitive information) and the per-invocationgen_ai.tool.call.idremain histogram dimensions.This follow-up keeps span telemetry unchanged while preventing sensitive and high-cardinality span-only metadata from flowing into metrics.
Description & Review Guide
error.typeonly on failures.execute_toolspans but no longer become metric dimensions. Existing low-cardinality dimensions and failure classification remain unchanged.Related Issue
N/A - telemetry correctness and cardinality follow-up to #9361.
AI Assistance
AI assistance details: Copilot assisted with implementation, regression tests, and the PR description.
Contribution Checklist
breaking changelabel (or add "[BREAKING]" to the title prefix, before or after the language prefix) — a workflow keeps the label and title prefix in sync automatically.