Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions prd-spec/bugfixes/2026-10-27-portal-metrics-counter-unit.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Bugfix PRD: chat_exception_counter unit

## Background / evidence
`veadk/tracing/telemetry/portal_metrics.py` creates `chat_exception_counter` with
`unit="time"`. Its description is "Number of exceptions occurred during chat
completions". A counter that counts discrete events must not carry a duration
unit; sibling `llm_invoke_counter` uses `"count"`, and `skill_download_counter`
/ `skill_load_counter` already use `"1"`.

## Goals
- Align `chat_exception_counter` to a count unit (`"1"`).
- Add a stdlib-only static regression guard for the counter unit metadata.

## Non-goals
- No change to span/event contracts, public API, dependencies, or metric names.
- No runtime wiring change; no new metric.

## Scenarios
- An operator building a dashboard from `llm_completions_exceptions` sees a
dimensionless count rather than a duration unit.

## Affected files
- `veadk/tracing/telemetry/portal_metrics.py` (one line: unit `"time"` -> `"1"`)
- `tests/test_portal_metrics_counter_unit.py` (new, static guard)

## Tests
- Static test, run locally in isolation: base main `2301477` fails the exception
counter assertion (RED); head passes (GREEN). The sibling-counter scope guard
passes on both revisions.
- The recorder-based runtime test (`test_portal_metrics.py`) requires google.adk
and is left to CI; not run locally.

## Risks
- Dashboard unit display only; no data migration. Risk: negligible.

## Acceptance
- `chat_exception_counter` unit is `"1"`; the static regression passes; no other
counter unit changed.
33 changes: 33 additions & 0 deletions prd-spec/bugfixes/2026-10-27-portal-metrics-counter-unit.zh.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Bugfix PRD:chat_exception_counter 单位

## 背景 / 证据
`veadk/tracing/telemetry/portal_metrics.py` 中 `chat_exception_counter` 使用
`unit="time"` 创建,但其 description 为「聊天补全期间发生的异常数量」。计数离散事件的
counter 不应使用时长单位;同文件 `llm_invoke_counter` 为 `"count"`,
`skill_download_counter` / `skill_load_counter` 已为 `"1"`。

## 目标
- 将 `chat_exception_counter` 对齐为计数单位(`"1"`)。
- 新增仅依赖标准库的静态回归守卫,校验 counter 单位元数据。

## 非目标
- 不改动 span/事件契约、公共 API、依赖或指标名。
- 不改动运行时装配;不新增指标。

## 场景
- 运营方基于 `llm_completions_exceptions` 搭建看板时看到的是无量纲计数,而非时长单位。

## 影响文件
- `veadk/tracing/telemetry/portal_metrics.py`(一行:unit `"time"` -> `"1"`)
- `tests/test_portal_metrics_counter_unit.py`(新增,静态守卫)

## 测试
- 静态测试在隔离环境本地运行:base main `2301477` 上 exception counter 断言失败
(RED),head 上通过(GREEN);sibling counter 范围守卫在两个版本上均通过。
- 基于 recorder 的运行时测试(`test_portal_metrics.py`)依赖 google.adk,本地不运行,交给 CI。

## 风险
- 仅影响看板单位展示;无数据迁移。风险可忽略。

## 验收
- `chat_exception_counter` 单位为 `"1"`;静态回归通过;其他 counter 单位未改动。
52 changes: 52 additions & 0 deletions tests/test_portal_metrics_counter_unit.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Copyright (c) 2026 Beijing Volcano Engine Technology Co., Ltd. and/or its affiliates.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""Minimal regression: chat_exception_counter must use a count unit, not "time".

This static source check requires only the Python standard library; it guards
telemetry metadata consistency against a copy-paste of a duration unit from a
histogram onto a counter. The recorder-based runtime test in
test_portal_metrics.py exercises runtime wiring and requires google.adk.
"""
import re
from pathlib import Path


def _counter_block(src, var):
m = re.search(var + r"\s*=\s*self\.meter\.create_counter\((.*?)\)", src, re.S)
assert m, var + " block not found"
unit = re.search(r'unit\s*=\s*"([^"]*)"', m.group(1))
return unit.group(1) if unit else None


def _source():
return (
Path(__file__).resolve().parent.parent
/ "veadk"
/ "tracing"
/ "telemetry"
/ "portal_metrics.py"
).read_text(encoding="utf-8")


def test_chat_exception_counter_uses_count_unit():
assert _counter_block(_source(), "self.chat_exception_counter") in ("1", "count")


def test_sibling_counters_unchanged():
# scope guard: these units must remain as-is (not altered by this change)
src = _source()
assert _counter_block(src, "self.llm_invoke_counter") == "count"
assert _counter_block(src, "self.skill_download_counter") == "1"
assert _counter_block(src, "self.skill_load_counter") == "1"
2 changes: 1 addition & 1 deletion veadk/tracing/telemetry/portal_metrics.py
Original file line number Diff line number Diff line change
Expand Up @@ -195,7 +195,7 @@ def __init__(self, name: str = "veadk_portal") -> None:
)
self.chat_exception_counter = self.meter.create_counter(
name=Meters.LLM_COMPLETIONS_EXCEPTIONS,
unit="time",
unit="1",
description="Number of exceptions occurred during chat completions",
)
self.streaming_time_to_first_token = self.meter.create_histogram(
Expand Down