From bc5033c7fc6bed84dd76c49c0d04186cb14f91de Mon Sep 17 00:00:00 2001 From: Saul Moro Date: Tue, 15 Sep 2026 17:23:37 +0200 Subject: [PATCH 1/2] fix(dashboard): normalize Unicode correction keywords --- CHANGELOG.md | 1 + docs/usage-guide.md | 6 ++- docs/usage-guide.zh-CN.md | 6 ++- src/__tests__/dashboard-collector.test.ts | 59 +++++++++++++++++++++++ src/dashboard-collector.ts | 4 +- 5 files changed, 70 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a0705a34..a6f9be34 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,6 +13,7 @@ All notable changes to this project will be documented in this file. See [standa ### 🐛 Bug Fixes +- Course-correction matching normalizes prompts and keywords to Unicode NFC, so composed and decomposed accents match. Stored prompt summaries and the 60-second correction window are unchanged. Fixes [#573](https://github.com/Tencent/teamai-cli/issues/573). - Course-correction detection matches keywords in space-separated scripts as whole words, so Spanish "segundo" no longer counts as `undo` (for [#564](https://github.com/Tencent/teamai-cli/issues/564)). - MCP `requires` is resolved from `PATH` (including Windows `PATHEXT`), so `teamai mcp inject` no longer skips servers such as `uvx` on Windows ([#540](https://github.com/Tencent/teamai-cli/pull/540), for [#539](https://github.com/Tencent/teamai-cli/issues/539)). - The GitHub and CNB providers resolve their CLI to a launchable absolute path and start it through cross-spawn, so on Windows they no longer answer "installed" while every call fails silently ([#520](https://github.com/Tencent/teamai-cli/pull/520)). diff --git a/docs/usage-guide.md b/docs/usage-guide.md index 6bed9187..5e7081e6 100644 --- a/docs/usage-guide.md +++ b/docs/usage-guide.md @@ -1242,7 +1242,7 @@ Each session card shows a `⚠ N` badge, counting the **number of human interven | `toolReject` | User rejected a tool call (permission deny) | A tool_result marked as rejected in the transcript | | `correction` | Within 60s after the agent stops, the user submits a follow-up prompt containing a correction keyword ("not right" / "redo" / "wrong" / 「違う」 / 「やり直し」 / etc. — Chinese, English and Japanese built in, plus any team keywords) | The stop → prompt_submit event pattern | -> Privacy: only counts are tracked — no prompt or transcript text is ever stored. +> Privacy: intervention metrics contain counts. Local dashboard events also store the first 200 characters of each prompt and may include the last assistant output. Keywords in a space-separated script (English, Spanish, ...) must appear as a whole word, so Spanish "segundo" does not count as `undo`. Chinese and Japanese keywords match as substrings. The built-in list covers only Chinese, English and Japanese; a correction typed in any other language is not detected until the team adds its own words in `teamai.yaml`. Team words are merged with the built-in list and matched case-insensitively under the same rules: @@ -1254,6 +1254,8 @@ sharing: The prompt is checked when the `UserPromptSubmit` hook captures it, so a change to the team keywords applies to new prompts after the next `teamai pull`; sessions recorded earlier are not re-evaluated. +Matching normalizes both the prompt and keywords to Unicode NFC. For example, `réessaye` matches `re\u0301essaye`, where `\u0301` is a combining acute accent. Accents remain significant, so `reessaye` does not match. Normalization applies only to matching and does not change the stored prompt summary or the 60-second correction window. + Intervention data is automatically aggregated and reported to the team's `stats/.yaml` during `teamai pull`, and shown in the "Session Autonomy" leaderboard of `teamai digest`, with team averages and per-person intervention rate rankings — useful for verifying whether a skill/rule reduces intervention rates after rollout. Tools without a transcript (e.g. Cursor) degrade gracefully, tracking only `correction`. #### Conversation Volume & Token Usage @@ -1265,7 +1267,7 @@ Each session card also shows two badges: | `💬 N` | The **number of human conversation turns** in the session (how many prompts were sent) | Count of `UserPromptSubmit` events | | `⛁ X` | The session's cumulative **token usage** (hover to see input / output / cache read / cache write breakdown) | Claude Code `message.usage`, CodeBuddy `requests[].usage`, or Codex's latest session-level `token_usage_record`; legacy `event_msg.token_count` snapshots are summed once per rollout file | -> Privacy: only turn counts and token counts are tracked — no prompt or transcript text is ever stored. +> Privacy: the shared `prompts` and `tokens` fields contain counts, without prompt or transcript text. These two metrics are likewise aggregated into `stats/.yaml` (as `prompts` and `tokens` fields) during `teamai pull`, and shown in the "Conversation Volume & Token Usage" section of `teamai digest`, with team-wide totals, bucketed token totals, and per-person token usage rankings. Tools without transcript access (e.g. Cursor) degrade gracefully: turn counts are still tracked, while tokens show as 0 / N/A. diff --git a/docs/usage-guide.zh-CN.md b/docs/usage-guide.zh-CN.md index 2b143d37..a3e73967 100644 --- a/docs/usage-guide.zh-CN.md +++ b/docs/usage-guide.zh-CN.md @@ -1209,7 +1209,7 @@ teamai dashboard --port 8080 | `toolReject` | 用户拒绝某个工具调用（permission deny） | transcript 中标记拒绝的 tool_result | | `correction` | agent stop 后 60s 内用户追加含「不对 / 重来 / 错了 / wrong / redo / 違う / やり直し」等纠偏词（内置中、英、日，外加团队自定义词）的 prompt | stop → prompt_submit 事件模式 | -> 隐私：只统计**次数**，不落地任何 prompt 或 transcript 原文。 +> 隐私：干预指标记录次数。本地 dashboard 事件还会保存每条 prompt 的前 200 个字符，也可能包含 agent 的最后一次回复。 以空格分词的文字（英语、西班牙语等）中的纠偏词必须整词匹配，因此西班牙语 "segundo" 不会被算作 `undo`；中文、日文纠偏词仍按子串匹配。内置列表只覆盖中、英、日三种语言，其他语言的纠偏在团队于 `teamai.yaml` 添加自己的词之前不会被识别。团队词与内置列表合并，忽略大小写，遵循同样的匹配规则： @@ -1221,6 +1221,8 @@ sharing: 匹配在 `UserPromptSubmit` hook 捕获 prompt 时完成，因此修改团队纠偏词后，下一次 `teamai pull` 之后的新 prompt 才会生效；之前记录的会话不会重新评估。 +匹配时，prompt 和纠偏词都会转换为 Unicode NFC 形式。例如，`réessaye` 可以匹配 `re\u0301essaye`，其中 `\u0301` 是组合尖音符。重音符号仍有区别，因此 `reessaye` 不匹配。规范化仅用于匹配，不会改变已存储的 prompt 摘要或 60 秒的纠偏时间窗口。 + 干预数据会随 `teamai pull` 自动聚合上报到团队 `stats/.yaml`，并在 `teamai digest` 的「会话自主性」榜单中给出团队均值与人均干预率排行，可用于验证某个 skill / rule 上线后干预率是否下降。无 transcript 的工具（如 Cursor）会优雅降级，只统计 `correction`。 #### 对话量与 Token 用量 @@ -1232,7 +1234,7 @@ sharing: | `💬 N` | 该会话里**人类对话的轮数**（发了几次 prompt） | `UserPromptSubmit` 事件数 | | `⛁ X` | 该会话累计 **token 用量**（鼠标悬停看 输入 / 输出 / 缓存读 / 缓存写 明细） | Claude Code `message.usage`、CodeBuddy `requests[].usage`，或 Codex 最新的会话级 `token_usage_record`；旧版 `event_msg.token_count` 按 rollout 文件各取最新快照后累加 | -> 隐私：只统计**轮数与 token 数量**，不落地任何 prompt 或 transcript 原文。 +> 隐私：共享的 `prompts` 和 `tokens` 字段仅包含数量，不包含 prompt 或 transcript 原文。 这两项同样随 `teamai pull` 聚合到 `stats/.yaml`（`prompts` 与 `tokens` 字段），并在 `teamai digest` 的「对话量与 Token 用量」板块给出团队对话总轮数、token 总量（分桶）与人均 token 用量排行。拿不到 transcript 的工具（如 Cursor）会优雅降级：仍统计对话轮数，token 显示为 0 / N/A。 diff --git a/src/__tests__/dashboard-collector.test.ts b/src/__tests__/dashboard-collector.test.ts index 62100c2a..19d762bd 100644 --- a/src/__tests__/dashboard-collector.test.ts +++ b/src/__tests__/dashboard-collector.test.ts @@ -163,6 +163,37 @@ describe('parseHookEvent', () => { } }); + it.each([ + ['NFC keyword and NFD prompt', 'r\u00e9essaye', 're\u0301essaye'], + ['NFD keyword and NFC prompt', 're\u0301essaye', 'r\u00e9essaye'], + ['NFD keyword and uppercase prompt', 're\u0301essaye', 'R\u00c9ESSAYE!'], + ['NFD prompt past the summary limit', 'r\u00e9essaye', `${'x '.repeat(150)}re\u0301essaye`], + ])('matches canonically equivalent text: %s', async (_label, keyword, prompt) => { + const event = await parseHookEvent( + JSON.stringify({ hook_event_name: 'UserPromptSubmit', session_id: 's', prompt }), + 'claude', + { correctionKeywords: [keyword] }, + ); + expect(event?.correction).toBe(true); + expect(event?.promptSummary).toBe(prompt.slice(0, 200)); + }); + + it.each([ + ['missing accent', 'reessaye'], + ['different accent', 're\u0300essaye'], + ['letter prefix', 'pre\u0301essaye'], + ['letter suffix', 're\u0301essayez'], + ['underscore prefix', 'test_re\u0301essaye'], + ['underscore suffix', 're\u0301essaye_it'], + ])('does not match a team keyword with a %s', async (_label, prompt) => { + const event = await parseHookEvent( + JSON.stringify({ hook_event_name: 'UserPromptSubmit', session_id: 's', prompt }), + 'claude', + { correctionKeywords: ['r\u00e9essaye'] }, + ); + expect(event?.correction).toBe(false); + }); + it('checks the full prompt, not only the 200-char summary', async () => { const prompt = `${'x '.repeat(150)}wrong`; const raw = JSON.stringify({ hook_event_name: 'UserPromptSubmit', session_id: 's', prompt }); @@ -734,6 +765,26 @@ describe('parseHookEvent interventions', () => { describe('rebuildSessions interventions', () => { const now = new Date().toISOString(); + it.each([ + ['NFC keyword and NFD prompt', 'r\u00e9essaye', 're\u0301essaye'], + ['NFD keyword and NFC prompt', 're\u0301essaye', 'r\u00e9essaye'], + ])('counts a Unicode correction within the time window: %s', async (_label, keyword, prompt) => { + const event = await parseHookEvent( + JSON.stringify({ hook_event_name: 'UserPromptSubmit', session_id: 's', prompt }), + 'claude', + { correctionKeywords: [keyword] }, + ); + if (!event) throw new Error('Expected a prompt-submit event'); + + for (const [gap, expected] of [[0, 1], [60_000, 1], [60_001, 0]]) { + const sessions = rebuildSessions([ + { type: 'stop', timestamp: now, sessionId: 's', tool: 'claude' }, + { ...event, timestamp: new Date(new Date(now).getTime() + gap).toISOString() }, + ]); + expect(sessions[0]?.interventions.correction, `gap ${gap}`).toBe(expected); + } + }); + it('defaults to zero interventions', () => { const sessions = rebuildSessions([ { type: 'session_start', timestamp: now, sessionId: 's1', tool: 'claude', cwd: '/p' }, @@ -742,6 +793,14 @@ describe('rebuildSessions interventions', () => { expect(sessions[0].interventionCount).toBe(0); }); + it('normalizes built-in keywords when a legacy event has no correction flag', () => { + const sessions = rebuildSessions([ + { type: 'stop', timestamp: now, sessionId: 's', tool: 'claude' }, + { type: 'prompt_submit', timestamp: now, sessionId: 's', tool: 'claude', promptSummary: '\u3061\u304b\u3099\u3046' }, + ]); + expect(sessions[0]?.interventions.correction).toBe(1); + }); + it('takes interrupt/toolReject from the latest stop snapshot (idempotent)', () => { const sessions = rebuildSessions([ { type: 'session_start', timestamp: now, sessionId: 's1', tool: 'claude', cwd: '/p' }, diff --git a/src/dashboard-collector.ts b/src/dashboard-collector.ts index 9370aa51..1e577269 100644 --- a/src/dashboard-collector.ts +++ b/src/dashboard-collector.ts @@ -824,7 +824,7 @@ function wordBoundaryPattern(keyword: string): RegExp { /** True when `lower` contains `keyword`, whole-word for spaced scripts, substring otherwise. */ function containsKeyword(lower: string, keyword: string): boolean { - const k = keyword.trim().toLowerCase(); + const k = keyword.trim().normalize('NFC').toLowerCase(); if (!k) return false; if (UNSPACED_SCRIPT_RE.test(k)) return lower.includes(k); return wordBoundaryPattern(k).test(lower); @@ -836,7 +836,7 @@ function containsKeyword(lower: string, keyword: string): boolean { */ function isCorrectionPrompt(text?: string, extraKeywords: readonly string[] = []): boolean { if (!text) return false; - const lower = text.toLowerCase(); + const lower = text.normalize('NFC').toLowerCase(); return [...CORRECTION_KEYWORDS, ...extraKeywords].some((k) => containsKeyword(lower, k)); } From 949f6cc6609f962bfd88f861b4f69d2f5755a890 Mon Sep 17 00:00:00 2001 From: Saul Moro Date: Tue, 15 Sep 2026 17:25:31 +0200 Subject: [PATCH 2/2] docs: limit guide changes to Unicode matching --- docs/usage-guide.md | 4 ++-- docs/usage-guide.zh-CN.md | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/usage-guide.md b/docs/usage-guide.md index 5e7081e6..2d66f296 100644 --- a/docs/usage-guide.md +++ b/docs/usage-guide.md @@ -1242,7 +1242,7 @@ Each session card shows a `⚠ N` badge, counting the **number of human interven | `toolReject` | User rejected a tool call (permission deny) | A tool_result marked as rejected in the transcript | | `correction` | Within 60s after the agent stops, the user submits a follow-up prompt containing a correction keyword ("not right" / "redo" / "wrong" / 「違う」 / 「やり直し」 / etc. — Chinese, English and Japanese built in, plus any team keywords) | The stop → prompt_submit event pattern | -> Privacy: intervention metrics contain counts. Local dashboard events also store the first 200 characters of each prompt and may include the last assistant output. +> Privacy: only counts are tracked — no prompt or transcript text is ever stored. Keywords in a space-separated script (English, Spanish, ...) must appear as a whole word, so Spanish "segundo" does not count as `undo`. Chinese and Japanese keywords match as substrings. The built-in list covers only Chinese, English and Japanese; a correction typed in any other language is not detected until the team adds its own words in `teamai.yaml`. Team words are merged with the built-in list and matched case-insensitively under the same rules: @@ -1267,7 +1267,7 @@ Each session card also shows two badges: | `💬 N` | The **number of human conversation turns** in the session (how many prompts were sent) | Count of `UserPromptSubmit` events | | `⛁ X` | The session's cumulative **token usage** (hover to see input / output / cache read / cache write breakdown) | Claude Code `message.usage`, CodeBuddy `requests[].usage`, or Codex's latest session-level `token_usage_record`; legacy `event_msg.token_count` snapshots are summed once per rollout file | -> Privacy: the shared `prompts` and `tokens` fields contain counts, without prompt or transcript text. +> Privacy: only turn counts and token counts are tracked — no prompt or transcript text is ever stored. These two metrics are likewise aggregated into `stats/.yaml` (as `prompts` and `tokens` fields) during `teamai pull`, and shown in the "Conversation Volume & Token Usage" section of `teamai digest`, with team-wide totals, bucketed token totals, and per-person token usage rankings. Tools without transcript access (e.g. Cursor) degrade gracefully: turn counts are still tracked, while tokens show as 0 / N/A. diff --git a/docs/usage-guide.zh-CN.md b/docs/usage-guide.zh-CN.md index a3e73967..c359b13d 100644 --- a/docs/usage-guide.zh-CN.md +++ b/docs/usage-guide.zh-CN.md @@ -1209,7 +1209,7 @@ teamai dashboard --port 8080 | `toolReject` | 用户拒绝某个工具调用（permission deny） | transcript 中标记拒绝的 tool_result | | `correction` | agent stop 后 60s 内用户追加含「不对 / 重来 / 错了 / wrong / redo / 違う / やり直し」等纠偏词（内置中、英、日，外加团队自定义词）的 prompt | stop → prompt_submit 事件模式 | -> 隐私：干预指标记录次数。本地 dashboard 事件还会保存每条 prompt 的前 200 个字符，也可能包含 agent 的最后一次回复。 +> 隐私：只统计**次数**，不落地任何 prompt 或 transcript 原文。 以空格分词的文字（英语、西班牙语等）中的纠偏词必须整词匹配，因此西班牙语 "segundo" 不会被算作 `undo`；中文、日文纠偏词仍按子串匹配。内置列表只覆盖中、英、日三种语言，其他语言的纠偏在团队于 `teamai.yaml` 添加自己的词之前不会被识别。团队词与内置列表合并，忽略大小写，遵循同样的匹配规则： @@ -1234,7 +1234,7 @@ sharing: | `💬 N` | 该会话里**人类对话的轮数**（发了几次 prompt） | `UserPromptSubmit` 事件数 | | `⛁ X` | 该会话累计 **token 用量**（鼠标悬停看 输入 / 输出 / 缓存读 / 缓存写 明细） | Claude Code `message.usage`、CodeBuddy `requests[].usage`，或 Codex 最新的会话级 `token_usage_record`；旧版 `event_msg.token_count` 按 rollout 文件各取最新快照后累加 | -> 隐私：共享的 `prompts` 和 `tokens` 字段仅包含数量，不包含 prompt 或 transcript 原文。 +> 隐私：只统计**轮数与 token 数量**，不落地任何 prompt 或 transcript 原文。 这两项同样随 `teamai pull` 聚合到 `stats/.yaml`（`prompts` 与 `tokens` 字段），并在 `teamai digest` 的「对话量与 Token 用量」板块给出团队对话总轮数、token 总量（分桶）与人均 token 用量排行。拿不到 transcript 的工具（如 Cursor）会优雅降级：仍统计对话轮数，token 显示为 0 / N/A。