fix(truncation): bound text on code points instead of UTF-16 units - #614
Open
MincongZhou wants to merge 1 commit into
Open
MincongZhou wants to merge 1 commit into
MincongZhou wants to merge 1 commit into
Conversation
normalizeSnippet, sanitizeLine, normalizeSubagentTitle and boundedActivityText measured with String#length and cut with slice(), so a bound landing inside a surrogate pair emitted a lone surrogate into session lists, background-terminal match records, subagent snapshots and the Web capability snapshot. Count and cut on code points instead, which is the pattern already used by post-edit, ask-user and by-the-way. Each site gets a regression test asserting the exact bounded string at an emoji boundary.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Four text bounds measure with
String.prototype.lengthand cut withslice(), so they count UTF-16 code units rather than code points. When a bound lands between the two halves of a surrogate pair, the result ends on a lone high surrogate — a value that is not well-formed text:extensions/sessions/sessions.ts:121normalizeSnippet"a".repeat(18) + 🚀"\uD83D…"extensions/background-terminals/src/watch.ts:156sanitizeLine"x".repeat(499) + 🚀"\uD83D…"extensions/subagents/navigation.ts:32normalizeSubagentTitle"x".repeat(159) + 🚀"x".repeat(159) + "\uD83D"extensions/shared/web-observer-registry.ts:168boundedActivityText"x".repeat(158) + 🚀 + "…""\uD83D…"Each of these values is displayed or serialized:
normalizeSnippetfeeds the session list and session search entries shown in the TUI and Web UI.sanitizeLineproduces the reported line of a background-terminal watch match, which is stored in the match record.normalizeSubagentTitlenormalizes titles "before they enter snapshots, artifacts, or the TUI" (its own doc comment), so a broken title is persisted before it is rendered.boundedActivityTextprojects activity text into the Web capability snapshot; a lone surrogate there cannot survive JSON representation intact and reaches the browser as a replacement character.extensions/shared/terminal-text.tsdoes not repair lone surrogates, so nothing downstream heals the value.Value
The value that crosses the bound can be an emoji or a supplementary-plane CJK character — common in titles and transcript lines for a coding agent that routinely carries user-supplied text. Today the bound corrupts the last visible character of such a string, and the corruption can be persisted into snapshots and artifacts rather than only being rendered once.
Fixing it keeps the bounded text well-formed and matches the behavior the repository already relies on elsewhere, including the two immediately preceding fixes in this area: #586 (
user-input-fold) and #566 (web).Approach
Count and cut on code points:
This is the pattern already used by
extensions/post-edit/index.ts,extensions/ask-user/index.ts, andextensions/subagents/src/by-the-way.ts, so the change aligns the four lagging call sites with the established convention rather than introducing a new one. No shared helper is added:AGENTS.mdasks for the smallest scoped change, and these four sites live in four extensions.Each site also gets a regression test asserting the exact bounded string, so a future
slice()refactor fails loudly instead of silently splitting a pair again.Validation
bun run check— pass (biome format check, biome lint with--error-on-warnings,tsc --noEmit, config/docs/discipline contracts, Web build).bun run test— pass.sessions8/8,watch16/16,navigation9/9,observer-registry13/13.U+1F680) as the boundary character and assert the complete emoji survives; each test fails against the pre-change implementation.Impact
MAXcode points. The existingWATCH_LINE_MAX_CHARScomment and theboundedActivityTextdoc are updated to state the unit explicitly.