Skip to content

Assert a chat turn ends in exactly one earned terminal - #858

Merged
WaylandYang merged 1 commit into
devfrom
test/one-earned-terminal
Sep 21, 2026
Merged

WaylandYang merged 1 commit into
devfrom
test/one-earned-terminal

Conversation

@WaylandYang

Copy link
Copy Markdown
Contributor

Which record

None. This adds a test; no decision changes and no status line needs updating. From #857.

Why

#845, #850, #851 and #852 each close one case of the chat path reporting success when something underneath it failed, and each ships its own regression. Four tests that each catch one leak do not catch the fifth, which is written the same way by whoever next adds a path that can fail after the first byte is out.

This asserts the rule instead of the instances. One table, one row per injection point, the same three assertions every time:

  1. no event: done reaches the client
  2. exactly one terminal is observed — done and error together occur once
  3. the terminal names a reason, so an error with empty data fails too

Adding a path that can fail costs one row. Not being able to write that row means the path has not decided how it ends, which is the thing review should catch.

The table today

row ends in state
the model answers normally done enforced — this is the control
the reply is still empty after the retry error enforced
the last turn returns bare tool-control markup error pending #845
markup after a plausible narration error pending #845

Rows whose fix has not landed keep a pending: Some("#845") field, print a skip line, and are left in the table rather than deleted — a known hole should be visible. Deleting that field when the PR lands is what starts enforcing the row.

The pending rows do fail on dev. Setting pending: None on both against 79caca04:

assertion `left == right` failed: 最后一轮吐的是裸的工具控制标记:失败不许报成 done
event: snapshot
event: conversation
event: step
event: delta      <- the markup is streamed to the user as the answer
event: done       <- and reported as success
  left: 1
 right: 0

So the row is not decorative; it catches the live bug and will go green when #845 lands.

Three choices worth explaining

A sibling module, not another mod line in chat_empty_reply_tests.rs. That file is where #848, #849, #850 and #852 all collide (see my note on #852). This declares chat_terminal_tests from chat.rs instead, so it adds nothing to the contested spot.

The fixture is reused, not copied. Seven lines in chat_empty_reply_tests.rs widen Reply, Scripted, Fx, fixture, ask and cleanup to pub(super). All of them are declaration lines, far from the mod block the other PRs touch. Copying the fixture would have been the parallel-implementation problem I raised on #827.

Terminals are counted by frame, not by substring. A model can legitimately discuss SSE and write event: done inside an answer. a_terminal_is_a_frame_not_a_substring pins that separately and needs no database — it is the floor the other three assertions stand on.

Not in the table yet

The injection手段 for these arrive with their own PRs, and each becomes one more row rather than another test file:

Validation

Linux/Windows-agnostic; run here against an isolated database templated from a clean base at migration 69.

cargo test -p utopia-server chat_terminal_tests -- --test-threads=1
  a_terminal_is_a_frame_not_a_substring ... ok
  a_turn_ends_in_exactly_one_earned_terminal ... ok
  2 passed; 0 failed

cargo fmt --all -- --check clean, cargo clippy -p utopia-server --all-targets -- -D warnings clean.

No CI step is added — this runs inside the existing utopia-server test target, and my note on #852 argues the five separate chat steps there should collapse rather than grow.

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: WaylandYang <wayland0916@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant