Skip to content

fix(*): report a delegated turn's real outcome and cost - #1121

Merged
0xKT merged 8 commits into
mainfrom
fix/acp_failures_read_as_success
Oct 27, 2026
Merged

0xKT merged 8 commits into
mainfrom
fix/acp_failures_read_as_success

Conversation

@LivXue

@LivXue LivXue commented Oct 27, 2026 •

Copy link
Copy Markdown
Member

Summary

A sub-agent run could end without doing its work and still be recorded as a success, and what ACP turns cost was either missing or misreported. This fixes five such paths. Each was reproduced on main first with Raven's real classes against a stub ACP agent, and the first and last also with real raven acp processes.

  • A turn its own budget ended read as finished. A Raven ACP agent whose turn was ended by its tool-call cap, wall clock or a stalled tool answered stopReason: end_turn, so the host took the wrap-up for the finished work. All five shipped specialists are such agents. The loop now stamps budget_stop_turn_at on the session beside the existing output_limit_turn_at, and the ACP server refines end_turn to max_turn_requests, which the host already marks as partial. The wall clock and a stalled tool map to the same value: the schema has no other word for a turn its own budget ended, and end_turn would claim it finished. The in-process backend appends the same kind of partial notice when its rounds run out, and keeps it out of the instance's own history so a resumed instance does not read it as its own words.
  • Copilot's provider refusal returned as the result. GitHub Copilot writes a refused model call (402, expired plan, bad key, unknown model) as the ordinary reply of a finished turn. Connect already read it; a dispatch returned the sentence as its result. The ACP backend now raises AcpProviderRefusalError with Copilot's reading of it, on the remote pick-up path too. Only a reply that opens like one of Copilot's own measured refusal sentences, and is as short as one, is read this way: an answer that opens with an error it found (Error: 502 Bad Gateway from staging) is still an answer, and a long reply is never run through the patterns (a 2.2 MB reply took 23 s through them, now under 1 ms). Connect reads the agent's own words off the error, so its text is not explained twice.
  • A lost resume answered silently. When a stateful instance's session/load was refused, a fresh session answered with no word that the earlier conversation was gone. The next reply on that handle now carries a [raven] note. The note is held per session and handle until a reply delivers it, so a first turn that fails does not lose it.
  • No tokens or cost for third-party ACP turns. The run record read token counts only from usage_update, where today's adapters do not put them. Counts are now read from the prompt response's usage where the dialect says it counts the whole turn, for the adapter version agentInfo reports. claude-agent-acp sums the turn, and so does codex-acp from 2.2.1, the pinned 2.2.2 included: it reports the thread's total at the turn's end less its total at the start. codex-acp 2.1.1 and earlier sent only the last model call (tokenUsage.last), so their counts stay unknown rather than undercounted. A third-party agent's turn is also booked in the usage ledger under the delegating conversation, as in-process sub-agent calls already are. Its cost is the increase in the session's cumulative cost, which is what the ACP schema defines usage_update.cost as; it is recorded as unknown for a session resumed after the process started. The schema's usage_update has no token counts, so a turn can arrive with a cost and no counts that are the turn's: it is booked with its cost and its counts stored as missing, not as zeros. A turn with neither is not booked. Raven agents are not booked here, because they book their own usage through the prompt's raven.usage.
  • Raven's own ACP server reported the turn's cost as the session's. The server sent each turn's cost as usage_update.cost, so a client that follows the schema showed the last turn's spend as the whole session's. The translator now keeps each session's running total and sends that. A session opened with session/load or session/resume starts from what it had already spent, read once from session.usage (the account that outlives the process); when that cannot be read, no cost is sent rather than a total counted from zero.

Also: Session.clear and undo_last_turn now forget the two turn markers, so a command right after /new is not reported as cut. This fixes the same hole in the existing max_tokens marker. The stopReason and usage_update rows of the ACP compatibility spec are updated to match.

Existing assertions that pinned the old behaviour were changed on purpose: TestUsage in tests/test_acp_updates.py asserted a turn's cost_usd as cost, two in-process tests asserted that a wrap-up after the rounds ran out is the whole reply, and test_prompt_usage_is_read_only_where_it_counts_the_turn asserted that no codex-acp version's prompt usage is read.

Known gaps, deliberately not fixed here:

  • codex-acp sends no cost in usage_update (1.13.1 and 2.2.2 alike, only used and size), so a Codex turn is booked with its counts and an unknown cost.
  • Per-turn token counts have no stable ACP field. The usage on the prompt response is an UNSTABLE draft, and the current draft has moved it to an IdleStateUpdate, so Raven's own server does not send counts yet and the five specialists' token columns stay empty.
  • The CLI backend's fallback after a failed resume is still silent. It only serves rows configured before their preset moved to ACP.
  • The pending lost-resume note and the client-side per-session spend live on the backend object, so a restart or a config hot-apply drops them. The next turn then carries no note, and a resumed session's cost is recorded as unknown.

Type

  • Fix
  • Feature
  • Docs
  • CI / tooling
  • Refactor
  • Other

Verification

  • Each fix was written test first and watched fail before the change. Mutants on the decisions were each killed: dropping Copilot's whole-message check, reading both usage sources, swapping the two stop reasons' order, reading Copilot's sentences for another preset, booking the reported total instead of the increase, treating an unknown baseline as zero, booking Raven agents, sending a turn's cost instead of the session's, ignoring an unreadable earlier spend, returning early again when counts are unknown, booking only when both counts and cost are known, writing unknown counts as known, moving the codex-acp version floor down to 2.1.1 or up to 2.3.0, and dropping the version on its way to the turn.

  • Full suite, python -m pytest -q -n 4 (lockfile identical to main): 110 failed, 36171 passed, 255 skipped under the repository venv. 102 of them are node and remote tests whose node daemon starts with python -I and so imports the installed checkout rather than the one under test; rerun with the branch's own venv, the 14 files holding all 110 fail only 8. Those 8 (provider proxy redaction, the node-dir resolver of the install script, a legacy Excel workbook, the node runtime reader) fail with the same IDs on unmodified main under its own venv: 0 introduced.

  • Focused: tests/test_acp_dialects.py, tests/test_subagent_acp.py, test_subagent_github_copilot.py, test_subagent_manager.py, test_acp_methods.py, test_acp_updates.py, test_agent_loop_max_iter_synthesis.py, test_remote_resume.py, test_session_manager.py, test_token_wise_usage_tracker.py pass.

  • End to end, budget stop: a real raven acp child with maxToolIterations 2 behind a stub model server now answers max_turn_requests, and the host's reply ends with its partial notice.

  • End to end, cost: real raven acp processes behind a model server billing 0.0125 per call. Two turns, then a third in a new process after session/load, report 0.0125, 0.025 and 0.0375; main reports 0.0125 each time.

  • End to end, Codex: one real codex-acp 2.2.2 turn through AcpAgentBackend. Its usage_update is used 18784 and size 475000 with no cost, and its prompt usage counts the turn (6491 input, 12288 cached reads, 5 output, 18784 total). The run record now shows 6491 / 5, and the ledger books one call with those counts and an unknown cost; before, the counts were left unread and nothing was booked.

  • ruff check, ruff format --check and ty check on the changed files; lint-imports (10 contracts kept); scripts/check_commit_messages.py, scripts/check_large_files.py and scripts/check_source_language.py over the branch: all pass.

  • Relevant tests pass locally

  • Relevant lint / type checks pass locally

  • User-facing docs or screenshots are updated when needed

Risk

User-visible changes: a capped sub-agent run's reply now ends with a partial notice; a Copilot run whose reply is a provider refusal now fails instead of completing; a reply after a lost resume carries a note; usage and spend totals now include third-party ACP agents' turns, so they rise for anyone delegating to claude-agent-acp; an ACP client of Raven now sees the session's running cost in usage_update.cost instead of the last turn's, and can receive max_turn_requests, a value the schema defines. Opening a session with session/load or session/resume now reads its usage record once, the same read the session's usage panel makes. Rollback is reverting the squash commit; nothing is migrated or stored in a new shape.

  • Security impact considered
  • Backward compatibility considered
  • Rollback path is clear for risky changes

Related Issues

N/A

@LivXue
LivXue requested review from 0xKT and gloryfromca October 27, 2026 02:48

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: Codex ACP turns still drop their reported cost from the delegated usage ledger.

I reviewed the full diff plus the affected ACP server/client callers and commit history. I also checked AGENTS.md and the relevant CONTEXT-MAP.md/CONTEXT.md contracts, backward compatibility, the updated tests for weakening, and the layer/import architecture. The outcome/refusal/partial-turn paths otherwise look coherent.

Verification: uv run pytest tests/test_acp_methods.py tests/test_acp_updates.py tests/test_agent_loop_max_iter_synthesis.py tests/test_remote_resume.py tests/test_session_manager.py tests/test_subagent_acp.py tests/test_subagent_github_copilot.py tests/test_subagent_manager.py -x passed 1169 tests after uv sync --all-packages; the first attempt failed during fixture setup because the fresh environment lacked raven_everos. uv run pytest tests/test_l4_entrances.py -x passed 6 tests. An isolated Codex ACP reproduction reported a turn cost of $0.0125 but yielded zero delegated ledger calls and no cost.

Comment thread raven/acp_client/acp_agent.py Resolved
@LivXue

LivXue commented Oct 27, 2026

Copy link
Copy Markdown
Member Author

Windows compatibility review — PR #1121 @ 9a5925323c1a0eb9e2fd1f592ddc9d3c4564418f

Scope: full base 4718421324 → this head (24 files): the ACP server/client outcome refinements (raven/acp/{methods,protocol,updates}.py, acp_client/**), delegated-turn cost/outcome plumbing in raven/agent/loop/turn_path.py, subagent/**, session/manager.py, and the tests including tests/acp_stub_server.py.

Result: no Windows-specific issue found.

What I checked:

  • A scan of the production diff found no OS-sensitive construct added: no subprocess creation, signals, process groups, sockets, filesystem or path handling, permissions, symlinks, or new I/O encoding. The ACP process transport (acp_client/windows.py, the launcher) is untouched.
  • The changes are protocol/verdict logic: budget_stop_turn_at stamping, refinement of end_turn to max_turn_requests, partial notices, Copilot refusal classification, and cost/outcome recording. All platform-neutral.
  • The test-helper changes (tests/acp_stub_server.py) are in-process protocol behavior, not process spawning.

Verification (native Windows 11, CPython 3.12.13, isolated worktrees D:\tmp\rv-1121 and D:\tmp\rv-1121-base, scratch HOME/USERPROFILE, PYTHONUTF8=1):

pytest tests/test_acp_methods.py tests/test_acp_updates.py \
       tests/test_agent_loop_max_iter_synthesis.py tests/test_remote_resume.py \
       tests/test_session_manager.py tests/test_subagent_acp.py \
       tests/test_subagent_github_copilot.py tests/test_subagent_manager.py -q
→ head 9a5925323c: 52 failed, 1116 passed, 1 skipped
→ base 4718421324: 52 failed, 1066 passed, 1 skipped

The failing IDs are identical between head and base (diffed as sets — no additions, no removals); they are the known Windows failures in test_subagent_manager.py (spawn/path/symlink fixtures) and their siblings, inherited from main. The 50 new head-only passing tests are this PR's.

Limitations: the PR body's real raven acp process reproductions were not repeated here; the run exercised the in-process stub and Raven's real classes only. The full suite was not run.

A sub-agent run could end without doing its work and still be recorded
as a success. Four ways, each reproduced on main first:

- A Raven ACP agent whose turn its own budget ended (the tool-call cap,
  the wall clock or a stalled tool) answered stopReason end_turn, so the
  host took the wrap-up for the finished work. It now answers
  max_turn_requests, which the host already marks as partial, and the
  in-process lane marks the same case in its reply.
- GitHub Copilot writes a refused model call (402, expired plan, bad
  key) as the ordinary reply of a finished turn. A dispatch returned
  that sentence as its result; it now fails with Copilot's reading of
  it, as Connect already did, and so does the remote pick-up path. Only
  a reply that opens like one of Copilot's own refusal sentences is
  read this way, so an answer that quotes an error is still an answer.
- When a stateful instance's session could not be resumed, a fresh
  session answered with no word that the earlier conversation was gone.
  The next reply on that handle now says so, even after a failed turn.
- ACP turns recorded no tokens and no cost. Tokens are read from the
  prompt response's usage where the adapter counts the whole turn
  (codex-acp sends its last call only, so not there), and a third-party
  agent's turn is booked in the usage ledger, its cost taken as the
  increase in the session's cumulative cost, as ACP defines it.

Session.clear and undo_last_turn also forget the turn markers the stop
reason reads, so a command after /new is not reported as cut.
Lane is the scheduler's per-conversation serial domain (CONTEXT.md).
The comments and test docstrings this branch added used the word for
the ACP and in-process sub-agent backends instead.
Copilot's measured refusals run to under 300 characters, and a long reply is work whatever it opens with. A long reply that opened like a refusal and never closed walked the refusal patterns once per restart: 1.6 s at 0.56 MB and 23 s at 2.2 MB on the event loop. Replies over 1000 characters are no longer read, and the same reply now takes under a millisecond.
The cumulative cost is kept to measure a resumed session's next turn by. A stateless agent opens a fresh session per dispatch and never resumes one, so an entry per dispatch only grew for the life of the process.
ACP defines usage_update.cost as the session's cumulative cost; the
server sent the turn's cost, so a client that follows the schema showed
the last turn's spend as the whole session's. The translator now keeps
each session's running total and sends that. A session opened with
session/load or session/resume starts from what it had already spent,
read once from session.usage, which outlives the process; when that
cannot be read, no cost is sent rather than a total counted from zero.

Measured with real raven acp processes and a model server billing
0.0125 per call: two turns then a third after a restart and a load now
report 0.0125, 0.025 and 0.0375; main reports 0.0125 each time.
codex-acp 2.2.1 builds the prompt response's usage from the thread's
token total at the turn's end less its total at the start; 2.1.1 and
earlier sent tokenUsage.last, the turn's final model call. The dialect
flag was measured on 1.13.1 and stayed false after the pin moved to
2.2.2, so every Codex turn's counts were left unknown.

The flag is now a method of the version agentInfo reports. A version
that is no release number is read as the older kind.
usage_update carries the context window and the session's cumulative
cost but no token counts, so a turn can arrive with a cost and no counts
that are the turn's. The ledger write returned early on missing counts
and dropped the cost with them. The turn is now booked when either is
known; its unknown counts are stored as missing rather than as zeros,
and a turn with neither is not booked.
@LivXue
LivXue force-pushed the fix/acp_failures_read_as_success branch from 9a59253 to a8a83dd Compare October 27, 2026 07:41
@LivXue
LivXue requested a review from gloryfromca October 27, 2026 07:42
…spellings

codex-acp publishes -preview.N builds. The floor reads a pre-release as
its release: 2.2.1's previews already counted the turn, and the last
previews of 2.1.2, never released, read as the older kind, leaving
their counts unknown rather than wrong.

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blockers; this can merge as far as I am concerned.

I reviewed the new cost-only booking and version-aware Codex usage commits against the current PR diff, their accounting callers, backward compatibility, the existing outcome paths, AGENTS.md and the relevant CONTEXT contracts. I also checked that the added tests preserve the old-version safeguard rather than weakening it. The prior blocker is fixed, its reproduction now passes, and I resolved my review thread.

Verification: uv run pytest tests/test_acp_methods.py tests/test_acp_updates.py tests/test_acp_dialects.py tests/test_agent_loop_max_iter_synthesis.py tests/test_remote_resume.py tests/test_session_manager.py tests/test_subagent_acp.py tests/test_subagent_github_copilot.py tests/test_subagent_manager.py tests/test_token_wise_usage_tracker.py tests/test_l4_entrances.py -x passed 1,281 tests.

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blockers; this can merge as far as I am concerned.

This revision adds only coverage for the existing codex-acp version-floor behavior: 2.2.1 preview builds use whole-turn usage, while 2.1.2 previews retain the conservative older reading. I checked the current diff, commit history, compatibility boundary, test strength, and the standing resolved thread; no production code or architecture contract changed.

Verification: uv run pytest tests/test_acp_dialects.py tests/test_subagent_acp.py tests/test_token_wise_usage_tracker.py -x passed 455 tests.

@LivXue

LivXue commented Oct 27, 2026

Copy link
Copy Markdown
Member Author

Windows compatibility re-review — PR #1121 @ 91de6b29d32e78f10d90d2b74a0ef617e8e7ddec

Scope: the head moved from the previously reviewed 9a592532. The delta is main's merges plus ACP dialect work: an agent_version(initialize) helper, prompt_usage_counts_the_turn made version-aware, the collector carrying that version, and ledger counts allowed to be absent when a cost exists (counts_known). The rest of the raw delta is main.

Result: no Windows-specific issue.

  • The changes are protocol/ledger logic: _agent_info/agent_version read a dict from the handshake; the dialect predicate takes a version string; note_ledger_usage gains a boolean. No subprocess, path, filesystem, signal, encoding, or platform primitive is added.

Verification (native Windows 11, CPython 3.12.13, isolated worktrees D:\tmp\rv-1121-n and D:\tmp\rv-1121-nbase, scratch HOME/USERPROFILE, PYTHONUTF8=1):

pytest tests/test_acp_methods.py tests/test_acp_updates.py \
       tests/test_subagent_acp.py tests/test_subagent_github_copilot.py -q
→ head: 35 failed, 722 passed, 1 skipped in 30.29s
→ base 4718421324: 35 failed, 674 passed, 1 skipped in 27.86s

The failing-ID sets are identical (diffed — no additions, no removals); they are the inherited ACP/third-party spawn and path fixtures already present at base. The 48 extra passing tests at head are this PR's.

Limitations: no real raven acp process was run; the run exercised the in-process stub and Raven's real classes only.

@0xKT

0xKT commented Oct 27, 2026

Copy link
Copy Markdown
Member

Not a blocker -- three points from the acceptance pass, none of which stops the merge. All ten changed test files pass on this head merged onto main (1277 passed, run in an isolated tree).

Severity of this one finding, not a verdict on the pull request. Any blocking findings from this pass are review threads on the changed files; GitHub renders those collapsed, as a file name with no text.

  1. The Copilot refusal is checked against the whole turn's text, not the reply the caller gets. _provider_refusal(text) receives text = collector.text (raven/acp_client/acp_agent.py:1993, checked at :2046), which joins every agent message of the turn. read_reply only accepts input that is at most _REFUSAL_MAX_CHARS and starts with one of the refusal openings (raven/agent/subagent/github_copilot.py:120-121). Take a Copilot dispatch that narrates first ("I'll read the README.") and then hits a refused model call ("Error: 402 Your credit balance is too low..."): the joined text does not start with the opening, so the check returns None. The run then completes with the refusal sentence as its result, which is the outcome d3da12e says it removes. remote_resume.py has the same text-vs-reply split. Checking collector.reply (the last message, which is what the caller receives) closes it without dropping anything the caller sees.

  2. The ACP cumulative cost can count a turn twice and drifts from session.usage. _deliver adds every message.complete usage cost_usd to the session total (raven/acp/updates.py:949-951). A CRON/wake turn on the ACP lane never writes its own usage (raven/rpc/spine.py:241), and the usage slot is popped only when the ending turn owns the lane (:596-604). So after a non-owning turn such as a sub-agent result re-entering the session, each later wake re-emits that turn's usage and the total grows by it again. In the other direction, cancelled and failed turns, and priced calls outside the decide step, reach telemetry (the seed's source, raven/acp/methods.py:402) but never the running total, so it reads low until the next load re-seeds it. Reporting only; billing and telemetry are unaffected. Not executed: a spine-level probe with a SUBAGENT turn followed by a CRON turn on one conversation would show the second message.complete carrying the first's cost.

  3. Smaller:

    • No test pins that a refused or empty ACP turn still books its spend.
    • The turn-marker keys (output_limit_turn_at, budget_stop_turn_at) are spelled at three sites: the writer, the ACP reader and Session._forget_turn_markers.
    • _turn_cost / _counted re-decide the reported-cost and count rules that providers/usage.py owns.

@LivXue

LivXue commented Oct 27, 2026

Copy link
Copy Markdown
Member Author

Thanks -- all five are taken in #1133, since this PR merged before they landed.

  1. Copilot refusal checked against the whole turn: confirmed. A Copilot dispatch that narrates ("I'll read the README.") and is then refused with a 402 returned the refusal as its result. Both checks now read collector.reply, the last message, which is what the caller receives (5bd10a9); the empty-turn check still reads the whole turn.

  2. Cumulative cost counting a turn twice: confirmed, and fixed in the spine rather than in the ACP translator. Deduplicating by turn_id in _deliver would not have caught it: the wake's message.complete carries the wake's own turn id with the earlier turn's usage. The CRON branch of RpcTurnRunner.run now passes a usage_sink like every other turn and stashes what it spent (ee59341), so a wake reports its own usage and cost -- to an ACP client, and to the TUI and web footers, which were shown the stale numbers too. The other direction stays as you describe: a cancelled or failed turn and a priced call outside the decide step are not in the running total until the next load reads it again. That is listed as a known gap.

  3. Smaller:

    • Refused and empty turns still booking: test_a_turn_that_answered_nothing_usable_still_books_its_spend drives both through a real backend and reads the ledger back; it fails when the booking is skipped for them (532e05f).
    • Turn-marker keys: OUTPUT_LIMIT_TURN_AT, BUDGET_STOP_TURN_AT and TURN_MARKERS in raven.session.manager, used by the writer, the ACP reader and _forget_turn_markers (6d8d960).
    • Duplicate rules: the client's _counted / _turn_cost and the server's _usd now use token_count / reported_cost (bfda01b). That closed a real gap: an inf amount passed all three, and a nan passed the client's check, since amount < 0 is false for nan. Either one as a session's spend baseline left every later turn of that session unpriced.

0xKT pushed a commit that referenced this pull request Oct 28, 2026
… silence (#1092)

## Summary

Supersedes #1052, which was closed unmerged. That PR ended a turn
without a reply whenever the model returned nothing. After a survey of
how other agents handle the same replies, empty and abnormal replies now
follow Hermes Agent, and Raven keeps its reasoning-effort step-down only
for a reply that stopped while still thinking. This also ends the Codex
HTTP 400 "Unsupported value: 'minimal'": the old empty-reply recovery
lowered the effort on every resend, down to `minimal`, which Codex
refuses. Neither recovery nor the in-turn compaction summary asks for
`minimal` any more; a session whose own effort is `minimal` still sends
it, and the compacting history strategy's checkpoint summary goes out at
the provider's configured effort.

Silence is now something the model says, not something it omits:

- A reply that is a silence marker ends the turn without a reply. A
  turn a person started honours only the bracketed `[SILENT]` the
  runtime teaches, as the whole reply (whitespace aside): such a turn is
  never taught a marker, so `Silent.` or `No reply.` there is the
  person's answer. That is stricter than Hermes, whose strict rule takes
  all four spellings. A proactive turn (cron, heartbeat, Sentinel, a
  sub-agent report) uses Hermes' rules over all four (`[SILENT]`,
  `SILENT`, `NO_REPLY`, `NO REPLY`): the whole reply, at most 64
  characters, or the marker on the first or last line. An A2A turn,
  whose peer waits on the text, gets its reply back as written.
- The marker rules live in `raven/agent/context/silence.py`, outside
  the kernel. The kernel carries only the outcome: `TurnOutcome.silent`
  and the `silent_turn` notice.
- The cron reminder and the host's note on an unprompted sub-agent
  report teach the marker, in Hermes' words.
- Streaming holds back text that may still turn out to be a marker, so
  a silenced reply never flashes on screen. On a person's turn only a
  reply that opens with whitespace or `[` is held, and an inline think
  block streams as it arrives while what follows it is held; a
  proactive turn holds the whole call. The hold lasts across a stream
  cut and its continuation.
- The marker row is stored with `silent: true` and the turn ends with a
  `silent_turn` notice; the TUI and the web UI draw one muted line
  ("Raven chose not to reply."). ACP replay, the session list, the
  session export and the cron run list do not show it as a reply. A
  reply an `after_send` hook composes in place of the marker is
  delivered and stored as that reply.
- A channel stops its typing indicator (`end_turn`) when a turn a user
  started, or one a user's message was merged into, ends without a
  reply. A silent cron run is a success: it wakes nobody and does not
  count toward the silent-fire limit.

An empty reply is recovered by kind, decided by two pure functions
(`reply_kind`, `decide`):

- only thinking, no text (cut at the output limit or not): two
  prefills when there is thinking to re-feed, then the same step asked
  again at `high`, then `low` (only the rungs below the turn's effort,
  each once per turn), then the fallback models, then a failure;
- plain empty: resent after Hermes' jittered backoff, up to three
  times, or once when the call reported a cost at or above
  `emptyRetryCostThresholdUsd` (default 0.25 USD); two zero-output
  replies from one model stop the resends early. On a streamed turn
  the first resend goes out without streaming (see below);
- empty after a tool call: Hermes' nudge;
- cut at the output limit with neither text nor thinking: a
  continuation with max_tokens doubled each time (x2, x4, x8), held to
  the model's own output ceiling and to the room its prompt leaves;
- a refusal or a content filter (read off the finish, or off the
  `refusal` field, which litellm moves into `provider_specific_fields`):
  never resent; one fallback model, whose answer opens a step of its
  own, then a failure that quotes the model's explanation;
- fallback models are tried once each, and a switch holds for the rest
  of the turn.

Failures are worded after Hermes, per kind, and `no_output` (on
`TurnFailed` and `AnswerlessTurnError`) marks a turn whose last reply
was empty. A transient `llm_retry` notice says when a resend is
waiting.

A stream that dies after showing text is continued (up to three times,
stitched into the same reply). If it stays cut, or the turn runs out of
iterations or time while a continuation is owed, the partial is
delivered with a note that it is partial, status `interrupted`, instead
of failing the turn. A cut is read before any marker, so a fragment that
happens to spell one is continued, and a stream that had already
received a refusal finish keeps it; the reply the turn ends on, the
joined parts with their closed think blocks taken out, is then read for
a marker like any other, and parts that join into nothing but a thought
take the thinking-only path an uncut thought takes. A continuation that
calls a tool keeps the words the reader saw on that call's row, as they
came. Providers now report the finish they actually got: `unknown` for a
missing one, `refusal` / `content_filter`, and an error for a 200
response that carries one. ACP reports a turn its own budget ended as
`max_turn_requests` (#1121); a cut stream delivered as a partial spent
no budget and stays `end_turn`.

Decisions beyond the design, so a reviewer does not rediscover them:

- Only a real mid-stream death is continued. The stream hand-back marks
  it `finish_reason="stream_cut"`; a reply that merely lacks a finish is
  delivered as is. Continuing on a missing finish would draw three
  continuations on every reply of a backend that never sends one.
- The output-limit continuation and the post-tool nudge keep Hermes'
  frame, but say that the cut left nothing and that the model may
  answer or make the call the task needs: Raven sends them only for a
  cut that produced nothing, which Hermes' wording does not fit.
- Once the message tool has replied in a turn
  (`MessageTool.sent_in_turn`: its send to the turn's own chat went
  out), an empty last reply that a nudge or a prefill does not mend ends
  the turn as answered, without the paid resends, step-downs and
  fallbacks; a refusal there ends it the same way. A marker after such a
  reply does not make the turn silent, so no `silent_turn` notice is
  sent.
- On a streamed turn, the first resend of a plain empty reply asks the
  same model without streaming, with no provider-side fallback, and its
  answer is shown through the same sinks and gates a stream's deltas
  pass; later resends stream again. A stream can come back empty where
  the plain call does not: the locked litellm 1.85 drops a streamed
  chunk that carries only a refusal, so a refusal an OpenAI-compatible
  model streams arrives as an empty stop, and the plain call reads it
  and hands it to the refusal path. A gateway whose streams fail is
  rescued the same way.
- Sub-agent and DAG lanes keep their own recovery; besides the note
  above, only the providers' finish reasons reach them (see Risk).
- `after_send` hooks still run on a silent turn, with
  `outbound_content=None`; `FACTORY_LOOP_SURFACE_VERSION` is now 4.

How a stored turn is read: the live turn files `replied_via_tool` on the
last assistant row a turn the message tool answered stores, and the
session list and the ACP replay read that flag from one place,
`raven/session/turn_shape.py`, instead of guessing from the calls they
can see. A send that failed or went to another chat answered nobody, so
both surfaces agree with the live view about which turn answered.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

At the PR head, on origin/main 06509f8 (Linux, 32 cores).

- The full Python suite: 36878 passed, 9 failed, 255 skipped, 2
  xfailed.

  ```
nice -n 19 uv run --frozen --all-extras pytest -n 4 -q -p
no:cacheprovider
  ```

  The same 9 fail on a clean archive of origin/main 06509f8 on this
  machine: five proxy tests in tests/test_config_update_providers.py,
  test_install_script, test_knowledge_parser_excel,
  test_subagent_node_runtime, and test_cli_agent_commands'
  empty-message-file test, whose error box is cut to the terminal
  width. They come from the environment, not from this branch.
- ui-web `npx vitest run`: 260 files, 4461 passed.
- ui-tui `npx vitest run --maxWorkers=2`: 145 files, 2275 passed, 13
  skipped.
- `make lint-python lint-imports lint-deps lint-types lint-ui lint-tui
  check-commits check-large-files check-source-language`: all exit 0.
  ui-tui `npm run lint:i18n` and `npm run lint:rpc`, and ui-web
  `npm run gen:check`: exit 0.
- The review round's fixes were each written test first, and each was
  checked by removing or inverting it and watching a test go red. An
  independent re-review repeated those mutants, probed the stream and
  marker cases through the real loop against the previous head, and
  fuzzed the stream gate over about 35,000 random two-part cut replies
  without finding a silent turn that showed its marker.
- The rebase onto 06509f8 replayed every commit unchanged; its one
  meeting point with main, #1121's budget-stop marker, is described in
  the summary. Earlier rebases met #1031 (compacting history), #1078
  (refused cancel), #1119 (the context ring a resume restores), #1047
  and #1114; what each needed is pinned by tests on this branch.
- Prompt text, by a paper exercise with fresh model readers, at an
  earlier head (no prompt text has changed since): a routine sub-agent
  report was answered with `[SILENT]` and a finished one with a real
  report; a plain one-shot reminder was still delivered; an hourly
  watch with nothing new stayed silent. With the text before this
  change, the two routine cases produced a message.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

What a user will notice: a reply that is only a silence marker is not
delivered; an empty reply is asked again, stepped down or switched to a
fallback before the turn fails, and fails in Hermes' wording; a cut
stream is continued and, if it stays cut, delivered as partial instead
of failing the turn.

Known gaps, each left as it is here:

- A continuation request that raises (a non-retryable error, a cancel,
  or a retryable error that outlasts the stream's own ladder, about
  105 s) fails the turn although the reader saw the partial.
- An error the provider raises mid-stream (LiteLLM raises its HTTP
  errors) is not switched to a fallback model: it is waited out and
  fails the turn, as before. A stall before any output, and an error
  that comes back as a response, do fall back.
- On a plain (non-streamed) call, a fallback the provider switched to
  on its own is not known to the loop, so a later empty-reply fallback
  can ask that model again.
- A refusal a fallback model then answers streams live in a step of
  its own, and only the fallback's answer is stored, so a reload shows
  the answer alone. Storing the refused words would need a row kind
  shown to the reader but never sent to the model, and none exists.
- When the message tool replied and the model then ended on a marker,
  a reload draws the muted line after the reply; on the web the reply
  then sits in the turn's folded steps.
- A silent turn the runtime opened is left out of an ACP session/load
  replay with its tool calls, as the session list leaves it out,
  unless the message tool replied in it.
- The cron rule applies to every proactive origin, including sub-agent
  result turns that are never taught the marker: such a reply whose
  first or last line is a bare "No reply" or "Silent" is not delivered.
- Silent cron runs never reach the silent-fire limit, so a recurring
  job that always stays silent is never paused, and each run adds about
  3.5 KB to its session file and curator traces until consolidation
  trims them. A job whose model always returns nothing keeps firing,
  visible only as lastStatus=error.
- A refusal an OpenAI-compatible model streams costs one extra call
  before it is handled as a refusal, and only the turn's first plain
  empty resend can read it: a fallback model is never resent to, so a
  streamed refusal from a fallback still reads as an empty reply. The
  unstreamed resend's answer is shown whole rather than as it is
  written, has none of a stream's idle watchdogs, and shows only the
  retry notice while a long-reasoning model works; a proxy that caps
  plain responses turns it into an error, which takes the usual error
  path. A test pins what litellm 1.85 passes on, so an upgrade that
  starts passing the words is noticed.
- The cost guard reads only a reported `usage.cost_usd`, so it does
  nothing on direct OpenAI, Azure, Codex or the Anthropic API unless a
  gateway adds the cost.
- The failure for a thinking-only reply quotes up to 500 characters of
  the model's thinking in the TUI and web detail; channels get the
  bounded 200-character text.
- `stream_cut` shows as the call's finish in the trace viewer.
- Sub-agent runs keep their own recovery, and with #1047 merged they
  read the providers' honest finish reasons this way: a refusal with
  text is the run's answer; a refusal, content filter or missing finish
  with no text is a blank reply, which gets the one wrap-up call and
  then fails as having no answer; a 200 response carrying an error fails
  the run as a failed model call, with no wrap-up. An empty stream whose
  prompt was never read is asked once more without streaming in a
  sub-agent's direct chat before it fails.
- Near the context window, an output-limit continuation asks for no more
  than the room its prompt leaves, as the first call did; a reply cut
  there with nothing in it is therefore usually cut again and fails
  after its continuations.
- A cut keeps its two scaffolding rows (the words so far and the ask
  to go on) in the window for the rest of the turn, so a continuation
  that calls a tool shows the model the partial twice, and a cut that
  joins into a bare thought is prefilled, stepped down or sent to a
  fallback after them: the re-ask reads as "continue the cut thought",
  and a model that goes on mid-thought has the thought's tail delivered.
  The stored history has neither row. Taking the rows out lost a hook's
  note and left the question autofill reading an old list.
- Three rare cut shapes still show or deliver part of a marker: a cut
  whose continuation is refused and then answered by a fallback shows
  the held marker in the refused step; a cut marker prefix whose
  continuation starts the marker over (`[SILEN`, then `[SILENT]`)
  delivers the fragment, as before; a hook rollback of the
  continuation's tool call drops the partial from that call's row, as
  before.
- A cron turn that took in a person's message and then fails with no
  output leaves that chat's typing indicator running until the next
  send.
- When a refused stop holds the lane, the stopped turn's late end is
  dropped, and with it any silent mark that turn set, so the next turn
  is not drawn as silent.
- Unchanged from main: the think-block pattern and the stream's opening
  hold are quadratic on very long runs of tags or whitespace.

Compatibility and rollback:

- #1031 (merged) also changes `raven/session/manager.py`: it adds a
  caller of `_scan_file` and changes neither its return shape nor the
  silent-turn rule.
- `after_send` hooks now also run on a silent turn with
  `outbound_content=None`; an out-of-tree hook that assumes a string
  must accept None.
- The new fields (`silent` and `replied_via_tool` on stored rows and on
  `TranscriptMessage`, `no_output` on failures, the `silent_turn` notice
  kind) are optional; an older Raven reading a session written by this
  one shows a marker row's text as a reply.
- New config key `agents.defaults.emptyRetryCostThresholdUsd`
  (default 0.25). Rollback is a revert of the squash commit; there is
  no data migration.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A. Supersedes #1052 (closed unmerged).
LivXue added a commit that referenced this pull request Oct 31, 2026
#1121 added a reader for agentInfo.version that never parses protocolVersion, and the private copy here was written before it existed. Delegate to it and keep only the nonfatal guard the reviewer asked for.
LivXue added a commit that referenced this pull request Nov 1, 2026
#1121 added a reader for agentInfo.version that never parses protocolVersion, and the private copy here was written before it existed. Delegate to it and keep only the nonfatal guard the reviewer asked for.
LivXue added a commit that referenced this pull request Nov 1, 2026
#1121 added a reader for agentInfo.version that never parses protocolVersion, and the private copy here was written before it existed. Delegate to it and keep only the nonfatal guard the reviewer asked for.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants