Skip to content

fix(*): follow up #1121 on Copilot refusals, wake usage and reported costs - #1133

Open
LivXue wants to merge 5 commits into
mainfrom
fix/acp_review_followups
Open

LivXue wants to merge 5 commits into
mainfrom
fix/acp_review_followups

Conversation

@LivXue

@LivXue LivXue commented Oct 27, 2026 •

Copy link
Copy Markdown
Member

Summary

Follow-up to #1121, which merged before its last review round landed. The acceptance pass on #1121 raised five points; this takes all five.

  • A Copilot refusal after an earlier message read as the result. The refusal check read the turn's whole text, every agent message joined. Copilot's refusals are matched by their opening and a short length, so a dispatch that narrated first ("I'll read the README.") and was then refused with a 402 never matched, and the refusal came back as the run's result. The check now reads collector.reply, the last message, which is what the caller receives, on the local and the remote pick-up path. The empty-turn check still reads the whole turn.
  • A wake turn reported the turn before's usage. A message.complete reports the lane's usage slot in the RPC spine. A cron or wake turn never wrote that slot, and a turn that does not own the lane, such as a sub-agent result re-entering the session, never clears it, so the next wake resent that turn's usage and cost. An ACP client of Raven added the same cost to the session's running total a second time, and the TUI and web footers were shown the stale numbers. The CRON branch of RpcTurnRunner.run now passes a usage_sink like every other turn and stashes what it spent. Deduplicating by turn id in the ACP translator would not have caught this: the wake's message.complete carries the wake's own turn id.
  • One rule for reported costs and counts. The ACP client's _counted and _turn_cost and the server's _usd re-decided what a reported amount and a token count are; they now use reported_cost and token_count from raven/providers/usage.py. That closes a gap: an inf amount passed all three, and a nan passed the client's check because amount < 0 is false for nan. Either one as a session's spend baseline left every later turn of that session unpriced.
  • Smaller. A test now pins that a refused or empty ACP turn still books its spend; only a comment held that order. The turn-marker keys are named once in raven.session.manager (OUTPUT_LIMIT_TURN_AT, BUDGET_STOP_TURN_AT, TURN_MARKERS) and used by the loop that writes them, the ACP server that reads them and the session that clears them. The stored names are unchanged, and the session tests still spell them out.

Known gap, deliberately not fixed here: the running cost counts completed turns. A cancelled or failed turn, and a priced call outside the turn's decide step, reach the usage record but not the running total until the next session/load or session/resume reads it again.

Type

  • Fix
  • Feature
  • Docs
  • CI / tooling
  • Refactor
  • Other

Verification

  • Test first. On this branch's base, with the source changes reverted and the tests kept, six of the new cases fail: the two refusal cases (local dispatch and remote pick-up), the wake usage case and three non-finite cost cases. The other six pass by design: the two guards that a refused or empty turn still books its spend, the case that a turn run without a stream fills its usage sink, and three amounts that were already turned away.

  • Mutants, each killed: booking skipped for an empty or refused turn (both guard cases fail), the wake turn's usage write dropped (the wake case fails, from the runner through the sink to the message.complete payload).

  • Full suite, python -m pytest -q -n 4 with the branch's own venv (lockfile identical to main), on the head rebased onto 959cd0a: 36892 passed, 9 failed, 255 skipped. None of the 9 is this change's: 8 (provider proxy redaction, the node-dir resolver of the install script, a legacy Excel workbook, the node runtime reader) fail with the same IDs on unmodified main; and test_an_empty_message_file_is_a_usage_error fails or passes with the length of the temp path its error panel wraps, and passes under a short --basetemp on this branch and on main.

  • Focused: tests/test_subagent_github_copilot.py, test_remote_resume.py, test_rpc_spine.py, test_agent_loop_usage_sink.py, test_subagent_acp.py, test_acp_updates.py, test_acp_methods.py, test_session_manager.py, test_agent_loop_max_iter_synthesis.py, test_agent_loop_empty_recovery.py, test_gateway_spine.py, test_rpc_turn_send.py, test_turn_wire_shape_conformance.py pass.

  • ruff check, ruff format --check and ty check on the changed files (ty reports 5 diagnostics, all on lines of tests/test_subagent_acp.py this branch does not touch); lint-imports (10 contracts kept); scripts/check_commit_messages.py, scripts/check_large_files.py and scripts/check_source_language.py over the branch: all pass.

  • Relevant tests pass locally

  • Relevant lint / type checks pass locally

  • User-facing docs or screenshots are updated when needed

Risk

User-visible changes: a Copilot run whose last message is its provider's refusal now fails even when it narrated first; after a wake turn, every RPC client is sent that turn's own usage rather than the turn before's, so a wake whose provider reported no usage shows zeros, as any other such turn does; an ACP client's running cost no longer counts the turn before a wake twice; a reported inf or nan cost is read as unknown. Rollback is reverting the squash commit; nothing is stored in a new shape.

  • Security impact considered
  • Backward compatibility considered
  • Rollback path is clear for risky changes

Related Issues

#1121

@LivXue

LivXue commented Oct 27, 2026

Copy link
Copy Markdown
Member Author

Windows compatibility review — PR #1133 @ bfda01b4d133426e6e974ff198f891f15100b6e7

Scope: full base 06509f8e → this head (14 files, +210/−50): the follow-up to #1121 — the Copilot refusal check now reads the last message (collector.reply) on both the local and remote pick-up paths, a cron/wake turn stashes its own usage slot so message.complete cannot report the previous turn's spend, token/cost reads go through providers/usage helpers (token_count, reported_cost), and the session turn-marker keys become named constants; plus the ACP stub and tests.

Result: no Windows-specific issue. The changes are protocol and ledger logic: message text selection, dict slots, numeric normalisation. No path, filesystem, subprocess, signal, encoding, or platform primitive is added.

Verification (native Windows 11, CPython 3.12.13, isolated worktrees D:\tmp\rv-1133 and D:\tmp\rv-1133-main, scratch HOME/USERPROFILE, PYTHONUTF8=1):

pytest tests/test_acp_updates.py tests/test_agent_loop_usage_sink.py \
       tests/test_remote_resume.py tests/test_rpc_spine.py \
       tests/test_subagent_acp.py tests/test_subagent_github_copilot.py -q
→ head: 35 failed, 700 passed, 1 skipped in 36.08s
→ main 06509f8e: 35 failed, 688 passed, 1 skipped in 35.08s

The failing-ID sets are identical (diffed — no additions, no removals): the known Windows ACP/spawn fixtures present on main.

Limitations: no real raven acp process or live Copilot dispatch; the changes are covered through the in-process stub and unit tests.

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blockers; this can merge as far as I am concerned.

Reviewed the full github/main...HEAD diff and the relevant ACP collector/refusal paths, remote resume path, RPC runner/sink lifecycle, cron and session-wake callers, shared usage normalization, session marker readers/writers/clearers, and the #1121 history. I also checked AGENTS.md, CLAUDE.md, and CONTEXT-MAP.md/CONTEXT.md; backward compatibility; the additive test changes and stub behavior; and found no weakened tests. The change keeps wire and stored field names stable and the behavioral fixes match their callers.

Verification: uv run pytest over the 13 focused ACP, RPC, agent-loop, session, gateway, and wire-shape files passed (1222 passed). The first attempt did not reach tests because a fresh partial uv environment lacked the workspace raven_everos package; after uv sync --all-packages --all-groups, the identical test command passed. Ruff check, Ruff format check, source-language check, and git diff --check also passed on the changed files/diff.

0xKT
0xKT previously approved these changes Oct 28, 2026
@0xKT
0xKT self-requested a review October 28, 2026 07:25
The refusal check read the turn's whole text, every agent message
joined. Copilot's refusals are matched by their opening and a short
length, so a turn that narrated before the refused call never matched,
and the refusal came back as the run's result. The check now reads the
last message, the one handed to the caller, here and on the remote
pick-up path; the empty-turn check still reads the whole turn.
A message.complete reports the lane's usage slot. A cron or wake turn
never wrote it, and a turn that does not own the lane never clears it,
so a wake after a sub-agent result resent that result's usage and cost.
An ACP client then added the same cost to the session's running total
twice, and every other client was shown the stale numbers. The cron
turn now passes a usage sink like any other turn and stashes what it
spent.
The turn is booked before the empty-turn and refusal checks fail it,
and only a comment held that order. The stub's empty prompts now report
the usage they were given, so both failures run through a real backend
and the ledger is read back.
output_limit_turn_at and budget_stop_turn_at were spelled out by the
loop that writes them, the ACP server that reads them and the session
that clears them. They are now constants in raven.session.manager; the
stored names are unchanged, and the session tests still spell them out.
The ACP client's _counted and _turn_cost and the server's _usd
re-decided what a reported amount and a token count are. They now use
reported_cost and token_count. The difference was the finite check: an
inf amount passed all three and a nan passed the client's, and either
one as a session's spend baseline left every later turn of that session
unpriced.
@0xKT

0xKT commented Oct 28, 2026

Copy link
Copy Markdown
Member

Merge conflict: @LivXue, this PR no longer merges onto the current main tip 959cd0a9 (the #1092 merge). Measured with git merge-tree --write-tree 959cd0a9 bfda01b4 -> exit 1, one conflict:

  • raven/agent/loop/turn_path.py

The earlier A on this PR was graded on a merge onto 06509f8e, so after the rebase it gets re-checked on the new merged tree.

@gloryfromca

Copy link
Copy Markdown
Member

Blocking: rebase onto current main and resolve the turn_path.py conflict before merge.

I independently confirmed the new mergeability report: github/main is now 959cd0a9, GitHub reports this head as dirty/non-mergeable, and git merge-tree --write-tree github/main HEAD exits 1 with a content conflict in raven/agent/loop/turn_path.py. This supersedes my earlier clean stance because the reviewed tree is no longer the tree that can land.

Baseline verification on the existing head still passes: uv run pytest tests/test_agent_loop_usage_sink.py tests/test_rpc_spine.py -q -> 117 passed. Those tests cannot validate the eventual merged tree; the rebased revision needs review and testing after the conflict is resolved.

@LivXue

LivXue commented Oct 28, 2026

Copy link
Copy Markdown
Member Author

Rebased onto main (959cd0a) to resolve the one conflict with #1092 in raven/agent/loop/turn_path.py: the budget-stop marker keeps #1092's narrower condition (budget_stopped from the hook) and this PR's BUDGET_STOP_TURN_AT constant. No other change; the affected tests and the full suite were re-run on the rebased head.

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blockers; this can merge as far as I am concerned.

The prior merge-conflict block is resolved. I reviewed the rebase delta and conflict resolution against current main (959cd0a9), checked git range-diff across all five commits, and verified the merged tree is clean. Four commits are patch-identical; the marker refactor preserves main's narrower budget_stopped condition and changes only the metadata-key spelling to BUDGET_STOP_TURN_AT. I also rechecked the affected callers, compatibility of the stored keys, the unchanged/additive tests, and the repository rules and architecture context covered in the earlier review.

Verification: the 13-file focused suite completed with 1,644 passed and one pre-existing timing-sensitive ACP cancellation test failure; that untouched test passed immediately in isolation (1 passed). Ruff check, Ruff format check, source-language check, and git diff --check passed. GitHub reports this head mergeable.

@LivXue

LivXue commented Oct 28, 2026

Copy link
Copy Markdown
Member Author

Windows compatibility re-review — PR #1133 @ 433cba80158a424ad2286c7a92dfbc454026a7d3

Scope: the branch is rebased onto current main; the PR's own diff at this head was taken against its new merge-base 959cd0a9 (14 files, +210/−50). git range-diff 06509f8e..bfda01b4 959cd0a9..433cba80 shows the five PR commits are content-identical to the previously reviewed head bfda01b4; the only textual difference is in commit 4, where the rebased main renamed the surrounding budget_stopped code it edits. No new PR behaviour. Re-ran the affected suites on Windows.

Result: no Windows-specific defect introduced. The change set is ACP usage/refusal accounting — Copilot-refusal reading, wake-turn usage, cost/count rules centralised in providers/usage.py, and turn-marker constants. No path, subprocess, signal, socket, encoding, or permission surface.

Verification on native Windows 11, detached worktree D:\tmp\rv-1133 at the reviewed head, offline venv, scratch HOME/USERPROFILE, PYTHONUTF8=1, PYTHONPATH pinned:

  • uv run --no-sync pytest tests/test_acp_updates.py tests/test_agent_loop_usage_sink.py tests/test_remote_resume.py tests/test_rpc_spine.py tests/test_subagent_acp.py tests/test_subagent_github_copilot.py -q → 35 failed, 701 passed, 1 skipped
  • the same six files at the new base 959cd0a9 → 35 failed, 689 passed, 1 skipped
  • failing-ID sets are identical (no ID appears only at head or only at base); the 12 added tests pass. The inherited failures are the known Windows test_subagent_acp launch/timing and file-listing cases plus one Copilot install-path case, all failing at the base too.

Coverage gaps: the full suite was not rerun; these files use the ACP stub rather than live agent binaries.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants