Skip to content

fix(session): a deleted chat's token_events go with it; its spend stays anonymous (F10) - #224

Merged
Broccolito merged 2 commits into
mainfrom
fix/token-events-follow-chat-deletion
Sep 11, 2026
Merged

fix(session): a deleted chat's token_events go with it; its spend stays anonymous (F10)#224
Broccolito merged 2 commits into
mainfrom
fix/token-events-follow-chat-deletion

Conversation

@Broccolito

Copy link
Copy Markdown
Collaborator

What this fixes

F10 of the 2026-09-10 provider QA run (recorded by the security tester as INFO "token_events rows survive chat deletion"; M24 of the 2026-09-09 drive). Deleting a chat removed its sessions, messages and message_blobs rows and left every token_events row it had written: model_id, provider, session_id and a timestamp per turn, i.e. a record of which model answered when, for a chat the user chose to delete. QA-e measured 18 such rows after deleting ten probe chats.

It is worse than hygiene, because session ids are reissued. create_session mints <day>_<MAX(N)+1>, so if you delete the newest chat of the day, the next chat created that day gets the same id and inherits the whole ledger. Its cost popover (GET /sessions/{id}/usage) then lists turns it never ran, on a provider it may never have used. A JOIN sessions at read time can't catch that, because the id exists again. Only deleting the rows together with the chat can. a_reissued_session_id_does_not_inherit_the_deleted_chats_usage reproduces it; before the fix, the new chat reported the deleted chat's versa_azure / gpt-5.5 turn.

What changed (all in crates/biorouter/src/session/session_manager.rs)

  1. The delete path removes the chat's token_events in the same transaction as the chat. An unknown id still rolls back and writes nothing.
  2. Existing orphans are pruned by a startup sweep in reconcile_usage_schema, not a numbered migration arm. A migration arm would consume a migration number, and this file already documents both failure modes: development branches have collided on numbers (v11–v14, 17), and work added to a number a tester's database has already passed never runs there (the O10 hazard, arms 18–20). The sweep is idempotent (a second open finds nothing), runs under the reconcile's BEGIN IMMEDIATE, and also repairs rows left by an older build sharing the same sessions.db (daemon, CLI, scheduled jobs).
  3. The usage aggregations join sessions, so a per-turn row whose chat is gone is never read by any total.
  4. Product reason → an anonymous aggregate, not the rows. The brief's point (4) applies, and the evidence is in the tree. Issue UCSF (and maybe external) token usage mismatch #1 (the origin of the usage feature) asked for totals a user can hold "vs the UCSF subsidized limit" on the Versa dashboard. 77eae66c ("make accounting durable") pinned that deletes do not lower usage: pr13_v12_database_reconciles_usage_and_adds_loop_schema asserts all_time.turns is unchanged after delete_session. A delete does not refund tokens at the provider. So a deleted chat's billable turns are first folded into a new table, deleted_chat_usage: one row per (local day, model, provider) holding sums plus per-bucket known-turn counts, with no session id, event key or turn time. Only then are the rows deleted.
    • Usage panel, biorouter usage, and the global per-model rollup read BILLABLE_USAGE = live chats' turns (joined) + the anonymous aggregate. Completeness semantics are unchanged: SUM(turns) / SUM(*_known) replace COUNT(*) / COUNT(col). The pr13 test passes unmodified.
    • Home heatmap and insight tiles count only chats that still exist, matching the heatmap's session and message series. This is the same split that already counts subagent spend in Usage but not on Home: Usage is provider spend, Home is activity.
    • The sweep folds before deleting too, so an upgrade changes no Usage number. Legacy rows that no total ever counted (unclassified, hidden, terminal) are dropped, not kept.
    • deleted_chat_usage is additive and created idempotently (house pattern, no migration number). clear_all_sessions wipes it (a reset is a reset).
  5. docs/cli/command-reference.md (session remove) documents the behaviour. There is no living usage or heatmap doc under docs/; the only hits are history and design records.

Residual, documented at the code: an orphan whose id has already been reissued (before this fix) reads as the new chat's, and existence can't tell them apart. The only discriminator would be a timestamp guess (a row older than its chat's created_at) running destructively on every open. I measured that population at 0 on a real 11,780-chat store (3,626 ledger rows, min(ts − created_at) = 0). The delete path closes it for every chat deleted from now on.

Tests: fail-before, then pass

The first commit adds only the tests; against the original production code they fail:

$ cargo test -p biorouter --lib -- <the 7 tests>    # tests-only commit d9f99637 on top of 7c96d796
test …deleting_a_chat_keeps_its_spend_anonymously_and_deletes_its_rows ... FAILED
    a deleted chat's usage rows must be deleted with it   left: 2   right: 0
test …a_reissued_session_id_does_not_inherit_the_deleted_chats_usage ... FAILED
    a new chat inherited the deleted chat's turns: [ModelUsageRow { model_id: Some("gpt-5.5"),
    provider: Some("versa_azure"), input_tokens: 900, …, total_tokens: Some(900), … }]
test …usage_aggregations_never_count_a_token_event_whose_chat_is_gone ... FAILED
    insights: all time   left: Some(7050)   right: Some(50)
test …reopening_the_store_retires_token_events_whose_chat_is_gone ... FAILED
    open 1   left: 2   right: 0
test …clear_all_sessions_removes_history_usage_and_side_tables ... FAILED   (extended)
    no such table: deleted_chat_usage
test …fresh_database_contains_full_v16_schema ... FAILED                     (extended)
    missing fresh-schema table deleted_chat_usage
test …pr13_v12_database_reconciles_usage_and_adds_loop_schema ... ok         (unmodified; must stay green)
test result: FAILED. 1 passed; 6 failed

After the fix:

# every run: BIOROUTER_DISABLE_KEYRING=true and a sandboxed BIOROUTER_PATH_ROOT
cargo test -p biorouter --lib session::                                     267 passed; 0 failed
cargo test -p biorouter-server --lib -- routes::session routes::usage        81 passed; 0 failed
cargo fmt --all -- --check                                                   clean
cargo clippy -p biorouter -p biorouter-server --all-targets -- -D warnings   clean
clippy::too_many_lines vs clippy-baselines/ (biorouter)                      no new entries

Runtime check (sandboxed daemon, real versa_azure turn)

BIOROUTER_PATH_ROOT sandbox cloned from ~/biorouter-runs/seed-config, running a biorouterd built from this branch. Steps: one orphan pre-seeded as a pre-fix delete leaves it; create a chat; one turn; DELETE /sessions/{id} over HTTP. The daemon and sandbox were removed afterwards.

seed: schema v20, 11,780 sessions, 3,626 token_events
+ one orphan seeded as a pre-fix delete leaves it: '20260910_99', versa_azure, 1,010 tokens

daemon start, first DB access (the startup sweep):
  sqlite> SELECT COUNT(*) FROM token_events WHERE session_id = '20260910_99';   -> 0
  sqlite> SELECT * FROM deleted_chat_usage;
  2026-09-10|gpt-5.5-2026-04-24|versa_azure|1|1000|1|10|1|1010|1|0|1|0|1
  log: "Retired 1 token_events rows whose chat no longer exists"

POST /agent/start -> 20260911_1
POST /reply "Reply with exactly one word: pong" -> Finish, "pong"
  sqlite> SELECT provider, model_id, billed_total_tokens, session_type FROM token_events WHERE session_id = '20260911_1';
  versa_azure|gpt-5.5-2026-04-24|26438|user

                                    before the chat    after the turn                 after DELETE (HTTP 200)
GET /sessions/activity  (today)     no entry           1 session, 26,438 tok, 2 msg   no entry
GET /sessions/insights  (7 days)    21,963,378         21,989,816                     21,963,378
GET /usage/summary      (MTD)       21,964,388 / 61    21,990,826 / 62                21,990,826 / 62 turns

sqlite> select count(*) from token_events where session_id='20260911_1';
0
sqlite> SELECT * FROM deleted_chat_usage;
2026-09-10|gpt-5.5-2026-04-24|versa_azure|1|1000|1|10|1|1010|1|0|1|0|1
2026-09-11|gpt-5.5-2026-04-24|versa_azure|1|18625|1|5|1|26438|1|7808|1|0|1
(sessions and messages for 20260911_1: 0 and 0)

Also measured, for the follow-up: messages_fts still holds 2 rows for 20260911_1.

Out of scope, filed separately

The same id reuse makes delete_session's other survivors inheritable: cross_affiliation_grants (a privacy-gate approval, security-relevant), messages_fts (the deleted chat's text stays on disk), and checkpoints. classification_audit survives by design (privacy-tiers.md §12.5), but its backfill guard joins on the bare id. I filed these as their own task rather than widening this LOW fix.

🤖 Generated with Claude Code

F10 of the 2026-09-10 provider QA run (M24 of the 2026-09-09 drive):
deleting a chat left every token_events row it had written - model,
provider and a timestamp per turn - and every usage total kept counting
them. Session ids are reissued (`<day>_<MAX(N)+1>`), so the next chat
created that day also inherited the deleted chat's ledger in its cost
popover.

Against the current production code these fail:
- deleting_a_chat_keeps_its_spend_anonymously_and_deletes_its_rows
- a_reissued_session_id_does_not_inherit_the_deleted_chats_usage
- usage_aggregations_never_count_a_token_event_whose_chat_is_gone
- reopening_the_store_retires_token_events_whose_chat_is_gone
- clear_all_sessions_removes_history_usage_and_side_tables (extended)
- fresh_database_contains_full_v16_schema (extended)

pr13_v12_database_reconciles_usage_and_adds_loop_schema, which pins
that a delete does not lower the usage totals, passes and must keep
passing.
…ys anonymous (F10)

Deleting a chat left every token_events row it had written - which model
and provider answered each turn, and when - and because session ids are
reissued (`<day>_<MAX(N)+1>`), the next chat created that day inherited
that ledger. A JOIN at read time cannot tell those rows apart, so:

- delete_session folds the chat's billable turns into deleted_chat_usage
  and deletes its token_events in the same transaction as the chat.
- deleted_chat_usage keeps sums and known-turn counts per local day,
  model and provider, with no session id, event key or turn time. There
  is a product reason to keep the spend: the Usage panel exists to be
  held against the provider's own meter and the UCSF allowance (issue
  #1), and 77eae66 pinned that a delete does not lower it - its test
  passes unmodified.
- Existing orphans are retired the same way (fold, then delete) by a
  startup sweep in reconcile_usage_schema, not a numbered migration arm:
  it is idempotent, merge-order safe, and repairs rows an older build
  sharing the database leaves behind. An upgrade changes no Usage number.
- Every aggregation joins sessions, so a per-turn row whose chat is gone
  is never read. The Usage report, summary and per-model rollup read
  BILLABLE_USAGE (live turns + the anonymous total); Home's heatmap and
  insight tiles count only chats that still exist, like their session
  and message series.
- clear_all_sessions wipes deleted_chat_usage too.

Documented in docs/cli/command-reference.md (session remove).
@Broccolito
Broccolito merged commit 95df9a5 into main Sep 11, 2026
16 checks passed
@Broccolito
Broccolito deleted the fix/token-events-follow-chat-deletion branch September 11, 2026 20:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant