fix(session-flow): audit-sessions token metrics lost on current transcripts (usage_split canary)

Problem

/session-flow:audit-sessions reports tokens.main and tokens.sub as unavailable (canary lost) across the whole store, so the token lens never fires. The drift section names the lost canary as invariant:usage_split.

Evidence

One sweep on Claude Code 2.1.296: 683 sessions, versions 2.1.241 through 2.1.296.

Metrics table: tokens.main and tokens.sub were unavailable (canary lost), n=683.
Drift: canary-lost: invariant:usage_split, plus 133 vanished key paths and 11 unknown record types (mode, atis-latch, ai-title, file-history-delta, cost-state, frame-link, relocated, and others).
Unchecked: "tokens.main: unavailable: a drift canary feeding it is lost".

A related question surfaced in the same sweep: hooks.stop_ms (median 4,472 ms, 600 of 683 sessions over threshold) may include hooks that run with asyncRewake: true, such as the testing plugin's test-judge collector. Those hooks do not block turn end. If the metric sums or includes them, it overstates the blocking cost.

Proposed approach

Find which transcript change broke the usage_split invariant, and update the shared reader and its canary.
Check whether hooks.stop_ms counts async hooks, and exclude them or report them separately.

Acceptance criteria

tokens.main and tokens.sub produce values on transcripts from 2.1.296.
The usage_split canary holds on current transcripts, or is replaced, with a fixture from the current format (no raw transcript content, per the eval-case transcript rule).
hooks.stop_ms either excludes async hooks or documents that it includes them.

Transcript source

A run does not need the local transcript store: it can produce current-format transcripts itself with the installed CLI (a short claude -p session that spawns one subagent and runs one async Stop hook) and probe their structure, printing key names and numbers only.

Proposed work class: work-class: scoped
Basis: automated intake triage; the lanes App applied this class.

Size: Medium
Complexity: Medium
Uncertainty: Knowable
Flags: none
Blocked by: none

Understood: /session-flow:audit-sessions loses tokens.main and tokens.sub because the usage_split canary trips on current (2.1.296) transcripts. Separately, hooks.stop_ms may include asyncRewake hooks that don't block turn end.

The approach and acceptance criteria are clear. Reading the shared reader and a current-format transcript would show which change broke the invariant. The probe needs a live claude -p run, which the issue lane can't do, so part of the verification will happen outside this lane.

Likely related: #4373 (listed in the issue). I found no duplicates among the open issues.

_{Automated triage, applied by the lanes App.}

Overnight worker: blocked, no PR opened. The token fix needs a look at current-format transcripts. This run was not allowed to read the local transcript store (~/.claude/projects): the auto-mode classifier denied it, so I stopped there.

State: no commits. The local branch fix/6785-usage-split-canary was never pushed, because it carries nothing yet.

What I found without transcripts. These are inferences from reading the minified writer code in the installed Claude Code 2.1.296 bundle. They are not checked against a real transcript.

When a message_delta stream event arrives, the writer merges its usage into the message's running usage. The input_tokens, cache_creation_input_tokens and cache_read_input_tokens values are replaced whenever the delta carries a positive value. The merged usage is then assigned to every record already yielded for that message.
So records that share one message.id can now legitimately differ on input and cache usage. That is exactly what invariant:usage_split counts (collect.py _usage_invariant). The invariant assumes streaming rewrites only output_tokens, and that assumption looks stale.
It is still open whether last-record-wins dedup (transcript_reader.UsageLedger) keeps the merged values. That depends on which records are serialized after the merge.
hooks.stop_ms: stop_hook_summary is built with one hookInfos entry per Stop hook that reports progress, and it now also carries hookCount and totalDurationMs. I did not confirm whether an asyncRewake hook gets an entry or a durationMs.

Next step (any agent with transcript read access):

Run a structural probe over a 2.1.29x corpus. For each message.id group, record whether input and cache usage differ, and which record position carries the merged (final) values. Print numbers and key names only.
If the last record always carries the merged usage, keep last-record-wins. Replace the canary with an invariant that checks this: no earlier record may exceed the last on an input or cache field. Otherwise switch the ledger to the merged value. Then update canaries.json (validated_against, basis).
Add a hand-written fixture: one message, two records, with the second record's usage merged from the delta. No excerpted transcript content, per the eval-case transcript rule.
For hooks.stop_ms, check one session that has an asyncRewake Stop hook to see whether its hookInfos entry carries the async run's duration. Then exclude that entry or document it.

Generated by Claude Code

Handoff note for whoever picks this up next. The overnight Claude run has stopped, and Kyle is moving issue work to Cursor.

Branch: none
PR: none
State: BLOCKED: auto mode denied reading ~/.claude/projects; token fix needs a transcript look (2.1.296 writer merges delta usage into earlier records, so usage_split assumption looks stale). Issue comment posted with probe plan
Notes: Needs Kyle to allow transcript reads or an attended session
Next step: Clear the blocker named above, then continue on the branch if there is one.

Where a branch is listed it is pushed, so it can be taken over as is. A draft PR shows a red ci-status by design until it is marked ready.

Generated by Claude Code

This was generated by AI during triage.

Held for a person: The token fix needs current-format transcripts from the local store, and auto mode denied that read (2026-10-11). It needs the owner to allow transcript reads or to run it in an attended session.

Released by the owner's HITL session (2026-10-24, C12): it waited only on a removed gate. Intake triage re-runs on it to set its fields, then admission takes it.

Needs a run of claude (eval, probe or /usage), which the issue lane cannot do (pr-pipeline.yaml:289-314). The owner's HITL session runs it on the operator machine; no owner action needed.

The triage sweep dispatched intake triage for this issue (attempt 1 of 2).

The lanes App's pull request #10041 merged with Refs:, so this issue stays open and nothing starts the lane on it again. work-unblock removed status: ready and agent-ready, so the hourly triage sweep sends it back to intake triage, which decides what work is left and admits it again.

The work-resolve-issue lane stopped partway through this issue and needs a person.

Why, in the run's own words:

The code side of ＃6785 is already on main, and what is left can only be checked by running against real transcripts, which this lane cannot do. Main (ce642ce) already carries all three code parts: invariant:usage_regress replaces invariant:usage_split (collect.py _usage_invariant / usage_regressions, canaries.json), there is a hand-written fixture (tests/fixtures/drift/usage/proj-u/usage-0001.jsonl) with tests in test_drift.py, the hooks.stop_ms basis in sweep-rules.json documents asyncRewake, and the changelog fragment .changes/session-flow/fix-6785-usage-regress-canary-4b7e2c19.md is in place. Still open is acceptance criterion 1: tokens.main and tokens.sub must produce values on 2.1.296 transcripts. The canaries.json basis itself says usage_regress 'has not been run over a corpus yet', and validated_against still reads 2.1.287. Checking either needs a claude run or reads of the local transcript store. This lane can do neither, and the 2026-10-24 issue comment assigns that run to the owner's HITL session. To close the issue, run /session-flow:audit-sessions over a 2.1.29x corpus, confirm that tokens.main and tokens.sub have values and that usage_regress is not lost, then update validated_against and verified in canaries.json.

What the run says it did not do:

Acceptance criterion 1 (tokens.main and tokens.sub produce values on 2.1.296 transcripts) was not verified. It needs a claude run or reads of the local transcript store, and this lane has neither. The issue comment of 2026-10-24 assigns it to the owner's HITL session.
The repo's pytest suite for audit-sessions was not run, because this lane runs no interpreter. It was not needed for a run with no changes.

Run: https://github.com/melodic-software/claude-code-plugins/actions/runs/40218329611

To retry, remove needs-human and add agent-ready again.

Navigation Menu

Repository navigation

fix(session-flow): audit-sessions token metrics lost on current transcripts (usage_split canary) #6785

Problem

Evidence

Proposed approach

Acceptance criteria

Related

Transcript source

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

fix(session-flow): audit-sessions token metrics lost on current transcripts (usage_split canary) #6785

Description

Problem

Evidence

Proposed approach

Acceptance criteria

Related

Transcript source

Activity

github-actions commented on Oct 10, 2026

kyle-sexton commented on Oct 11, 2026

kyle-sexton commented on Oct 11, 2026

kyle-sexton commented on Oct 19, 2026

kyle-sexton commented on Oct 24, 2026

kyle-sexton commented on Oct 24, 2026

github-actions commented on Oct 25, 2026

melodic-automation-lanes commented on Oct 25, 2026

melodic-automation-lanes commented on Oct 25, 2026

Metadata

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Issue actions