Skip to content

perf(ui): durable message appends rematerialize every loaded Turn #5508

Description

@liuxiaocs7

What happened

Appending one durable message to a long loaded conversation rematerializes every historical Turn before restoring unchanged Turn identities through deep value comparison. In a 5,000-Turn / 20,000-message fixture, one append takes 20.18 ms p50 / 28.71 ms p95 in projection alone, excluding React and layout. That synchronous work can consume more than a 60 Hz frame budget before rendering begins.

The existing identity reconciliation avoids some downstream React renders, but still pays for deriving all of the unchanged history. Expected: an ordinary append should rematerialize only affected Turns and their tool dependencies, while preserving the full-projection semantics and unchanged Turn references.

This is the durable-message refresh path (assistant step, tool result, Turn state, etc.). Ordinary live text deltas with the same messages reference already skip settled-history materialization.

How to reproduce

  1. Load a Desktop conversation with several thousand Turns into the Renderer. The relevant size is loaded history, not just history on disk.
  2. Append one durable assistant message to the last Turn while retaining the historical message objects, as the Desktop transcript RangeStore does for a tail update.
  3. Profile createTranscriptProjection().project() in packages/ui/src/transcript-projection.ts. A new array currently invokes materializeTurns for the entire history, followed by value reconciliation.
  4. Observe historical timeline rebuilding even though the published historical Turn references remain unchanged.

The measured fixture starts each Turn with four messages: user, Read call, text result and assistant. History is installed through the real RangeStore in 128 KiB batches; updates then append one assistant step at a time. No reproduction program or benchmark fixture is included in this issue or the proposed patch.

Environment

  • Maka source baseline: fcc8db89e058b4c2064a9683d6a9f18c2a14ba3b
  • Surface: Desktop Renderer / shared UI projection
  • Machine: Apple M4 Pro, macOS arm64 (Darwin 24.6.0)
  • Node.js: v24.14.0
  • Measurement date: 2026-09-19

Logs, screenshots, or additional context

Local source-bundled measurements used real, frozen RangeStore snapshots. Each size had 20 warmups and 100 measured consecutive appends. p50 averages the middle observations; p95 uses nearest rank. Fixture generation, serialization and validation are outside the projection timer.

Loaded Turns Initial messages Projection p50 (ms) Projection p95 (ms)
100 400 0.363 0.516
1,000 4,000 3.702 5.284
5,000 20,000 20.179 28.713
10,000 40,000 44.595 58.129

A separate, untimed structural probe at 5,000 Turns counted 20,001 materialized messages, 5,000 rebuilt timelines and 4,999 unchanged historical timelines rebuilt for a single append. All 4,999 unchanged Turn references were eventually restored, so an identity-only regression test misses the wasted computation.

Related work: #2030 / #2034 established projection identity preservation. #2049 identified the remaining reconciliation cost but was closed after incremental IPC and a bounded resident Renderer window. #5366 subsequently removed Renderer eviction and loads full history up to an initial 64 MiB read budget. The earlier bounded-window rationale therefore no longer covers this case. #2913 and #4677 track broader performance work.

A narrow fix can index immutable message snapshots and rematerialize affected Turns plus connected toolUseId / ShellRun dependencies in storage order. Replacements, prepends, locale changes and live-to-durable filtering must retain the complete projection fallback. A full prefix reference scan and output-array traversal may remain linear; this does not require claiming an O(1) append pipeline or changing Store/protocol behavior.

Posted by Codex at @liuxiaocs7's request. The follow-up take command is also automated on their behalf; substantive implementation assistance will be disclosed in the PR.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions