Skip to content

perf(runtime-host): baseline materialized Session copy cost - #5526

Merged
M4n5ter merged 3 commits into
mainfrom
feat/issue-5439-copy-context
Sep 21, 2026
Merged

M4n5ter merged 3 commits into
mainfrom
feat/issue-5439-copy-context

Conversation

@likun666661

Copy link
Copy Markdown
Member

Refs #5439

Summary

  • add npm --workspace @maka/runtime-host run benchmark:session-copy
  • benchmark the production session.branch.create handler over 4/16/64/128 settled turns
  • report copy latency, storage bytes added, source/child message counts, invocation counts, and message amplification
  • document the current Session copy sharing boundary and follow-up measurement gaps

This is the baseline/control-group slice for #5439. It does not introduce durable shared-prefix storage or parent-prefix dependencies.

Verification

  • npx biome check packages/runtime-host/scripts/session-copy-cost-benchmark.mjs docs/architecture/session-copy-cost-baseline.md packages/runtime-host/package.json
  • node --check packages/runtime-host/scripts/session-copy-cost-benchmark.mjs
  • git diff --check
  • @maka/core and @maka/storage builds pass in the isolated worktree
  • full @maka/runtime build is currently blocked by the local dependency state (@slack/socket-mode types unavailable and HttpsProxyAgent generic mismatch), so the benchmark could not be executed against a freshly built runtime dist

@github-actions github-actions Bot added the effort/M Under 500 readable lines label Sep 20, 2026

@Astro-Han Astro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ran the committed script against a representative built dist — it deterministically fails on the first fixture. Two P-level findings inline.

What verified clean: the measurement path is real — session.branch.create drives the production HostSessionRevisionCoordinator.#copy, turns are genuinely settled (4 runs / 12 messages materialized), the input contract matches session-revision.ts, the doc's copy-boundary table matches current source (blobId sharing, todo initializeCopy, linked-children modes, turn_state exclusion), the benchmark:session-copy script entry is inert to CI globs, and ASF headers pass.

On numbers provenance: good news — the doc contains no fabricated figures; it's a methodology document. But combined with the P1, the PR as committed delivers instructions for a baseline that has never been produced. Once the script actually runs, the doc should carry at least one recorded row (commit + platform) or the "baseline" framing should soften.

P3s:

  • directoryBytes (lines ~158-167) is TOCTOU on SQLite transient files — -wal/-shm can vanish between readdir and stat → ENOENT crash. Catch and skip ENOENT.
  • The script leans on TS-private internals (deps.store.readHeaderRecordSnapshot, listInvocations) and a recoverInterruptedSessionsStrict monkey-patch — works on compiled JS, but a rename breaks it silently-ish; a public accessor or composition-returned handle is sturdier.
  • A turn.start result of 'blocked' burns the full 10s deadline and throws a misleading timeout — check the result kind.
  • The doc asks readers to record commit/Node/platform, but the script prints only console.table — emit process.version/process.platform/git SHA so output is self-describing.

if (!sourceSummary || !sourceTurnId) throw new Error('Benchmark source did not settle');
const targetSessionId = randomUUID();
const startedAt = performance.now();
const copied = await composition.handlers['session.branch.create'](

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 — dead on arrival: the one-shot copy races two settlement windows. I ran the unmodified script twice; both runs died at historySize=4:

  • session_busyisSessionActive clears only when the run's done settles, strictly after the projected turn_state:'completed' that waitForCompletedTurn waits for;
  • source_revision_conflict (expected 20, actual 21) — expectedSourceRevision is read once at line 100, but the source header keeps revising after the final turn (post-turn bookkeeping).

Both outcomes are retryable-by-design, so the fix is small: retry on session_busy (idle wait) and re-read readHeaderRecordSnapshot inside the retry on source_revision_conflict. I verified this in a tmp copy: with attempts=2 the copy commits cleanly (~11ms, 12 messages, amplification 1.00). Without it the PR's sole deliverable produces zero rows.

},
operationContext,
);
const copyMs = performance.now() - startedAt;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 — single-sample latency. copyMs is one performance.now() delta per fixture, run sequentially in ascending order in one process — heap/GC carryover plus single-run noise. For data meant to drive the #5439 design decision (does copy cost grow with retained history), repeat N times and report median (and ideally a warmup pass), or state the limitation in the doc.

@M4n5ter
M4n5ter merged commit 879e0a4 into main Sep 21, 2026
13 checks passed
@M4n5ter
M4n5ter deleted the feat/issue-5439-copy-context branch September 21, 2026 13:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/M Under 500 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants