[hsusul]

Why

A running agent made Lore burn up to a full core. Each append re-parsed the whole session file, deleted every row, and inserted them again, so every pause in the agent's writes cost work in proportion to the whole session. Now an append parses only the new lines and inserts only their rows.

What changed

• Each adapter's parser is a run from a ParseCheckpoint. A whole-file parse starts from the empty checkpoint. An append starts from the checkpoint the last ingest stored in the new ingest_state.resume_json column (migration 0014).
• ingest_file takes the append path when the earlier bytes still hash to the last full_hash and that ingest ended on a line boundary. Lines that refer back to earlier rows resolve against the archive. A tool result updates its earlier call, and parents and file events find earlier messages and calls.
• The snapshot size is now the length of the bytes read, not the earlier stat. An append between the two used to drop the checkpoint.

Scope

• Claude Code and Codex. An adapter that does not implement the checkpoint methods keeps the whole-file path.
• Rewrites, truncation, oversized files, a half-written last line, and sessions fed by more than one file keep the whole-file path. So does an append that resolves a reference an earlier line left open, or that restates Codex session_meta.
• The harness that produced the numbers is not in this PR.

Tradeoffs

• The append path still reads and hashes the whole file once. That proves the earlier bytes are unchanged, not just their first 4 KiB. It leaves a small cost that grows with session size, measured at about 1.5 ms per MB. Trusting the 4 KiB prefix hash would make an append cost flat, but a mid-file rewrite would then leave wrong rows.
• The checkpoint lives in the archive, in the same transaction as the rows, rather than in worker memory. That costs a migration, and builds without migration 0014 report this archive as too new.

Blast Radius

Every ingest path runs through the refactored parsers and row insertion. Whole-file output is unchanged. All existing lore-core tests pass, and new tests compare every append step against a fresh ingest of the same file.

Verification

• cargo test -p lore-core --test persist an_append_keeps_existing_search_rows_and_adds_only_the_new_one fails on the test commit (ids [6, 7, 8, 9] against [1, 2, 3, 4]) and passes on head.
• cargo test -p lore-core --test incremental_ingest grows Claude Code and Codex sessions line by line, in chunks, and through half-written lines. After every step it compares all session rows with a fresh ingest, ignoring ids and timestamps. The secret case asserts the redacted search text and the finding.
• Live append harness, 6.8 MB session, one turn every 700 ms for 115 s, release build, 5 interleaved runs per side: CPU per re-ingest 1.196 s → 0.0148 s (median). Whole-process CPU 52.6 s → 1.70 s, which is 46% → 1.5% of a core. At 13.7 MB: 2.38 s → 0.025 s per re-ingest, 65% → 2.5% of a core. First scan of 3000 Claude and 1000 Codex sessions: 6.7 s → 6.9 s (median of 5).