Skip to content

perf(sessions): skip unchanged host bookkeeping writes on idle passes - #3079

Open
devin-ai-integration[bot] wants to merge 1 commit into
masterfrom
devin/1791140000-idle-history-commits-nothing
Open

devin-ai-integration[bot] wants to merge 1 commit into
masterfrom
devin/1791140000-idle-history-commits-nothing

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • An idle history pass no longer commits anything to the session stores. Before this change, each pass rewrote every provider's host-coverage://<provider>/v1 row and the Kimi and Pi discovery frontiers, even when nothing had changed. That was 9 WAL commits per idle project pass, plus about the same on user-sessions.db. The daemon runs this pass every minute (HISTORY_IDLE_RECHECK_INTERVAL).
  • The new revise_host_record (runtime/source.rs) writes a record only when its byte_offset or file_id changes. Those are the only fields readers check. mtime is still bumped on a real change, which lets a record move byte_offset backwards past the monotonic cursor guard.

Motivation

Refs #2521, Refs #2658.

I re-measured after #2777 and #2948. The capture path already reads only the objects a commit names. In load_current_mutations, SourceObjectCoverageV1::Complete is reached only for complete-snapshot-absence sources. Next I looked at an idle daemon with nothing being ingested, using an isolated profile and project. Temporary SQL probes, since removed, plus parse_offsets snapshots showed what changed each minute: host-coverage://{claude,cursor,kimi,opencode,pi}/v1 and host-frontier://{kimi,pi}/discovery/v1 were rewritten on both stores with only mtime incremented. Every such commit adds WAL frames that readers and checkpoints then reread for as long as the daemon is idle. #2658 lists this idle traffic as the thing that makes whole-daemon bytes per message unmeasurable.

Changes

  • crates/tracedecay-sessions/src/runtime/source.rs: persist_host_provider_coverage now goes through the new revise_host_record, which skips unchanged records.
  • hosts/kimi.rs, hosts/pi.rs: the discovery frontier writes use revise_host_record. Reaching the end on an already-reset frontier no longer writes.
  • ingest/project_provider.rs, ingest/user_provider.rs: removed the Codex callers' stored_coverage != Some(coverage) guard. The shared write now owns that check for every provider.
  • crates/tracedecay-session-runtime/tests/session_store_read_cost.rs: new idle_history_pass_commits_nothing. It runs the production project ingest pass twice over an empty host home on a registered store. It asserts that the first pass commits and the second appends no WAL commit frames.

Test plan

  • bash scripts/require-exact-test.sh cargo test -p tracedecay-session-runtime --features test-helpers --test session_store_read_cost -- idle_history_pass_commits_nothing --exact: 1 passed. Without the tracedecay-sessions change it fails with left: Some(9) right: Some(0).
  • cargo test -p tracedecay-sessions --lib -- runtime::hosts runtime::ingest runtime::source: 353 passed.
  • cargo clippy -p tracedecay-sessions -p tracedecay-session-runtime --all-targets --features tracedecay-session-runtime/test-helpers -- -D warnings: clean. cargo fmt --all -- --check: clean.
  • Tested manually on a real daemon built from this branch, with an isolated HOME/TRACEDECAY_DATA_DIR and an enrolled project. Before the change, coverage and frontier rows were rewritten every idle minute. After it, parse_offsets and both stores' WAL mxFrame were byte-identical across 130 s (two idle passes).

Not covered: operator-scale read volume (the 2 TB / 3 h in #2521) is not re-measured here. The per-message store-growth target in #2658 is unchanged by this PR.

Checklist

  • CHANGELOG.md: not updated; release-please generates it from commits
  • No secrets, credentials, or .env files included
  • Breaking changes documented (none)

Link to Devin session: https://app.devin.ai/sessions/868821e98d0743dc83648344702f3479
Open in Devin Desktop: https://app.devin.ai/desktop/session/868821e98d0743dc83648344702f3479?variant=devin
Requested by: @ScriptedAlchemy


Devin Review

The daemon reruns the history pass every idle minute. Each pass rewrote every provider's host-coverage row and the Kimi and Pi discovery frontiers with a bumped mtime even when nothing changed: nine commits per idle project pass, plus the same on the user store, each adding WAL frames that readers and checkpoints reread.

revise_host_record now writes such a record only when its byte_offset or file_id changes; mtime stays the revision that lets a changed record move backwards past the monotonic cursor guard. The Codex callers' own unchanged-coverage guard is gone, since the shared write owns it.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@changeset-bot

changeset-bot Bot commented Oct 4, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 56adcc9

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Devin Review: 1 flag

Not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)

Devin Review

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant