feat(session): harden per-session SQLite sharding#21579
feat(session): harden per-session SQLite sharding#21579sjawhar wants to merge 1 commit intoanomalyco:devfrom
Conversation
|
The following comment was made by an LLM, it may be inaccurate: The search results only return the current PR (21579) itself across all keyword combinations. The other PRs found in one search (#16730, #20344) are related to memory/database optimization and cross-session learning, but not directly addressing the same SQLite sharding and retry/backoff hardening work. No duplicate PRs found |
902b08c to
96df768
Compare
|
Windows-only CI note for reviewers:\n\n- is still hitting the same Windows-specific file-lock/concurrency class we saw earlier: fails on while unlinking a shard file, and has one worker exit non-zero under concurrent install.\n- is not the same EBUSY pattern. The current failures are app/UI-side: times out waiting for , cannot find the control, times out selecting agent sound, and the run shows repeated plus a Solid proxy error in app state.\n- Linux CI is green for , , and , and the local verification for this branch is green. From the DB sharding perspective, the remaining CI red is Windows-only.\n\nLeaving this note here so the remaining Windows failures are documented explicitly in the PR while mergeability is discussed. |
|
Windows-only CI note for reviewers (corrected):
Leaving this note here so the remaining Windows failures are documented explicitly in the PR while mergeability is discussed. |
fb4a3da to
737e1dd
Compare
78c4d67 to
f56244c
Compare
f56244c to
a2c0fc1
Compare
Issue for this PR
Closes #20935
Type of change
What does this PR do?
This adds per-session-tree SQLite sharding for message, part, todo, and event storage while keeping session and project metadata in the global database. Old sessions still fall back to the global database when no shard exists yet, so the new path can coexist with pre-sharding data.
It also hardens the global write path with
SQLITE_BUSY*retry/backoff, WAL/checkpoint monitoring, and targeted routing plus load coverage. During soak verification on currentdev, project bootstrap writes also needed the same immediate transaction wrapper so worker startup would not surface rareSQLITE_BUSY_RECOVERYfailures before shard activity began.How did you verify your code works?
cd packages/opencode && bun typecheckcd packages/opencode && bun test test/storage/retry.test.ts test/storage/pragma.test.ts test/storage/db.test.ts test/storage/wal-monitoring.test.ts test/storage/checkpoint-monitoring.test.ts test/storage/write-metrics.test.ts test/session/concurrency-load.test.tscd packages/opencode && OPENCODE_LOAD_SOAK_MS=300000 bun test test/session/concurrency-load.test.ts -t "soaks multi-process contention for the configured duration"Screenshots / recordings
N/A (storage/runtime change)
Checklist