fix(graph-db): layer a linked worktree over its sibling's graph - #3071
Open
devin-ai-integration[bot] wants to merge 3 commits into
Open
devin-ai-integration[bot] wants to merge 3 commits into
devin-ai-integration[bot] wants to merge 3 commits into
Conversation
A linked worktree's first generation at the same tree resealed every row of the graph its sibling scope already sealed. It now layers over the sibling's installed flat sealed generation of the same projector: the base container is hard-linked and its relations are re-attested under the worktree projection, so the delta is just the generation marker and the row sum and digest match a cold build. Fixes #2402 Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Contributor
Author
|
I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".
|
|
Re-attestation buffered every relation's identity and lanes before rewriting the row index. Each relation's lanes are now written into a copy of the index as they are recovered, so memory no longer grows with the sibling's relation count. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Re-attestation read the sibling's original row index and entity row offsets, which vanish if the sibling retires after the layer pins them. It now reads the hard-linked copies in the layer directory. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Motivation
Fixes #2402. A primary sealed graph of about 51 MB grew to about 76 MB after adding an identical linked worktree, because the projection-exact
GraphLayeredRowSpill::createrejected any base from another namespace.Changes
sealed_store.rs/registry/publication.rs:sibling_sealed_baseslists installed flat (non-layered) sealed generations of the same projector under other namespaces of the same projectGraphDb. This uses the sealed registry, with no filesystem scan and no new shared map.sibling_layered_row_spillopens a delta over one of them.sealed_layer.rs:GraphSiblingSealedBaseV1,GraphLayeredRowSpill::create_over_sibling, andGraphSealedBaseV1::reattested, which streams the base relations under the child projection into the layer's own row index (the shared base index is never rewritten).finishre-attests before it checks the projection.generation.rs:GraphGenerationManifestIdentity::stored_underrecords the physical namespace where the base rows actually live.SealedBaseReceiptV1::identityrestores it on reopen.generation/recovered.rs/row_index.rs:recovered_relation_lanesandRowIndex::write_with_relation_lanes.code_graph_manifest.rs/code_graph.rs:LayeredBaseV1::{Parent, Sibling}. Generations with no parent try sibling bases. Correctness comes from the normal layered delta against the base (any rows that differ go into the delta), not from assuming the trees are identical. An identical tree produces a one-row delta.Not covered by the test: the test does not assert that the primary's old directory is physically removed. That generation stays retained here because the primary's next generation is itself layered over it, so it is a replay dependency. The base bytes are hard links, so removing the primary directory would not unlink the layer's copy.
Test plan
layered_refresh_tests::a_linked_worktree_at_the_same_tree_layers_over_its_siblings_sealed_graph. It seals a primary, adds an identicalgit worktreeand seals it, then checks:base.grafeois the same inode as the primarygeneration.grafeo;unset RUST_LOG && bash scripts/require-exact-test.sh cargo test -p tracedecay-store-runtime --lib session_registry::code_graph::layered_refresh_tests::a_linked_worktree_at_the_same_tree_layers_over_its_siblings_sealed_graph -- --exact: 1 passed. With the production changes stashed, it fails with "the linked worktree layers over its sibling".cargo test -p tracedecay-store-runtime --lib session_registry::code_graph: 50 passed, 2 ignoredcargo test -p tracedecay-graph-db: 150 passed, 1 ignoredcargo fmt --all -- --check,cargo clippy -p tracedecay-graph-db -p tracedecay-store-runtime --all-targets -- -D warnings: cleanpython3 scripts/linux-test-partitions.py checkandnode scripts/lint-commit-range.mjs --repository . origin/master HEAD: passcargo nextest run --workspaceleft to CIChecklist
CHANGELOG.mdupdated (not updated).envfiles includedLink to Devin session: https://app.devin.ai/sessions/a95fd88f68bd47058c44696790e4d379
Open in Devin Desktop: https://app.devin.ai/desktop/session/a95fd88f68bd47058c44696790e4d379?variant=devin
Requested by: @ScriptedAlchemy