fix(rollout): make WAL time trigger a true OR with count trigger#147
Merged
Conversation
The periodic time-based WAL cleanup was gated by a second count threshold (cleanup_min_generations), so "interval elapsed" would not merge unless enough generations had also accumulated. That turned the intended "time OR count, whichever fires first" trigger into an AND: a shard whose interval elapsed but stayed below the count threshold never merged. Make the time trigger unconditional on count: cleanup_own_shard now merges whatever is pending (>=1 generation) the moment a pass runs. Remove the cleanup_min_generations option, the rollout_cleanup_min_generations server config, and the ROLLOUT_CLEANUP_MIN_GENERATIONS env/CLI flag. The count trigger (merge_after_generations) is unchanged; the two are now a strict OR. Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
cleanup_min_generations), so "interval elapsed" would not merge unless enough generations had also accumulated. That turned the intended "time OR count, whichever fires first" trigger into an AND — a shard whose 5-minute interval elapsed but stayed below the count threshold never merged back into the base table.cleanup_own_shardmerges whatever is pending (>=1 generation) the moment a pass runs.cleanup_min_generationsoption, therollout_cleanup_min_generationsserver config, and theROLLOUT_CLEANUP_MIN_GENERATIONSenv/CLI flag. The count trigger (merge_after_generations) is unchanged; the two triggers are now a strict OR.Test plan
cargo build -p lance-context-core -p lance-context-servercargo test -p lance-context-core --lib rollout(17 passed)cargo test -p lance-context-server(39 passed)cleanup_own_shard_merges_only_at_min_generations→cleanup_own_shard_merges_whatever_is_pending, asserting a single pending generation is merged on the next pass regardless of count.🤖 Generated with Claude Code