Repository navigation
fix(kll): return exact quantile endpoints after compaction - #209
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
KLL stores the exact stream extrema, but quantile queries can miss them after compaction when they fall out of the retained items. Calling the partition API also used to overwrite the cached sorted view, so a later query could return a different result.
Insert a missing endpoint in the sorted view and transfer one unit of weight from its neighbor, following Java's
IncludeMinMax. Quantile endpoints and partition boundaries then agree, and the partition call no longer changes later queries. This follows the interoperability direction of apache/datasketches-cpp#567.The new regression drops one endpoint regardless of compaction parity. It checks both search criteria, the sorted-view iterator's total weight, and query results before and after partitioning. It fails on the base revision and passed 100 runs with the fix. The full
go test -race ./...suite passed with the pinned C++ and Java snapshots and regenerated Go snapshots.go vet ./..., formatting, andgit diff --checkpassed on Go 1.26.5.Implementation and tests prepared with OpenAI Codex assistance.