Skip to content

fix(kll): return exact quantile endpoints after compaction - #209

Merged
proost merged 1 commit into
apache:mainfrom
jaideeppyne:fix/kll-exact-quantile-endpoints
Nov 18, 2026
Merged

proost merged 1 commit into
apache:mainfrom
jaideeppyne:fix/kll-exact-quantile-endpoints

Conversation

@jaideeppyne

Copy link
Copy Markdown
Contributor

KLL stores the exact stream extrema, but quantile queries can miss them after compaction when they fall out of the retained items. Calling the partition API also used to overwrite the cached sorted view, so a later query could return a different result.

Insert a missing endpoint in the sorted view and transfer one unit of weight from its neighbor, following Java's IncludeMinMax. Quantile endpoints and partition boundaries then agree, and the partition call no longer changes later queries. This follows the interoperability direction of apache/datasketches-cpp#567.

The new regression drops one endpoint regardless of compaction parity. It checks both search criteria, the sorted-view iterator's total weight, and query results before and after partitioning. It fails on the base revision and passed 100 runs with the fix. The full go test -race ./... suite passed with the pinned C++ and Java snapshots and regenerated Go snapshots. go vet ./..., formatting, and git diff --check passed on Go 1.26.5.

Implementation and tests prepared with OpenAI Codex assistance.

@proost proost left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@proost
proost merged commit 6535f0a into apache:main Nov 18, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants