Repository navigation
Conversation
…numerators Doubling denom until it exceeds numer overflows to zero once numer >= 2^63, so the loop never terminated. ub_on_num_levels(n) calls it with the merged stream weight, which made a merge reaching n >= 2^63 hang. Compare against numer / 2 instead, which is equivalent and cannot overflow. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
update(item, weight) is equivalent to calling update(item) weight times. A weight that fits into the free space of level zero is applied as plain updates; a larger one builds an exact sketch holding one copy of the item at each level whose bit is set in the weight and merges it, so the cost is logarithmic in the weight. This mirrors the weighted update in datasketches-java. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The const_iterator started at level 0 without skipping empty levels. When level 0 is empty, which a merge can leave, every item was reported with weight 1 instead of the weight of its level. Quantile queries were not affected: the sorted view reads the levels directly. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Reject weights above 2^61 - 1 (kll_constants::MAX_WEIGHT), as datasketches-java does, instead of folding the high bits into level 60. Java cannot read an image with n >= 2^63. - Allocate the items storage before writing into it. Since items are allocated lazily, the small-weight path wrote through a null pointer on a new sketch. Allocating first also leaves the sketch unchanged when the allocation fails. - Construct each copy before counting it, so that a failed copy leaves a valid sketch. - A weight of 1 is a plain update, as in Java. - Check the item before the weight, as in Java. - Tests: the maximum weight, weights above it, weight 1, the first weighted update of a new sketch on both paths, and allocation failures. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Coverage Report for CI Build 44436712044Warning Build has drifted: This PR's base is out of sync with its target branch, so coverage data may include unrelated changes. Coverage increased (+0.1%) to 96.974%Details
Uncovered ChangesNo uncovered changes found. Coverage Regressions92 previously-covered lines in 13 files lost coverage.
Coverage Stats
💛 - Coveralls |
Both deserializers, from bytes and from a stream, must allocate only what a newly built empty sketch does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
proost
requested changes
Nov 18, 2026
| } | ||
| items_size_ = levels_[num_levels_]; | ||
| items_ = allocator_.allocate(items_size_); | ||
| for (uint32_t i = 0; i < items_size_; ++i) new (&items_[i]) T(item); |
Member
There was a problem hiding this comment.
Because of this logic, If bad_alloc happens, we should deallocate in L248. If not, this is memory leak.
Member
Author
There was a problem hiding this comment.
You're right, thanks. If a copy of the item throws, the destructor doesn't run, so the storage and the copies already made leaked. Fixed in 0ab4ab6: the constructor now destroys the copies it made and frees the storage before rethrowing.
The constructor that builds the weighted sketch now destroys the copies it made and frees its storage if a copy throws. The weighted update resets the sorted view before changing anything. merge() now forwards an rvalue source to merge_higher_levels(), so its higher levels are moved rather than copied, and the merge the weighted update makes copies no items. A new test makes each item copy of a weighted update throw in turn, and checks for leaks and a stale sorted view. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
proost
approved these changes
Nov 19, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
kll_sketch::update(item, weight), equivalent to callingupdate(item)weighttimes. It follows the weighted update of the KLL sketches in datasketches-java.This builds on #542 by Shani Elharrar, whose two commits are kept here unchanged, with fixes on top.
Behavior
weightis smaller than the free space in level zero, the item is insertedweighttimes as plain updates. No compaction can happen on this path.weight, and merges it in. This costs one merge with a sketch of at most 61 items, however large the weight.kll_constants::MAX_WEIGHT), throwsstd::invalid_argument. These are the same rules as in Java, which cannot read an image with n >= 2^63.update(item).Fixes on top of #542
update(item), so a failed allocation also leaves the sketch unchanged.Also included
kll_helper::floor_of_log2_of_fractionlooped forever oncenumer >= 2^63, because doublingdenomoverflowed to zero. Any merge that reached n >= 2^63 hung. This is the first commit from Add weighted update to kll_sketch #542. Java computes the same bound without a loop, so it is not affected.Tests
kll_sketch_test.cpp: zero weight, NaN, same as repeated updates in exact mode, one item with a large weight, the maximum weight with a serialization round trip, weights above the maximum, weight 1, the first weighted update of a new sketch on both paths, large weights into a full sketch, estimation mode within the k=200 error bound, and strings.kll_sketch_allocation_test.cpp: a failed first weighted update leaves the sketch unchanged on both paths, and failing each allocation of a stream of weighted updates in turn leaves a valid, usable sketch.All tests pass under ASan and UBSan.
🤖 Generated with Claude Code