Repository navigation
Conversation
KllSketch::update_with_weight(item, weight) is equivalent to calling update(item) weight times. A weight that fits into the free capacity is applied as plain updates; a larger one builds an exact sketch holding one copy of the item at each level whose bit is set in the weight and merges it, so the cost is logarithmic in the weight. This mirrors the weighted update in datasketches-java. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
The randomized accuracy test can fail legitimately, and the complexity documentation overstates the implementation guarantee.
Review effort: Balanced
Findings: 1
Open (3)
What changed in this PR
Adds efficient weighted updates to KLL sketches.
Changes:
- Adds
update_with_weightand internal merge refactoring. - Adds weighted-update integration tests and changelog documentation.
| File | Description |
|---|---|
datasketches/src/kll/sketch.rs |
Implements weighted updates. |
tests-integration/tests/kll_test/update.rs |
Tests weighted-update behavior. |
tests-integration/tests/kll_test/main.rs |
Registers the new tests. |
CHANGELOG.md |
Documents the API. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| (rank - true_rank).abs() <= RANK_EPS_FOR_K_200, | ||
| "item {item}: rank {rank}, true rank {true_rank}" |
| ### New features | ||
|
|
||
| * `KllSketch` is now available behind the `kll` feature, with rank, quantile, PMF, and CDF queries, merging, serialization, custom ordered item types, and a `KllFloat` adapter for non-NaN floating-point values. | ||
| * `KllSketch` is now available behind the `kll` feature, with rank, quantile, PMF, and CDF queries, merging, serialization, custom ordered item types, and a `KllFloat` adapter for non-NaN floating-point values. `KllSketch::update_with_weight` adds an item repeated a given number of times, at a cost logarithmic in the weight. |
| /// The result is equivalent to calling [`update`](Self::update) `weight` times, at a cost | ||
| /// that grows with the logarithm of `weight` rather than with `weight` itself. A weight of | ||
| /// zero is a no-op. |
Adapt apache#293 to the 0.6.0 API, retain the ordinary update path for short runs, and cover weight, overflow, merge and serialization invariants. Signed-off-by: WenyXu <wenymedia@gmail.com>
| /// ); | ||
| /// ``` | ||
| pub fn update_with_weight(&mut self, item: T, weight: u64) { | ||
| if weight == 0 { |
There was a problem hiding this comment.
This silently accepts a zero weight, while both references reject it: datasketches-java KllItemsSketch.update(item, weight) throws Weight is less than one., and the companion cpp PR #542 throws weight must be positive. Following CountMinSketch::update_with_weight is defensible, but please record the divergence here so callers porting from Java or C++ are not surprised by the silent success. I verified the Rust branches and the level folding against both references; only this case differs.
| .n | ||
| .checked_add(weight) | ||
| .expect("total stream weight overflow"); | ||
| if weight < (self.capacity - self.num_retained) as u64 { |
There was a problem hiding this comment.
weight == self.capacity - self.num_retained cannot compact either, since internal_update checks num_retained >= capacity before pushing. With <, that boundary takes the merge path and drops out of exact mode for no reason: at k=200, 150 update calls plus update_with_weight(x, 50) keep 153 retained items in estimation mode, while 50 update calls keep all 200. <= costs nothing; if you prefer parity with Java and cpp, a comment saying so would help.
| } | ||
|
|
||
| #[test] | ||
| fn weighted_update_matches_repeated_updates_in_exact_mode() { |
There was a problem hiding this comment.
This test never reaches the branch it looks like it covers: weights 1..=15 total 120 against a capacity of 200, so every call takes the O(weight) fast path and the merge path is untested here. Adding the boundary case (150 updates, then update_with_weight(x, 50)) would cover the guard in sketch.rs and pin the one weight where the choice changes the sketch's mode.


Adds
KllSketch::update_with_weight(item, weight), equivalent to callingupdate(item)weighttimes. This mirrorsupdate(item, weight)on the KLL sketches in datasketches-java. The name followsCountMinSketch::update_with_weight.Behavior
weightis smaller than the free capacity, the item is insertedweighttimes as plain updates. No compaction can happen on this path.hcounts as2^h, the exact sketch holds one copy of the item at each level whose bit is set inweight. Bits above the top level fold intoweight >> 60copies at level 60, because level capacities are only defined up toMAX_NUM_LEVELS(61). This path costs one merge with a sketch of at most 75 items, however large the weight.FrequentItemsSketch::update_with_count.u64::MAX, it panics without modifying the sketch, the same asupdate.merge's body moves into a privatemerge_unchecked, so the weighted path can skip checks it has already done without discarding aResult.mergeis otherwise unchanged.Tests
New module
tests-integration/tests/kll_test/update.rs:2^40 + 12345weightu64::MAX - 1weight, with a serialization round tripcargo x test,cargo clippy --all-features --all-targets --workspace -- -D warningsandcargo +nightly fmt --all --checkpass locally. I did not run taplo, typos or hawkeye locally, so CI will be their first run.Companion PR for datasketches-cpp: apache/datasketches-cpp#542
🤖 Generated with Claude Code