Repository navigation
KLL: allocate heap item storage on first use; add ensureCapacity() - #800
Merged
Merged
Conversation
Heap KLL sketches (doubles, floats, longs and items) no longer allocate item storage when they are created, when an empty image is heapified, or on reset(). The storage is allocated on the first update, or on the first merge of a non-empty sketch, before anything else changes. ensureCapacity() allocates it up front. It has no effect once the storage exists, nor for sketches backed by a MemorySegment. This follows the lazy allocation convention agreed on dev@. Results and the serialized form are unchanged. An OutOfMemoryError from the first allocation now comes from the first update or merge instead of the constructor; the class javadoc says so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Nov 12, 2026
proost
reviewed
Nov 15, 2026
An updatable image of an empty sketch (no items, base level only) now heapifies without item storage, like a compact empty image; it is allocated on first use. The items sketch now allocates its array only in the single-item and full branches, so an unsupported updatable image is rejected without allocating first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This applies the lazy allocation convention agreed on dev@ (vote, proposal) to the heap KLL sketches. It is the Java counterpart of apache/datasketches-cpp#558. The t-digest part is #801 and the REQ part is #802.
Behavior
KllDoublesSketch,KllFloatsSketch,KllLongsSketchandKllItemsSketchallocate their item storage on first use. That's the first update, or the first merge of a non-empty sketch.reset()releases it.KllSketch.ensureCapacity(): a new public method that allocates the initial storage of k items up front. It does nothing once the storage exists, and nothing for sketches backed by aMemorySegment, which are unchanged.KllSketchclass javadoc says that anOutOfMemoryErrorfrom the first allocation now comes from the first update or merge, not from sketch creation. ThenewHeapInstancefactories point toensureCapacity().Tests
The new
KllLazyAllocationTestcovers the doubles, floats, longs and items sketches:toString;ensureCapacity()and a sketch without it produce byte-identical compact and updatable images. These comparisons stay below k items, because KLL compaction is random.ensureCapacity(), the first k updates allocate nothing more. A second call changes nothing.reset()releases the storage, and the sketch then matches a fresh one.The full suite passes (2,395 tests).
javac -Xlint:all -Xdoclint:all,-missingreports nothing in the changed files.🤖 Generated with Claude Code