Skip to content

Quantiles items: keep the sketch consistent when an update throws - #805

Merged
proost merged 1 commit into
mainfrom
quantiles-items-update-safety
Nov 19, 2026
Merged

proost merged 1 commit into
mainfrom
quantiles-items-update-safety

Conversation

@leerho

@leerho leerho commented Nov 19, 2026

Copy link
Copy Markdown
Member

QuantilesItemsSketch.update() could leave the sketch broken if processing a full base buffer threw. This fixes that, and makes the merge code allocate before it changes the target.

The problem

update() added 1 to n_ and baseBufferCount_ before calling ItemsUtil.processFullBaseBuffer(). That call can throw:

  • OutOfMemoryError while growing the levels;
  • whatever the user's comparator throws while sorting the base buffer, for example ClassCastException, NullPointerException, or TimSort's "Comparison method violates its general contract!".

The base buffer count then stayed at 2k. If the levels hadn't grown, every later update threw ArrayIndexOutOfBoundsException. If they had, later updates silently wrote over level 0.

update() also stored a new maximum before comparing the item with the minimum, so a comparator that threw on that second comparison left a maximum that was never counted.

The fix

  • update() compares the item with the min and max first, and commits n_, the base buffer count, min and max only after processing succeeds. This follows HeapUpdateDoublesSketch.update().
  • processFullBaseBuffer() grows the levels first, then sorts a copy of the base buffer and propagates from that copy. A comparator that throws during the sort therefore leaves the base buffer as it was. The copy is 2k references, allocated once per full base buffer.
  • Merging (ItemsMergeImpl.mergeInto and downSamplingMergeInto) now allocates the target's space and its work buffers before changing the target, so an OutOfMemoryError there leaves the target as it was.
  • Docs: the QuantilesItemsSketch class javadoc says which failures leave the sketch unchanged. If the comparator throws while sorted levels are being merged, or doesn't impose a consistent total order, the sketch's state is undefined. Making that case safe too would need scratch space for the whole carry chain on every propagation.

Results and the serialized form don't change. A probe over many sketch sizes, values of k, and unions (including down-sampling ones) gave byte-identical images on main and on this branch.

Tests

The new ItemsUpdateFailureTest:

  • Makes the comparator throw while the full base buffer is sorted, both when the levels were just grown (n = 2k) and when no growth was needed (n = 6k). The failed item isn't counted. After many more updates the sketch is byte-identical to one that never saw that item, with the same bit pattern, min, max and median.
  • Makes the comparator throw on the minimum check after the maximum check passed. The max is unchanged, and the next update works.

Both tests fail on main.

The full suite passes (2,431 tests). javac -Xlint:all -Xdoclint:all,-missing reports nothing in the changed files.

C++ classic quantiles has the same flaw; DS-CPP plans a fix there separately.

🤖 Generated with Claude Code

update() counted the item in n and the base buffer count before
processing a full base buffer. If processing threw, the base buffer count
stayed at 2k: later updates then either threw
ArrayIndexOutOfBoundsException or silently wrote over level 0. update()
also stored a new maximum before the minimum comparison, which could throw.

update() now compares first and commits n, the base buffer count, min and
max only after processing succeeds, as HeapUpdateDoublesSketch does. The
full base buffer is sorted as a copy, so a comparator that throws during
the sort leaves the base buffer as it was. The merge code now allocates
all the space it needs before it changes the target.

Results and the serialized form are unchanged. The class javadoc states
which failures leave the sketch unchanged; a comparator that throws while
levels are merged, or is inconsistent, leaves it undefined.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@proost proost left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@proost
proost merged commit 598a549 into main Nov 19, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants