Skip to content

perf: tdigest grows its buffers as they fill - #561

Merged
leerho merged 3 commits into
apache:masterfrom
rustyconover:perf/tdigest-lazy-buffers
Nov 15, 2026
Merged

leerho merged 3 commits into
apache:masterfrom
rustyconover:perf/tdigest-lazy-buffers

Conversation

@rustyconover

Copy link
Copy Markdown
Contributor

tdigest's constructor reserves room for 2k + fudge centroids and BUFFER_MULTIPLIER times as many buffered values. For k = 100 with double that's about 10 KB per digest before anything is added, and it happens again on every deserialize().

This is the same kind of change as #558 for kll_sketch. Instead of reserving everything up front:

  • update() grows the buffer geometrically, starting at 8 values and doubling up to its existing limit.
  • compress() and merge() grow the centroids before they need them, also up to the existing limit.

Results and the serialized format are unchanged.

Test

Added tdigest custom allocator: buffers grow as they fill to tdigest_custom_allocator_test.cpp, using test_allocator:

  • An empty digest allocates nothing. Before this change it fails at REQUIRE( test_allocator_total_bytes == 0 ).
  • Three values take at most 8 values of buffer.
  • A digest of 100,000 updates gives the same quantiles as one built with std::allocator without the change.

All 17 test suites pass under CTest (GCC 14, aarch64).

We found this in the DuckDB DataSketches extension. For GROUP BY queries with many groups, DuckDB creates a state for each input row and merges them later, so the up-front reservation multiplied by millions of states. DuckDB made the same change to its own t-digest.

🤖 Generated with Claude Code

The constructor reserved room for 2k + fudge centroids and four times as many
buffered values (about 10 KB for k = 100 with doubles), also when
deserializing. Code that holds many small t-digests paid for all of it.
Grow the buffer geometrically up to its limit on update, and the centroids
before a compress or merge needs them. Results are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rustyconover
rustyconover force-pushed the perf/tdigest-lazy-buffers branch from 3dee23d to 15024ac Compare October 29, 2026 16:05
@leerho

leerho commented Oct 30, 2026

Copy link
Copy Markdown
Member

Thanks, Rusty. Same situation as #558: we'd like one allocation convention across the sketches and languages before merging either. The dev@ proposal, a lazy consensus ending Nov 7, 01:00 UTC, is here: https://lists.apache.org/thread/rv730qo61qqkd99dcy4wnglf3hz1s0fz

I'm marking this PR as a draft until that resolves. If it passes, this PR fits it well. The geometric growth up to the existing limits is the behavior the proposal describes, and we'd only add a reserve().

@rustyconover

Copy link
Copy Markdown
Contributor Author

@leerho thanks for outlining the remaining work. The proposal's deadline has now passed, so I've followed up on #558 to confirm the outcome, when both PRs can leave draft, and who will handle reserve(), allocation-failure tests and documentation. We can coordinate ownership there for both PRs.

All 21 checks on this PR's current commit (15024ac6) are passing, including the compiler matrix, sanitizers and libc++ hardening.

@leerho
leerho marked this pull request as ready for review November 11, 2026 20:39
leerho
leerho previously requested changes Nov 11, 2026

@leerho leerho left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The lazy-allocation proposal passed (https://lists.apache.org/thread/d9pzln7m5q1of82ghhotrr3h0j6p514m), so I'm taking this out of draft. As with #558 (see my comment there), I'll add reserve(), the allocation-failure tests and the documentation on top of this PR, in C++ and Java. Marking changes requested so it isn't merged before those are in.

@rustyconover

Copy link
Copy Markdown
Contributor Author

Thanks @leerho you're awesome!

leerho and others added 2 commits November 12, 2026 11:28
…g storage

reserve() allocates the full storage for centroids and buffered values
right after construction, for callers who want allocation failures there
or don't want updates to pay for growing the storage. merge() now
allocates before it widens min and max, so a failed merge leaves the
t-Digest unchanged.

The new tests use an allocator that fails on purpose. A failure at any
allocation of reserve(), a stream of updates or a merge leaves the
t-Digest exactly as it was, and it stays usable.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@leerho

leerho commented Nov 12, 2026

Copy link
Copy Markdown
Member

I've pushed the reserve(), allocation-failure tests and documentation on top of Rusty's commit, and merged master into the branch. merge() now allocates before it widens min and max, so a failed merge leaves the t-Digest unchanged. The new tests fail each allocation in turn and check that the t-Digest is exactly as it was, byte for byte. This doesn't depend on the KLL PRs. Once the new commits are reviewed, I'll lift my change request.

@leerho

leerho commented Nov 15, 2026

Copy link
Copy Markdown
Member

@proost, @tisonkun,
Since I contributed some of the fixes here, I can't approve this PR.
If one of you could do that it would be great!

@tisonkun tisonkun left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The allocation changes look sound. Compression still follows the existing logical buffer limit, and the additional reservations happen before the digest contents are modified. I found no blocking correctness issues.

Two non-blocking suggestions:

  1. In the reserve() documentation (tdigest.hpp, lines 131–135), clarify that it only preallocates the persistent centroid and value buffers. compress() and merge() still allocate temporary storage, so an update() that triggers compression can still throw std::bad_alloc after reserve() succeeds. The wording "if allocation failures should happen here" could otherwise suggest that all allocation failures are moved to this call.

  2. In tdigest_custom_allocator_test.cpp, lines 57–65, both grown and reference use the modified implementation; only their allocators differ. This checks allocator independence, rather than equivalence with the previous implementation. Please consider adjusting the comment and the corresponding claim in the PR description. The new allocation test already compares serialized contents with and without reserve(), so that coverage does not need to be duplicated.

Validation: the complete tdigest_test suite passed with AppleClang 21, C++11, ASan and UBSan. I also compared the base and head across 36 combinations of value type, k and input distribution, including compression, copying, merging, self-merging and updates after both stream and byte-buffer deserialization; the final serialized-output hashes matched in every case.

@leerho
leerho merged commit 680207e into apache:master Nov 15, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants