Repository navigation
feat: reuse range locality for aggregation - #4
Merged
benbellick merged 2 commits intoSep 20, 2026
Conversation
This was referenced Sep 18, 2026
benbellick
force-pushed
the
feat/issue-25344-range-transform-locality
branch
from
September 18, 2026 19:50
a5d4922 to
a60803e
Compare
benbellick
force-pushed
the
feat/issue-25345-range-date-trunc-aggregation-stacked
branch
7 times, most recently
from
September 18, 2026 20:50
dc82815 to
a0d48c1
Compare
benbellick
force-pushed
the
feat/issue-25344-range-transform-locality
branch
from
September 20, 2026 17:38
381dd98 to
0731094
Compare
benbellick
force-pushed
the
feat/issue-25345-range-date-trunc-aggregation-stacked
branch
12 times, most recently
from
September 20, 2026 19:32
01ae845 to
ab074cd
Compare
benbellick
force-pushed
the
feat/issue-25345-range-date-trunc-aggregation-stacked
branch
from
September 20, 2026 19:39
ab074cd to
0d691f7
Compare
benbellick
merged commit Sep 20, 2026
0d691f7
into
feat/issue-25344-range-transform-locality
20 of 34 checks passed
Owner
Author
|
Combined into #3 so the range-locality analysis and its aggregation consumer can be reviewed as one complete optimization. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Closes apache#25345.
Rationale for this change
Grouped aggregation only requires each equality group to remain within one input partition. The preceding range-locality analysis PR can prove that property for audited transformations of concrete range boundaries, allowing aggregation to avoid a hash shuffle without claiming general partition-layout compatibility.
For example, hourly truncation of a timestamp range split at an hour boundary keeps every
(key, hour)group local. A split inside an hour does not, so repartitioning remains required.What changes are included in this PR?
Aggregation enforcement and validation now share one locality policy across combined and uncombined partial/final plans. Useful range partitioning is preserved, repeated optimization remains stable, grouping sets remain excluded, and joins cannot consume the weaker locality guarantee.
The SQLLogicTest removes repartitioning for aligned hourly
date_truncboundaries while retaining it for overlapping day buckets anddate_bin.This PR is stacked on the preceding range-transformation locality analysis and should be reviewed after it.
Are these changes tested?
Optimizer regressions cover uncombined partial/final validation, repeated enforcement after aggregate combination, grouping-set rejection, and join isolation. The targeted SQLLogicTest and full extended workspace suite pass.
A reproducible Criterion benchmark uses eight in-memory hourly range partitions and eight rows per group. It builds the data and physical plan outside the timed section, then measures execution with and without the otherwise unnecessary hash repartition. Two synthetic workloads showed approximately 35–36% lower execution time.
Are there any user-facing changes?
Eligible grouped aggregations can avoid hash repartitioning. Other operators continue to require ordinary distribution compatibility.
Note: This PR was developed with AI assistance. All changes have been reviewed, and I take full responsibility for this contribution.