Skip to content

feat: reuse range locality for aggregation - #4

Merged
benbellick merged 2 commits into
feat/issue-25344-range-transform-localityfrom
feat/issue-25345-range-date-trunc-aggregation-stacked
Sep 20, 2026
Merged

benbellick merged 2 commits into
feat/issue-25344-range-transform-localityfrom
feat/issue-25345-range-date-trunc-aggregation-stacked

Conversation

@benbellick

@benbellick benbellick commented Sep 18, 2026 •

Copy link
Copy Markdown
Owner

Which issue does this PR close?

Closes apache#25345.

Rationale for this change

Grouped aggregation only requires each equality group to remain within one input partition. The preceding range-locality analysis PR can prove that property for audited transformations of concrete range boundaries, allowing aggregation to avoid a hash shuffle without claiming general partition-layout compatibility.

For example, hourly truncation of a timestamp range split at an hour boundary keeps every (key, hour) group local. A split inside an hour does not, so repartitioning remains required.

What changes are included in this PR?

Aggregation enforcement and validation now share one locality policy across combined and uncombined partial/final plans. Useful range partitioning is preserved, repeated optimization remains stable, grouping sets remain excluded, and joins cannot consume the weaker locality guarantee.

The SQLLogicTest removes repartitioning for aligned hourly date_trunc boundaries while retaining it for overlapping day buckets and date_bin.

This PR is stacked on the preceding range-transformation locality analysis and should be reviewed after it.

Are these changes tested?

Optimizer regressions cover uncombined partial/final validation, repeated enforcement after aggregate combination, grouping-set rejection, and join isolation. The targeted SQLLogicTest and full extended workspace suite pass.

A reproducible Criterion benchmark uses eight in-memory hourly range partitions and eight rows per group. It builds the data and physical plan outside the timed section, then measures execution with and without the otherwise unnecessary hash repartition. Two synthetic workloads showed approximately 35–36% lower execution time.

Are there any user-facing changes?

Eligible grouped aggregations can avoid hash repartitioning. Other operators continue to require ordinary distribution compatibility.


Note: This PR was developed with AI assistance. All changes have been reviewed, and I take full responsibility for this contribution.

@benbellick
benbellick force-pushed the feat/issue-25344-range-transform-locality branch from a5d4922 to a60803e Compare September 18, 2026 19:50
@benbellick
benbellick force-pushed the feat/issue-25345-range-date-trunc-aggregation-stacked branch 7 times, most recently from dc82815 to a0d48c1 Compare September 18, 2026 20:50
@benbellick
benbellick force-pushed the feat/issue-25344-range-transform-locality branch from 381dd98 to 0731094 Compare September 20, 2026 17:38
@benbellick
benbellick force-pushed the feat/issue-25345-range-date-trunc-aggregation-stacked branch 12 times, most recently from 01ae845 to ab074cd Compare September 20, 2026 19:32
@benbellick
benbellick force-pushed the feat/issue-25345-range-date-trunc-aggregation-stacked branch from ab074cd to 0d691f7 Compare September 20, 2026 19:39
@benbellick
benbellick merged commit 0d691f7 into feat/issue-25344-range-transform-locality Sep 20, 2026
20 of 34 checks passed
@benbellick

Copy link
Copy Markdown
Owner Author

Combined into #3 so the range-locality analysis and its aggregation consumer can be reviewed as one complete optimization.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant