Repository navigation
refactor: Implement optimizer hook for combining partial/final aggregate in AggregateExec - #25962
Conversation
…ate in AggregateExec
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #25962 +/- ##
==========================================
- Coverage 82.59% 82.59% -0.01%
==========================================
Files 1145 1145
Lines 443934 444086 +152
Branches 443934 444086 +152
==========================================
+ Hits 366669 366790 +121
- Misses 55009 55025 +16
- Partials 22256 22271 +15 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
kosiew
left a comment
There was a problem hiding this comment.
Thanks for this refactor. Moving the combination logic into AggregateExec::try_combine_partial_final is appropriate, and the existing transformation behavior appears preserved.
The new boundary is cleaner: the optimizer handles traversal, while AggregateExec owns compatibility checks, reconstruction, and internal state. This removes cross-crate coupling to AggregateKind and allows its implementation details to remain private.
I'm approving because the mode mappings, expression checks, error handling, and distinct-limit preservation remain equivalent. The existing regression test also covers the distinct-limit behavior. I have one non-blocking documentation suggestion to clarify the helper's optimizer-phase prerequisite.
| Some(Transformed::yes(self)) | ||
| } | ||
|
|
||
| /// Combine adjacent partial and final aggregates in the given plan if they are |
There was a problem hiding this comment.
Could we mention here that this helper should run after distribution enforcement, normally EnsureRequirements? It checks expression compatibility but not distribution requirements. A documentation note would clarify the contract without adding a runtime check.
|
Merged up and fixed a merge conflict. I am trying to clear the merge queue |
Which issue does this PR close?
AggregateExec#25928Rationale for this change
See issue for backgrounds.
It implements a
AggregateExec::try_combine_partial_final()helper, to replace the full logic inCombineParitalFinalAggregateoptimizer rule (validation + transformation)The benefit of this pattern is
(The current implementation already did a great job unifying all logic inside the optimizer implementation, however it's common for other rules to implement partial logic in optimizer, and partial logic in
ExecutionPlan, which introduces inconsistency. Moving all of them to a atomic optimizer hook inExecutionPlanseems like a good pattern to avoid such issues)AggregateExecfor optimizer usage, they're in different crate.What changes are included in this PR?
What is the testing strategy for this PR?
Existing tests
Are there any user-facing changes?
No