Repository navigation
Conversation
Carry forward the downstream API from fork #73; row order within each row group is unchanged.
Port the fork #73 planner behavior and its four ordering regressions to DF56.
Keep DF56 typed storage and restore the fork review fix rejecting mismatched timezone or decimal metadata. Cover string variants using the upstream generic downcast guard.
Keep fork #73 behavior: replacing incompatible Interleave children returns a Union with every child partition. Update the upstream deferred-invariant regressions to require an executable fallback.
Port fork #77. Adapt Median through the shared DF56 percentile implementation, retaining decimal metadata and the per-group percentile state column. Preserve upstream ordered first/last fast paths and add regression coverage for the changed state layout.
Port fork #91, including preserving reads and filter/null/merge regressions, while retaining DF56 ordered-aggregate fast paths.
Port fork #84 while preserving DF56 null-regex and custom terminator support. CSV scan field 9 now belongs to upstream terminator; use field 10 for header matching and regenerate protobuf bindings. Serialized CSV plans must not be exchanged between DF55-fork and DF56-fork processes.
github-actions
Bot
added
documentation
xudong963
marked this pull request as ready for review
November 13, 2026 07:15
zhuqi-lucas
approved these changes
Nov 13, 2026
zhuqi-lucas
left a comment
zhuqi-lucas
left a comment
There was a problem hiding this comment.
LGTM. Every fork patch on branch-55 is carried here or already in upstream branch-56, and the fork tests survive with no fewer assertions. Two small test notes inline.
| } | ||
|
|
||
| #[test] | ||
| fn preserving_reads_keep_median_decimal_metadata() -> Result<()> { |
There was a problem hiding this comment.
The fork's median.rs::groups_preserving_reads_keep_median_state covered the Float64 case (repeated reads, state values, empty selection). Could we keep a Float64 median version alongside this decimal one?
|
|
||
| #[test] | ||
| fn test_interleave_rebuild_defers_partitioning_invariant() -> Result<()> { | ||
| fn test_interleave_rebuild_falls_back_to_union() -> Result<()> { |
There was a problem hiding this comment.
The fork's interleave_child_rewrite_falls_back_to_union also asserted that a compatible rewrite stays an InterleaveExec with 3 partitions. Could we keep that half too?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
None — maintained fork upgrade.
Rationale for this change
Upgrade the fork to the finalized Apache
branch-56atcce6374e6256, while preserving the DF55 fork behavior.What changes are included in this PR?
964f925bd024baseline: finer sorts above merges, ListingTable expression adapters, per-partition constant ordering, and the final changelog.What is the testing strategy for this PR?
cargo fmt --check, all-target/all-feature Clippy, affected crate tests, andlimit.slt,window.slt, andcsv_files.sltpass locally. GitHub CI is running on2435565f6c41; the local CI-profile workspace run passed 1,502 DataFusion tests before a macOSrust-objcopyartifact failure prevented the extended-test binary from launching.Are there any user-facing changes?
DF56 / Arrow 60.1. CSV physical-plan header matching uses protobuf field 10 because upstream uses field 9 for the terminator. Drain old serialized plans and upgrade producers/consumers together.