chore: test latest object_store before 0.14.2 release - #25113
Draft
alamb wants to merge 8 commits into
Draft
Conversation
Pin arrow/parquet crates to arrow-rs rev 901e69f913bb8a1738009218898959e8f4cd7ec8 via [patch.crates-io] until arrow 60.0.0 is released. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tore 0.14, etc) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
# Conflicts: # datafusion/core/tests/user_defined/user_defined_aggregates.rs # datafusion/core/tests/user_defined/user_defined_scalar_functions.rs # datafusion/core/tests/user_defined/user_defined_window_functions.rs # datafusion/ffi/src/udaf/mod.rs # datafusion/optimizer/src/optimizer.rs
…anges - data_buffers() now returns &Arc<[Buffer]> and GenericByteViewArray::new_unchecked takes Arc<[Buffer]> - ParquetMetaData::column_index()/offset_index() replaced by page_index() returning the new PageIndex struct - PageIndex cannot be constructed outside the parquet crate (apache/arrow-rs#10824): patch serialized index bytes in statistics_order_tests and drop the synthetic page index from single_column_metadata until upstream exposes a constructor - New ColumnOrder/SortOrder variants: INT96 now reports SortOrder::INT96_TIMESTAMP (still treated as untrusted for min/max), and ColumnOrder union fields 2/3 are now recognized
github-actions
Bot
added
sql
Open
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
0.14.2(non-breaking) - Target September 2026 arrow-rs-object-store#832Note: this PR is for testing only and is not intended to be merged. Once
object_store0.14.2 is released, DataFusion will pick it up via the normal dependency update process.Rationale for this change
Test DataFusion against the head of apache/arrow-rs-object-store ahead of the
0.14.2release, to find regressions before the release is cut (the same exercise as #24926 for sqlparser 0.63.0).What changes are included in this PR?
object_store0.13.2 because theparquetcrate from arrow-rs 59 requires it, so testingobject_store0.14.x requires the arrow-rsmainpin from that PR as well.object_storeto git rev135984fdf91f813c48c47b6288586d9b0e205164(current head ofmain, the code that will become 0.14.2 — 21 commits pastv0.14.1)No code changes were required on top of #24366.
Are these changes tested?
Existing CI runs against the pinned
object_store.Are there any user-facing changes?
No