Skip to content

docs: plan Parquet pruning optimizations - #2

Draft
rgbuilds wants to merge 1 commit into
mainfrom
cursor/parquet-pruning-plans-5669
Draft

rgbuilds wants to merge 1 commit into
mainfrom
cursor/parquet-pruning-plans-5669

Conversation

@rgbuilds

@rgbuilds rgbuilds commented Oct 3, 2026

Copy link
Copy Markdown
Owner

Which issue does this PR close?

  • Closes #.

Rationale for this change

Document implementation and validation strategies for two potential Parquet pruning performance improvements: lazy fully-matched row-group detection and batched Bloom-filter I/O.

What changes are included in this PR?

  • Add a plan for making fully-matched row-group classification conditional on downstream consumers.
  • Add a plan for replacing serialized Bloom-filter reads with bounded batched I/O.
  • Define correctness coverage, benchmarks, risks, acceptance criteria, and staged PR strategies for both efforts.

What is the testing strategy for this PR?

Documentation formatting and Rust formatting pass. The required all-targets/all-features clippy command was attempted but could not build dependencies because cmake is not installed on the worker; it failed in the protobuf-src and snmalloc-sys build scripts before linting DataFusion code.

Are there any user-facing changes?

No. These are planning documents only.

Open in Web Open in Cursor 

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant