Skip to content

fix(scan): prune NOT IN predicates using file stats - #795

Open
jackylee-ch wants to merge 2 commits into
apache:mainfrom
jackylee-ch:fix/prune-not-in-stats
Open

fix(scan): prune NOT IN predicates using file stats#795
jackylee-ch wants to merge 2 commits into
apache:mainfrom
jackylee-ch:fix/prune-not-in-stats

Conversation

@jackylee-ch

@jackylee-ch jackylee-ch commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

On the stats rule, data_leaf_may_match returned true for NotIn
unconditionally, so col NOT IN (...) never used min/max or the null count to
skip anything. That was deliberate — #382 scoped IN pruning and said NOT IN
would keep failing open — and this lifts it.

The rule is the n-ary form of the NotEq rule above it: skip a file only when
some literal equals both bounds, as Java NotIn#test does. Its null-literal arm
has no counterpart because Datum has no null variant. Partly null files are
still pruned: a null row satisfies neither NotIn nor the equality that prunes
it.

Not gated on supports_in_min_max_pruning, unlike IN: NotEq is not gated
either, and x NOT IN (5) has to answer like x <> 5 — a test pins that. So
every consumer gains the rule at once (manifest partition stats, key stats,
Parquet row groups and page index, mosaic), not only data-file stats as in #382.

Both operators now fail open for FLOAT and DOUBLE, which fixes a pre-existing
NotEq defect rather than extending it. datum_cmp compares floats with IEEE
partial_cmp, so -0.0 equals +0.0 there while the residual filter tells them
apart; and NaN enters neither min/max nor the null count, so a file holding
[1.0, NaN] reports min == max == 1.0 over two rows. Measured: <> 1.0
returned [3] where the filter alone returns [2, 3], and <> 0.0 returned
nothing where it returns the -0.0 row. Java is unaffected on the first count —
compareLiteral uses Double.compareTo, which orders -0.0 below +0.0.

Elsewhere results are unchanged, only how many files are read: on a three-file
table where one file holds a single value, value NOT IN (20) moves
manifest_entries_pruned_by_data_stats from 0 to 1, final_files from 3 to 2.

Comment thread crates/paimon/src/predicate_stats.rs Outdated
Comment on lines +157 to +159
PredicateOperator::NotIn => !literals
.iter()
.any(|literal| min_value == *literal && max_value == *literal),

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can drop matching FLOAT/DOUBLE rows: Datum treats signed zeros as equal while the residual distinguishes them, and Parquet min/max excludes NaNs. For example, [1.0, NaN] NOT IN (1.0, 2.0) is pruned entirely instead of retaining the NaN row.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed, and it already bites <> on main: on a file holding [1.0, NaN] (min == max == 1.0, NaN counted as neither bound nor null) <> 1.0 returned [3] while the filter alone returns [2, 3]; on a -0.0 file <> 0.0 returned nothing. datum_cmp is IEEE where the residual is bitwise, and Java's compareLiteral uses Double.compareTo. Both operators now fail open for FLOAT/DOUBLE, so this removes the existing loss rather than extending it. Ordering rules and Eq are untouched.

@QuakeWang QuakeWang left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants