IS NULL / IS NOT NULL on a required nested field is rewritten without checking optional ancestors #3292

Apache Iceberg Rust version

main (0.10.x)

Describe the bug

Predicate::bind rewrites IS NULL on a required field to AlwaysFalse and IS NOT NULL to AlwaysTrue from the leaf field's own required flag alone, without checking whether its ancestor structs are nullable:

https://github.com/apache/iceberg-rust/blob/86d661880/crates/iceberg/src/expr/predicate.rs#L388-L398

The spec says a null parent struct implies its leaves are null: "if a parent struct column is null it implies the leaf column is null" (format/spec.md). So the rewrite only holds when every ancestor of the field is also required. Java applies it that way: it walks the ancestors and only shortcuts when all of them are required (UnboundPredicate in the Java library).

With person optional and person.age required, person.age IS NULL binds to AlwaysFalse, so a scan over a file where some row has a null person returns no rows instead of the rows whose person is null. IS NOT NULL has the mirror problem: it binds to AlwaysTrue and keeps rows it should drop.

To Reproduce

Schema person: optional struct<age: required int>, and a file with one row whose person is null. Reference::new("person.age").is_null().bind(schema, false) returns AlwaysFalse, and a scan with that predicate drops the null-person row instead of keeping it.

Expected behavior

IS NULL on a required field should rewrite to AlwaysFalse (and IS NOT NULL to AlwaysTrue) only when the field and all of its ancestor structs are required. When any ancestor is optional, keep the predicate and evaluate it.

Willingness to contribute

I'm willing to work on a fix.

Context

Found while reviewing #2906 (predicates on struct-nested leaf columns). That PR fixes the row-filter build path; this binding gap is separate and predates it, but it becomes reachable once predicates on nested leaves work.