deps: upgrade to DataFusion 56.0.0

contrib/delta/native/Cargo.toml

@@ -47,4 +47,4 @@ crate-type = ["rlib"]
 datafusion-comet-proto = { path = "../../../native/proto" }
 # For `ExecutionPlan` (return type), `SchemaRef` (the shim hands these in), and `DataFusionError`
 # (the not-implemented error). Versions/features mirror core so the shared lockfile resolves cleanly.
-datafusion = { version = "55.2.0", default-features = false, features = ["parquet"] }
+datafusion = { git = "https://github.com/apache/datafusion", rev = "350ca76505c9513d16e7dd9910bcd9bde2c911ed", default-features = false, features = ["parquet"] }

contrib/lance/native/Cargo.toml

@@ -28,5 +28,5 @@ license = "Apache-2.0"
 crate-type = ["rlib"]
 
 [dependencies]
-datafusion = { version = "55.0.0", default-features = false }
+datafusion = { git = "https://github.com/apache/datafusion", rev = "350ca76505c9513d16e7dd9910bcd9bde2c911ed", default-features = false }
 datafusion-comet-proto = { path = "../../../native/proto" }

docs/source/user-guide/latest/iceberg.md

@@ -822,6 +822,12 @@ The native Iceberg writer is disabled by default. With `spark.comet.write.iceber
   iceberg-java rejects the value. The native writer also trims or lowercases a few other settings
   that iceberg-java reads as written
   ([#7008](https://github.com/apache/datafusion-comet/issues/7008)).
+- **All-NaN `float` and `double` columns.** When a data file holds a column chunk with only NaN
+  values, the native writer records NaN as that column's lower and upper bound in the manifest.
+  iceberg-java records no bound, and the Iceberg spec does not allow NaN as a bound. This comes
+  from the Parquet writer, which writes NaN statistics for such a chunk since Parquet 60, and from
+  iceberg-rust, which copies them into the bounds
+  ([#7242](https://github.com/apache/datafusion-comet/issues/7242)).
 
 The physical differences that every native data file has, and why they are safe, are listed in
 [Accepted divergences](#accepted-divergences).

docs/source/user-guide/latest/migration-guide.md

@@ -57,6 +57,25 @@ Treat setting one of these keys as a temporary measure. If you find you cannot s
 legacy behavior, please open an issue describing your use case so it can be considered before the
 key is removed.
 
+## Upgrading to Comet 1.3.0
+
+Comet `1.3.0` makes no behavior changes that need a `spark.comet.legacy.*` key. The known issue
+below needs none either, but check whether it applies to your deployment.
+
+### Native Iceberg Writer Bounds for All-NaN Columns
+
+This applies only if you enable the native Iceberg writer with `spark.comet.write.iceberg.enabled`,
+which is off by default. Comet `1.3.0` moves to Parquet 60, which writes NaN min and max statistics
+for a `float` or `double` column chunk that holds only NaN values. Earlier versions wrote none.
+The native writer copies those statistics into the data file's lower and upper bounds, so the
+manifest records NaN as a bound where iceberg-java records none, and the Iceberg spec does not
+allow NaN as a bound. Comet `1.2.0` and earlier do not write these bounds.
+
+The data and the NaN counts are unaffected, and no setting restores the previous bounds. Disable
+`spark.comet.write.iceberg.enabled` to have iceberg-java write the files. The fix belongs in
+iceberg-rust and is tracked in
+[#7242](https://github.com/apache/datafusion-comet/issues/7242).
+
 ## Upgrading to Comet 1.2.0
 
 Comet `1.2.0` makes no behavior changes that need a `spark.comet.legacy.*` key. The changes below