docs: [branch-1.2] generate docs for the 1.2.0 release

docs/source/user-guide/latest/compatibility/expressions/spark-3.4/aggregate.md

@@ -0,0 +1,123 @@
+<!---
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# Aggregate Expressions
+
+<!--BEGIN:EXPR_COMPAT[aggregate]-->
+
+## ApproximatePercentile
+
+The following cases are not supported by Comet and always fall back to Spark, regardless of any `allowIncompatible` setting:
+
+- The percentage argument must be a foldable literal.
+- The accuracy argument must be a foldable literal.
+- Only byte, short, int, long, float, and double input types are supported.
+
+## Average
+
+The following cases are not supported by Comet and always fall back to Spark, regardless of any `allowIncompatible` setting:
+
+- YearMonthIntervalType and DayTimeIntervalType inputs are not supported
+
+## CollectSet
+
+The following incompatibilities cause `CollectSet` to fall back to Spark by default. Set `spark.comet.expression.CollectSet.allowIncompatible=true` to enable Comet acceleration despite these differences.
+
+- Before Spark 4.2, Comet deduplicates NaN values (treats `NaN == NaN`) while Spark treats each NaN as a distinct value. Comet treats -0.0 and 0.0 as distinct while Spark treats them as equal. When `spark.comet.exec.strictFloatingPoint=true`, `collect_set` on floating-point types falls back to Spark on those versions unless `spark.comet.expression.CollectSet.allowIncompatible=true` is set.
+
+## First
+
+The following differences from Spark are always present and do not require any additional configuration:
+
+- This function is not deterministic. Results may not match Spark.
+
+## HyperLogLogPlusPlus
+
+The following cases are not supported by Comet and always fall back to Spark, regardless of any `allowIncompatible` setting:
+
+- Input type must be boolean, integral, floating-point, decimal, date/time, string, or binary
+- `DecimalType` with precision > 18 is not supported
+- Collated (non-UTF8_BINARY) strings are not supported
+
+## Last
+
+The following differences from Spark are always present and do not require any additional configuration:
+
+- This function is not deterministic. Results may not match Spark.
+
+## Max
+
+The following cases are not supported by Comet and always fall back to Spark, regardless of any `allowIncompatible` setting:
+
+- Input of a type other than boolean, integral, floating-point, decimal, date, timestamp, string or binary, such as timestamp without time zone, intervals, arrays and structs.
+- A string with a non-default collation (Spark 4.0+), which Spark orders by its collation while Comet compares raw bytes.
+
+## MaxBy
+
+The following differences from Spark are always present and do not require any additional configuration:
+
+- This function is non-deterministic when multiple rows share the maximum ordering value. Results may differ from Spark in that case.
+
+The following cases are not supported by Comet and always fall back to Spark, regardless of any `allowIncompatible` setting:
+
+- The value and ordering must both be fixed-length types (boolean, integral, floating-point, decimal, date, or timestamp). A variable-length or nested type such as string, binary, or struct falls back to Spark.
+
+## Min
+
+The following cases are not supported by Comet and always fall back to Spark, regardless of any `allowIncompatible` setting:
+
+- Input of a type other than boolean, integral, floating-point, decimal, date, timestamp, string or binary, such as timestamp without time zone, intervals, arrays and structs.
+- A string with a non-default collation (Spark 4.0+), which Spark orders by its collation while Comet compares raw bytes.
+
+## MinBy
+
+The following differences from Spark are always present and do not require any additional configuration:
+
+- This function is non-deterministic when multiple rows share the minimum ordering value. Results may differ from Spark in that case.
+
+The following cases are not supported by Comet and always fall back to Spark, regardless of any `allowIncompatible` setting:
+
+- The value and ordering must both be fixed-length types (boolean, integral, floating-point, decimal, date, or timestamp). A variable-length or nested type such as string, binary, or struct falls back to Spark.
+
+## Mode
+
+The following incompatibilities cause `Mode` to fall back to Spark by default. Set `spark.comet.expression.Mode.allowIncompatible=true` to enable Comet acceleration despite these differences.
+
+- mode breaks ties non-deterministically in Spark (the result depends on JVM hash-map iteration order); Comet returns the smallest of the tied values instead (https://github.com/apache/datafusion-comet/issues/3970)
+
+## Percentile
+
+The following cases are not supported by Comet and always fall back to Spark, regardless of any `allowIncompatible` setting:
+
+- An array of percentages is not supported.
+- The percentage argument must be a literal.
+- A frequency argument is not supported.
+- Descending order in `WITHIN GROUP (ORDER BY ... DESC)` is not supported.
+- Only numeric input types are supported.
+
+## PivotFirst
+
+The following cases are not supported by Comet and always fall back to Spark, regardless of any `allowIncompatible` setting:
+
+- Value data type other than Boolean, Byte, Short, Int, Long, Float, Double or Decimal
+- Pivot column of a type other than Boolean, Byte, Short, Int, Long, Float, Double, Decimal, String (UTF8_BINARY collation), Date, Timestamp or TimestampNTZ, which includes the struct that a pivot on more than one column builds
+- Pivot values list is empty
+- Pivot values list contains duplicates, including `0.0` next to `-0.0`
+
+<!--END:EXPR_COMPAT-->

docs/source/user-guide/latest/compatibility/expressions/spark-3.4/array.md

@@ -0,0 +1,113 @@
+<!---
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# Array Expressions
+
+<!--BEGIN:EXPR_COMPAT[array]-->
+
+## ArrayAppend
+
+The following cases are not supported by Comet and always fall back to Spark, regardless of any `allowIncompatible` setting:
+
+- array_append with a nullable array and a nondeterministic operand such as `rand()`: the native NULL guard evaluates the operands more than once, on subsets of the rows
+
+## ArrayContains
+
+For applicable cases that are not selected for native execution automatically, `ArrayContains` is evaluated in the JVM using Spark's own code-generated implementation (run inside the Comet pipeline) by default. Set `spark.comet.expression.ArrayContains.allowIncompatible=true` to explicitly select Comet's native implementation, which has the following differences from Spark:
+
+- Spark compares array elements with ordering.equiv, so -0.0 matches +0.0 and all NaNs match each other; Comet's native array_contains applies that only to flat FLOAT and DOUBLE elements, and nested float elements stay on Spark's comparison
+- array_contains evaluates its value eagerly, while Spark skips it when the array is null
+- Spark compares non-UTF8_BINARY collated string elements under their collation, while Comet's native array_contains compares raw bytes
+
+## ArrayDistinct
+
+The following incompatibilities cause `ArrayDistinct` to fall back to Spark by default. Set `spark.comet.expression.ArrayDistinct.allowIncompatible=true` to enable Comet acceleration despite these differences.
+
+- Floating-point elements match Spark's signed-zero semantics natively only on Spark 4.0.5+, 4.1.4+ and 4.2+, which treat -0.0 and 0.0 as one value in these functions (SPARK-54918, SPARK-59602)
+- Spark compares non-UTF8_BINARY collated string elements under their collation, while Comet's native array_distinct compares raw bytes
+
+## ArrayExcept
+
+By default, `ArrayExcept` is evaluated in the JVM using Spark's own code-generated implementation (run inside the Comet pipeline), which matches Spark exactly. Set `spark.comet.expression.ArrayExcept.allowIncompatible=true` to opt into Comet's native implementation instead, which has the following differences from Spark:
+
+- Null handling and ordering may differ from Spark
+
+## ArrayIntersect
+
+By default, `ArrayIntersect` is evaluated in the JVM using Spark's own code-generated implementation (run inside the Comet pipeline), which matches Spark exactly. Set `spark.comet.expression.ArrayIntersect.allowIncompatible=true` to opt into Comet's native implementation instead, which has the following differences from Spark:
+
+- Result array element order may differ from Spark when the right array is longer than the left (DataFusion probes the longer side).
+- array_intersect does not propagate non-UTF8_BINARY collations to the output array elements (https://github.com/apache/datafusion-comet/issues/2190)
+
+## ArrayJoin
+
+By default, `ArrayJoin` is evaluated in the JVM using Spark's own code-generated implementation (run inside the Comet pipeline), which matches Spark exactly. Set `spark.comet.expression.ArrayJoin.allowIncompatible=true` to opt into Comet's native implementation instead, which has the following differences from Spark:
+
+- array_join does not propagate non-UTF8_BINARY collations to the output string (https://github.com/apache/datafusion-comet/issues/2190)
+- array_join evaluates its delimiter and null replacement eagerly, while Spark short-circuits past them (https://github.com/apache/datafusion-comet/issues/3178)
+
+## ArrayMax
+
+For applicable cases that are not selected for native execution automatically, `ArrayMax` is evaluated in the JVM using Spark's own code-generated implementation (run inside the Comet pipeline) by default. Set `spark.comet.expression.ArrayMax.allowIncompatible=true` to explicitly select Comet's native implementation, which has the following differences from Spark:
+
+- Array extrema use binary string ordering for non-UTF8_BINARY collations (https://github.com/apache/datafusion-comet/issues/4496).
+
+## ArrayMin
+
+For applicable cases that are not selected for native execution automatically, `ArrayMin` is evaluated in the JVM using Spark's own code-generated implementation (run inside the Comet pipeline) by default. Set `spark.comet.expression.ArrayMin.allowIncompatible=true` to explicitly select Comet's native implementation, which has the following differences from Spark:
+
+- Array extrema use binary string ordering for non-UTF8_BINARY collations (https://github.com/apache/datafusion-comet/issues/4496).
+
+## ArrayUnion
+
+The following incompatibilities cause `ArrayUnion` to fall back to Spark by default. Set `spark.comet.expression.ArrayUnion.allowIncompatible=true` to enable Comet acceleration despite these differences.
+
+- Floating-point elements match Spark's signed-zero semantics natively only on Spark 4.0.5+, 4.1.4+ and 4.2+, which treat -0.0 and 0.0 as one value in these functions (SPARK-54918, SPARK-59602)
+- Spark compares non-UTF8_BINARY collated string elements under their collation, while Comet's native array_union compares raw bytes
+
+## ArraysOverlap
+
+For applicable cases that are not selected for native execution automatically, `ArraysOverlap` is evaluated in the JVM using Spark's own code-generated implementation (run inside the Comet pipeline) by default. Set `spark.comet.expression.ArraysOverlap.allowIncompatible=true` to explicitly select Comet's native implementation, which has the following differences from Spark:
+
+- Spark compares non-UTF8_BINARY collated string elements under their collation, while Comet's native arrays_overlap compares raw bytes
+
+## ArraysZip
+
+The following cases are not supported by Comet and always fall back to Spark, regardless of any `allowIncompatible` setting:
+
+- Not all input data types are supported; falls back to Spark for unsupported types
+
+## Sequence
+
+The following differences from Spark are always present and do not require any additional configuration:
+
+- When `stop - start` overflows a long but the sequence is short, Spark 4.0.5, 4.1.4, 4.2.1 and later return its elements (SPARK-58821), and earlier releases raise `INTERNAL_ERROR`. Comet chooses the behavior by the Spark version number. On a vendor build that reports an older version but includes SPARK-58821, Comet raises the error where Spark returns the elements.
+
+The following cases have no native implementation and always run in the JVM using Spark's code-generated implementation (inside the Comet pipeline):
+
+- date and timestamp element types run through the JVM codegen dispatcher
+- sequence arguments must be literals or column references; other shapes run through the JVM codegen dispatcher to preserve Spark's per-row null short-circuit
+
+## Shuffle
+
+The following cases are not supported by Comet and always fall back to Spark, regardless of any `allowIncompatible` setting:
+
+- shuffle requires a resolved random seed
+
+<!--END:EXPR_COMPAT-->