Is your feature request related to a problem? Please describe.
Spark datediff(endDate, startDate) is not supported in Auron's native execution path. NativeConverters does not handle Spark's DateDiff expression, so queries using it remain on the fallback path.
Describe the solution you'd like
Add Spark-compatible native support for datediff(endDate, startDate) through the extension-function path:
- convert Spark
DateDiff in NativeConverters
- register and implement a native
Spark_DateDiff function
- return the signed number of days from
startDate to endDate
- preserve null propagation and Spark input coercion behavior
- add coverage in
AuronFunctionSuite and Rust unit tests
Expected examples:
SELECT datediff(DATE '2009-07-31', DATE '2009-07-30'); -- 1
SELECT datediff(DATE '2009-07-30', DATE '2009-07-31'); -- -1
SELECT datediff(DATE '2024-03-01', DATE '2024-02-28'); -- 2
SELECT datediff(CAST(NULL AS DATE), DATE '2024-01-01'); -- NULL
Tests should cover positive, negative, and zero differences, leap-year and year-boundary dates, and null inputs.
Describe alternatives you've considered
Keep the current Spark fallback. This preserves correctness, but datediff cannot benefit from native execution.
Additional context
Related native date-function implementations:
Is your feature request related to a problem? Please describe.
Spark
datediff(endDate, startDate)is not supported in Auron's native execution path.NativeConvertersdoes not handle Spark'sDateDiffexpression, so queries using it remain on the fallback path.Describe the solution you'd like
Add Spark-compatible native support for
datediff(endDate, startDate)through the extension-function path:DateDiffinNativeConvertersSpark_DateDifffunctionstartDatetoendDateAuronFunctionSuiteand Rust unit testsExpected examples:
Tests should cover positive, negative, and zero differences, leap-year and year-boundary dates, and null inputs.
Describe alternatives you've considered
Keep the current Spark fallback. This preserves correctness, but datediff cannot benefit from native execution.
Additional context
Related native date-function implementations: