Skip to content

Implement native function of datediff #2501

Description

@Sigma-Ma

Is your feature request related to a problem? Please describe.

Spark datediff(endDate, startDate) is not supported in Auron's native execution path. NativeConverters does not handle Spark's DateDiff expression, so queries using it remain on the fallback path.

Describe the solution you'd like

Add Spark-compatible native support for datediff(endDate, startDate) through the extension-function path:

  • convert Spark DateDiff in NativeConverters
  • register and implement a native Spark_DateDiff function
  • return the signed number of days from startDate to endDate
  • preserve null propagation and Spark input coercion behavior
  • add coverage in AuronFunctionSuite and Rust unit tests

Expected examples:

SELECT datediff(DATE '2009-07-31', DATE '2009-07-30'); -- 1
SELECT datediff(DATE '2009-07-30', DATE '2009-07-31'); -- -1
SELECT datediff(DATE '2024-03-01', DATE '2024-02-28'); -- 2
SELECT datediff(CAST(NULL AS DATE), DATE '2024-01-01'); -- NULL

Tests should cover positive, negative, and zero differences, leap-year and year-boundary dates, and null inputs.

Describe alternatives you've considered

Keep the current Spark fallback. This preserves correctness, but datediff cannot benefit from native execution.

Additional context

Related native date-function implementations:

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions