Skip to content

isthmus: same-named Calcite operators from different SqlLibrary sets make ORDER BY over reverse/least/greatest fail validation #1274

Description

@nielspardon

SubstraitOperatorTable's LIBRARY_OPERATOR_TABLE enables six SqlLibrary sets at once, and several of those contribute operators that share a name while differing only in SqlKind. lookupOperatorOverloads then returns more than one candidate, which is enough to break validation of any expression that Calcite re-resolves by name — in particular anything an ORDER BY refers to.

With two candidates, SqlUtil.lookupSubjectRoutines skips its size() < 2 early return and runs filterOperatorRoutinesByKind(routines, kind), which keeps only operators whose getKind().getFunctionKind() equals the requested kind. For these pairs the requested kind is OTHER_FUNCTION while the candidates' own kinds are REVERSE / REVERSE_SPARK (and LEAST / LEAST_PG, GREATEST / GREATEST_PG), so every candidate is filtered out and lookupRoutine returns null.

Reproduced against main with CREATE TABLE t (c16 CHAR(16), vc32 VARCHAR(32), vc VARCHAR, d DOUBLE, i INT):

SELECT c16 FROM t ORDER BY reverse(c16)          -> No match found for function signature REVERSE(<CHARACTER>)
SELECT reverse(c16) FROM t ORDER BY 1            -> No match found for function signature REVERSE(<CHARACTER>)
SELECT reverse(c16) AS r FROM t ORDER BY r       -> No match found for function signature REVERSE(<CHARACTER>)
SELECT i FROM t ORDER BY least(i, i)             -> No match found for function signature LEAST(<NUMERIC>, <NUMERIC>)
SELECT i FROM t ORDER BY greatest(i, i)          -> No match found for function signature GREATEST(<NUMERIC>, <NUMERIC>)

SELECT d FROM t ORDER BY sinh(d)                 -> ok
SELECT d FROM t ORDER BY log2(d)                 -> ok
SELECT vc32 FROM t ORDER BY starts_with(vc32,'a')-> ok
SELECT c16 FROM t ORDER BY initcap(c16)          -> ok
SELECT reverse(c16), i FROM t ORDER BY i         -> ok    (ORDER BY does not reference the call)

The split tracks the overload count exactly:

REVERSE     count=2   [REVERSE/kind=REVERSE]   [REVERSE/kind=REVERSE_SPARK]
LEAST       count=2   [LEAST/kind=LEAST]       [LEAST/kind=LEAST_PG]
GREATEST    count=2   [GREATEST/kind=GREATEST] [GREATEST/kind=GREATEST_PG]
INITCAP     count=1     SINH count=1     LOG2 count=1     STARTS_WITH count=1

least and greatest have been mapped for a long time, so this is not new — it is simply invisible, because plain projection works and no test puts these functions in an ORDER BY.

The mechanism that fixes it already exists in the file: SUBSTRAIT_SCALAR_OPERATOR_TABLE is consulted before the library table and early-returns on a hit, so registering one chosen operator per colliding name makes resolution single-valued and deterministic. Verified for reverse — registering it there takes the overload count to 1 and all three ORDER BY forms above pass, with :isthmus:check and :isthmus-cli:check green.

Note this is distinct from the N:1 reverse-lookup collision in #1012 (several Calcite operators for one Substrait name); here the collision is on the Calcite side, before any Substrait mapping is consulted. It is also unrelated to #562, which was duplicate names across extension YAMLs.

Found while reviewing #1251, which adds reverse and so adds a third instance.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions