Skip to content

isthmus: REVERSE over a binary column silently converts through a latin-1 cast to reverse:str #1275

Description

@nielspardon

functions_string.yaml declares reverse over string, varchar<L1> and fixedchar<L1> only — there is no binary implementation. Calcite's REVERSE is declared with OperandTypes.CHARACTER, but Calcite's implicit type coercion will satisfy that from a binary operand by inserting a character cast, so a binary argument converts rather than being rejected.

With CREATE TABLE bins (b VARBINARY(10)), SELECT reverse(b) FROM bins converts to reverse:str with the argument wrapped as Cast{type=Str, input=FieldReference{type=Binary}}, unparsing as:

REVERSE(CAST("B" AS VARCHAR CHARACTER SET "ISO-8859-1"))

The emitted plan therefore claims a string result derived from latin-1-decoded bytes. A consumer reading that string as UTF-8 does not get the reversed input, and byte sequences that are not valid UTF-8 have no faithful representation at all — reversing a decoded character sequence is not the same operation as reversing bytes.

The coercion itself is generic to Calcite and reaches lower/upper the same way, but there the result is at worst a re-encoding of the same characters; for reverse the ordering makes the result materially wrong.

Worth deciding whether the fix is to reject binary operands for these mappings outright (so the failure is loud), or to keep the coercion and document it. Either way the current behaviour is a silent wrong answer.

Found while reviewing #1251, which adds the reverse mapping.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions