Skip to content

isthmus: SqlExpressionToSubstrait builds its base schema with the names-discarding toSubstrait, so nested field names are dropped #1256

Description

@nielspardon

SqlExpressionToSubstrait has its own private toNamedStruct, and it builds the struct with the single-argument TypeConverter.toSubstrait(RelDataType):

for (Map.Entry<String, RelDataType> entry : nameToTypeMap.entrySet()) {
  names.add(entry.getKey());
  types.add(converterProvider.getTypeConverter().toSubstrait(entry.getValue()));
}
return NamedStruct.of(names, Type.Struct.builder().fields(types).nullable(false).build());

That overload allocates a throwaway list for the depth-first field names and never hands it back, so only the top-level column names reach names while the struct keeps every nested field. NamedStruct.of does not check the two against each other, so the malformed schema is written out:

new SqlExpressionToSubstrait().convert("n + 1", List.of("CREATE TABLE t (s ROW(x INT, y VARCHAR), n BIGINT)"))

// baseSchema.names = [S, N]          -- 2 names
// baseSchema.struct = struct{ struct{ i32, string }, i64 }   -- 4 depth-first name slots

TypeConverter.toNamedStruct on the same row type produces [s, x, y, n], which is what the plan path emits, so plans and extended expressions disagree on the same DDL. Reading the extended expression back and applying the names — toCalcite(factory, struct, names) — then indexes past the end of the list.

The fix is to call the existing TypeConverter.toNamedStruct rather than re-deriving the schema, which also removes one of the duplicate depth-first walkers noted in #1178.

Distinct from #1200 (the same class binds column indexes within their own table), #1201 and #1202.

Measured on main at 944b921.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingisthmus

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions