DDL support for Indexes over unnestedRecordTypes

[g31pranjal]

Motivation

CREATE INDEX ... AS SELECT and CREATE INDEX ... ON <view> can already index an unnesting of an array, by emitting a fan-out key expression on the stored table:

CREATE INDEX mv1 AS SELECT SQ.x, t.p
  FROM T AS t, (SELECT x FROM t.a) SQ ORDER BY SQ.x, t.p

concat( field("A").nest(field("values", FanOut).nest("X")), field("P") )

That works for most shapes, but there is a class of index it simply cannot express, which was previously rejected outright:

Index with multiple disconnected references to the same column are not supported

This PR defines those indexes on an UnnestedRecordType instead, where each unnested array becomes a nested constituent and the key expression navigates from a constituent alias. A constituent is navigated with FanType.None and holds exactly one array element per synthetic record, so it can be referenced at any number of key positions.

The rule: a synthetic type is created only when a fan-out cannot express the index

A fan-out is fine as long as every column read through one unnesting sits in a contiguous run of the index key — those columns are emitted under a single navigation into the array and yield one index entry per element.

A synthetic type is required iff two or more columns reached through the same unnesting are non-adjacent in the index key.

Shape	Example	Representation
One column per element	SELECT SQ.x, t.p	fan-out on stored table
Several adjacent columns of one element	T1.col5, X.col3, X.col4	fan-out (merged into one navigation)
Two independent unnestings	FROM T1, T1.A X, T1.A Y → X.col3, Y.col4	fan-out
Chained, columns adjacent	b.x, c.y	fan-out
Same unnesting, split	X.col2, T1.col5, X.col3	synthetic type
Chained, split	b.x, a.k, c.y	synthetic type

Two notes on rows 3 and 4, since both look like they might need correlation but do not:

• Two independent unnestings each get their own fan-out, producing an n² cross-product. That is the intended meaning — FROM T1, T1.A X, T1.A Y is a cross join and no correlation between X and Y is wanted. A synthetic type would produce the same n² via two constituents, so it adds nothing.
• Chained but adjacent columns share a single outer navigation, so y is already correlated to the element that supplied x — one entry per (p, q) pair.

Because of this, the discriminator is the unnesting, not the array field: X and Y above target the same field A yet are distinct unnestings. Every unnesting a column is read through counts, not just the innermost — under chained unnesting b.x and c.y have different innermost unnestings but both traverse the outer one.

What is now supported

Index shapes. Every row of the table above now produces working metadata; the two marked synthetic type are what previously failed. Beyond the split cases, a synthetic type also covers chained unnesting — (SELECT * FROM b.q) c where b is itself unnested — and several independent unnestings in one index. One constituent is registered per unnested struct array, in a deterministic order, and each constituent’s parent is either the stored record or the constituent it is unnested from, so the constituents always form a proper chain. A scalar array never becomes a constituent: it stays a fan-out under whichever constituent owns it, which is what lets struct and scalar arrays be mixed in a single index.

Syntax. All of the above is reachable from either DDL form, CREATE INDEX ... ON <view> and CREATE INDEX ... AS SELECT, with the unnesting written either as a correlated subquery (SELECT x FROM t.a) SQ or as a PartiQL path t.a AS M. The four combinations converge on the same generator and produce identical metadata.

Carried through. Everything the fan-out path already supported still applies on a synthetic type: VALUE and VERSION index types, keyWithValue split points, and serialization to RecordMetaData for either array storage form — a plain repeated field, or the nullable { repeated T values; } wrapper, where the constituent is the element type and the nesting expression steps through values.

Left alone. An index that does not need a synthetic type keeps its previous representation byte for byte; every pre-existing IndexTest expectation for fan-out shapes passes unchanged.

Limitations

Predicates are not supported on an index over a synthetic type. Explicitly left unsupported for now.

Aggregate indexes always use the stored-table fan-out, unchanged.

Future work

Planner support. Nothing yet matches a query against an index defined on a synthetic type, so these indexes are written but never chosen. Needed for both unnested and joined synthetic types.

JoinedRecordType. Joined synthetic types would take the same DDL route as unnested ones. RecordLayerSyntheticTable is sealed and currently has a single subclass, RecordLayerUnnestedSyntheticTable; a joined type would be the second.

Composability of synthetic types. Unnesting over a joined type, and joining over an unnested type, so that the two kinds compose rather than being mutually exclusive choices.

Implementation notes

• RecordLayerSyntheticTable — sealed base implementing View; a synthetic type is a virtual, SQL-defined, indexed type. RecordLayerUnnestedSyntheticTable is the one subclass.
• RecordLayerSchemaTemplate keeps synthetic tables in their own set, with addSyntheticTable and name-collision checks shared with tables/views/routines. accept() visits them explicitly, since getViews() returns only plain views.
• MaterializedViewIndexGenerator decides representation after the key columns and their order are known, then either registers a synthetic type and builds constituent-alias paths, or falls through to the existing fan-out path.
• A scalar array never needs a constituent: its elements have no fields, so a scalar unnesting contributes at most one key column and can therefore never be the non-adjacent reference that forces correlation. It stays a fan-out, emitted under the constituent that owns the array when the array sits inside an unnested struct element. The serializer keeps a defensive assert for the invariant.
• A constituent stores the nesting KeyExpression, not an array field name — the record layer persists and evaluates that expression, so keeping it verbatim round-trips any shape, including two-hop paths like map.entry that a name cannot hold. The serializer walks the expression’s field path to find the element descriptor, replacing the wrapped/unwrapped special case.
• GenerationResult → IndexGenerationResult, extracted to its own file. Both index generators and the visitor chain return it, so a synthetic type is always plumbed to a caller that can register it.
• Explode identity is carried by a marker stamped onto an AnnotatedAccessor, so two unnestings of the same array field stay distinct. Constituent parents are resolved by dereferencing the collection value rather than trusting its raw correlation, which for a chained explode points at the enclosing subquery quantifier rather than at the constituent.

Tests

28 new tests, 2 converted, 1 new file.

UnnestedSyntheticTypeParsingTest (new, 16 tests) — 4 scenarios × 4 spellings: single struct array, three struct arrays, scalar array, struct + scalar. Asserts synthetic name, parent table, constituent array fields and parent aliases, index type and table, and the exact key expression.

IndexTest (+8 new, 2 converted, 106 total):

• createIndexWithRepeatedNestedSplitByField, createVersionIndexWithRepeatedNestedSplitByVersion - converted from shouldFailWith to positive assertions; these are the newly-supported shapes.
• createIndexWithRepeatedNestedSplitByOtherRepeated — split by a different unnesting.
• createdIndexWorksChainedUnnesting{Adjacent,SplitByParent,InnerSplit,OuterSplit} — chained unnesting, including the fan-out control case.
• createIndexWithPredicateOverUnnestedSyntheticTypeIsNotSupported / ...OverUnnestingIsSupported / createIndexWithPredicateIsSupportedWhenUnnestingNeedsNoSyntheticType — the predicate restriction and both sides of its boundary: the same columns and predicate are accepted once the key order makes the shape expressible as a fan-out.

The syntheticIndexIs helper additionally asserts two invariants, each of which caught a real bug during development: that every constituent's parent is the stored-record constituent or another constituent, and that the metadata actually serializes with the synthetic type present.

SchemaTemplateSerDeTests (+4 tests) — round trips a synthetic type through RecordMetaData and back via fromRecordMetadata, asserting that the nesting expression form, constituent parent chain, array field names, and index key expression all survive. One test per storage form — nullable (wrapped) and ARRAY NOT NULL (plain repeated) — plus a chained two-constituent case where the inner array hangs off the outer constituent’s element type, and a map.entry case whose path has two meaningful hops.

These caught a real bug: the deserializer recognised only the unwrapped field(name, FanOut) nesting expression, so a nullable array’s wrapped field(name).nest(field("values", FanOut)) fell through to the constituent alias and rebuilt the array field as the alias rather than the field name — metadata that then fails on re-serialization. That prompted storing the nesting expression instead of a name, which removed the guesswork entirely.

DdlTestUtil.shouldWorkWithInjectedFactory was hoisted out of DdlStatementParsingTest so the new test class does not duplicate the harness.