fix(meerkat-core): dedupe array join keys before UNNEST in v2 joins

[PutluruAravindaReddy]

Summary

• Problem: v2 array joins use UNNEST to turn an ID list into rows before joining. Every list entry becomes a row, so when the same ID appears more than once, the joined row repeats. Example: an SLA reuses the same metric definitions across many policies, so a ticket joined to its SLA metrics showed 66 rows instead of 4. The v1 CONTAINS join matched each ID once, and v2 was meant to differ from v1 only in speed.
• Fix: remove duplicate IDs (list_distinct) before UNNEST. list_distinct also drops NULLs, so when the list has a NULL, one NULL is added back. NULL entries behave exactly as they do today.
• Scope: only applied when a table has a single array join column. With two or more, DuckDB pairs the unnested values by position, and de-duplicating each list separately would break that pairing, so those stay unchanged.
• Uses only SQL functions the production query server allows (list_append and list_count are rejected there).

What changes

ID list	Before	After
[a, b]	2 rows	2 rows
[a, a, b]	3 rows	2 rows
[a, NULL, a]	3 rows	2 rows
[NULL]	1 row	1 row
[] or NULL	row dropped	row dropped
Two array join columns on one table	paired by position	unchanged

Only duplicate values change. Everything else produces the same rows as before.

When this happens, and why it wasn't caught earlier

The repetition comes from the array side of a join: a column holding a list of IDs, matched against the other table's id.

 ticket → sla (normal join) → policies[*].metrics[*].metric_definition_id → metric_definition
                              ↑ array of IDs; UNNEST expands every entry

• ticket → sla is a normal one-to-one join, so it isn't affected.
• sla → metric_definition joins through an array. One SLA lists 66 metric IDs across its policies but only 4 are unique, because each policy repeats the same metrics. UNNEST turns all 66 into rows, so each ticket repeats 66 times.
• Any other array join with repeated IDs behaves the same way, for example issue.owned_by_ids → dev_user.id if an owner appeared twice.
• Not affected: joins on single-value columns (sla_id, part_id), and arrays that are only shown as columns, not used in a join.

Why now: the UNNEST join was added in #259 (May 2026); the v1 CONTAINS join before it didn't have this problem. Most ID arrays (owners, tags, parts) don't repeat IDs, and the existing test fixtures had none, so results looked correct. SLA policies are the first common case where repeats are normal. The editor preview also caps at 20 rows, which hid how large the repetition really was.

DevRev

work-item: CEAAA-103

Test plan

• New DuckDB test (joins each distinct id once when the array has duplicates or NULLs) fails without the fix (7 rows) and passes with it (5 rows).
• New unit test: a table with two array join columns keeps plain UNNEST.
• Updated 7 assertions that check the exact generated SQL.
• meerkat-core: 565 passed (11 pre-existing skips). meerkat-node: 789 passed (117 pre-existing expected failures, 1 skip).
• Lint: 0 errors, no warnings in touched files.
• Ran the generated SQL on the production query server for the affected widget: one ticket went from 66 rows to 4.

Not in this PR

• Version bump (separate PR).
• v2 drops base rows whose array is empty or NULL (existing behaviour; v1 kept them).
• Tables with two array join columns can still repeat rows.

[jayachandranramadoss-devrev]

@shriram-devrev Could you PTAL