[mohammadnaqvi04]

Which issue does this PR close?

Part of #24960. This PR adds a safe rejection until IN/NOT IN count-bug compensation is implemented.

Rationale for this change

A correlated IN/NOT IN over an aggregate without GROUP BY returns wrong rows when an outer row has no match (see #24960 (comment)). #25391 fixed EXISTS/NOT EXISTS. IN/NOT IN will be covered in a follow-up, so until then these queries should error instead of returning wrong results.

What changes are included in this PR?

build_join in decorrelate_predicate_subquery.rs now keeps these subqueries correlated, so they fail to plan with a not-implemented error. The exception is an IN in a WHERE over an aggregate that is NULL on empty input (e.g. max, sum), which the current rewrite already answers correctly.

What is the testing strategy for this PR?

• The existing IN over max test still passes unchanged.
• The existing NOT IN over count test now expects the error.
• New tests expect the error for IN over count in a filter and in a projection, NOT IN over max, and IN over max in a projection.

Are there any user-facing changes?

These queries now return a not-implemented error instead of wrong results.

[mohammadnaqvi04]

@jayzhan211 this addresses your comment (#24960 (comment)) similar to how #26544 handles its no row for an unmatched outer row case.

[jayzhan211]

nit: could this point at the tracking issue, so whoever adds the IN count-bug compensation knows to remove the block?

-    // Stopgap until a follow-up adds count-bug compensation for `IN`/`NOT IN`.
+    // TODO(#24960): stopgap until count-bug compensation covers `IN`/`NOT IN`.

[mohammadnaqvi04]

Done, now points at #24960.

[jayzhan211]

nit: the extra rewrite(true) only finds something when a groupless aggregate was pulled up, and the first rewrite already records that. Gating on it skips the second pass for every other IN subquery. Optimizer tests and subquery.slt still pass with this:

-    if !exists_sub_query {
+    if !exists_sub_query && pull_up.pulled_up_scalar_agg {

[mohammadnaqvi04]

This makes sense. The second pass can't find anything when pulled_up_scalar_agg isn't set. Done.

[jayzhan211]

nit: the LeftMark branch that is allowed through (an IN under OR in a WHERE) has no test. Something like this would pin it down:

SELECT t1_id FROM t1
WHERE t1_int + 2 IN (SELECT max(t2_int) FROM t2 WHERE t2.t2_id = t1.t1_id) OR t1_id = 33
ORDER BY t1_id;
-- 11
-- 33

[mohammadnaqvi04]

Added, thanks.

[jayzhan211]

@mohammadnaqvi04 Thanks for the fix! I left a few nits inline that are still open, so please take a look when you get a chance.

[alamb]

This PR appears to be waiting on the author to respond to reviewer feedback so marking as draft. Please mark ready to review when it is ready for another look