Repository navigation
Conversation
There was a problem hiding this comment.
🟢 Approval recommended
No unresolved review issues remain, and validation passed.
0 open findings
What changed in this PR
Fixes nullable three-valued AND evaluation in tree-model CASE expressions by caching the computed result.
Changes:
- Corrects
LogicAndColumnTransformercaching. - Adds nullable
ANDunit coverage. - Adds a
CASEintegration regression test.
| File | Description |
|---|---|
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/queryengine/transformation/dag/column/binary/LogicBinaryColumnTransformerTest.java |
Tests nullable AND combinations and selection masks. |
iotdb-core/calc-commons/src/main/java/org/apache/iotdb/calc/transformation/dag/column/binary/LogicAndColumnTransformer.java |
Caches the computed AND result. |
integration-test/src/test/java/org/apache/iotdb/db/it/query/IoTDBCaseWhenThenIT.java |
Adds SQL-level regression coverage. |
🧠 Review effort: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
JackieTien97
left a comment
There was a problem hiding this comment.
The fix looks right; it now matches LogicOrColumnTransformer.evaluateWithSelection. The comments below are about test coverage.
| if (!selection[i] || expected[i] == null) { | ||
| Assert.assertTrue(result.isNull(i)); | ||
| } else { | ||
| Assert.assertEquals(expected[i], result.getBoolean(i)); |
There was a problem hiding this comment.
This branch only checks getBoolean(i), and a NULL position in BooleanColumn reads as false, so the test can't tell FALSE from NULL. That leaves the "left is FALSE, so the result is FALSE" half of the fix unprotected:
- Changing line 73 of
LogicAndColumnTransformertobuilder.appendNull()makes rows 3-5 (left = false) return NULL, and both new tests still pass. - With the pre-fix code, the test only catches row 2 (
NULL AND TRUE, line 166). Rows 3-5 also came back NULL there, which the assertion on this line accepts.
The loop also reads selection after evaluateWithSelection has run, so it wouldn't notice the method mutating it: dropping the selection.clone() on line 36 likewise turns rows 3-5 into NULL, and the tests still pass.
Suggestion: keep a copy of the selection and assert non-null explicitly.
boolean[] selected = selection.clone();
transformer.evaluateWithSelection(selection);
Column result = transformer.getColumn();
Assert.assertEquals(expected.length, result.getPositionCount());
for (int i = 0; i < expected.length; i++) {
if (!selected[i] || expected[i] == null) {
Assert.assertTrue(result.isNull(i));
} else {
Assert.assertFalse(result.isNull(i));
Assert.assertEquals(expected[i], result.getBoolean(i));
}
}With these assertions, the pre-fix code and both changes above fail, and the current code passes.
Minor: the partial mask i % 2 == 0 leaves rows 1 and 7 unselected, and they are the only rows that reach the "right is false" branch (lines 58-60), so that branch never runs with a partial selection. A mask that keeps row 1 or 7 selected would cover it.
| "INSERT INTO root.sg.case_null(time, s1, s2) VALUES (2, 1, 1), (3, 0, 1), (4, 101, 1)" | ||
| }); | ||
| resultSetEqualTest( | ||
| "SELECT CASE WHEN s1 > 100 THEN 1 WHEN s1 > 0 AND s2 > 0 THEN 2 ELSE 3 END AS result " |
There was a problem hiding this comment.
Since the AND only appears as a WHEN condition, NULL and FALSE both mean "no match", so this query only covers the NULL AND TRUE half of the fix (time 1: 2.0 before the fix, 3.0 after). Time 3 (0 > 0 AND 1 > 0) is 3.0 whether the AND returns FALSE or NULL: with line 73 of LogicAndColumnTransformer changed to builder.appendNull(), this query still returns 3.0, 2.0, 3.0, 1.0.
A query that exposes the AND result itself would cover the FALSE half too. On the same data:
| query | before fix | after fix |
|---|---|---|
CASE WHEN s1 > 100 THEN 1 WHEN NOT (s1 > 0 AND s2 > 0) THEN 2 ELSE 3 END |
3.0, 3.0, 3.0, 1.0 |
3.0, 3.0, 2.0, 1.0 |
CASE WHEN s1 > 100 THEN false WHEN s2 > 0 THEN (s1 > 0 AND s2 > 0) END |
true, true, null, false |
null, true, false, false |
Both also fail with the appendNull() change (time 3 becomes 3.0 / null). I got these values by running the expressions through ColumnTransformerVisitor + FilterAndProjectOperator on this PR's code, with and without the fix.
| right.addReferenceCount(); | ||
| left.initFromTsBlock(block); | ||
| right.initFromTsBlock(block); | ||
| LogicAndColumnTransformer transformer = new LogicAndColumnTransformer(returnType, left, right); |
There was a problem hiding this comment.
The bug came from LogicAndColumnTransformer and LogicOrColumnTransformer drifting apart (OR already ends evaluateWithSelection with builder.build()), but this helper hard-codes AND. So OR's evaluateWithSelection still has no NULL/selection coverage: testLogicOr only goes through evaluate(), and this is the only unit test that uses LogicOrColumnTransformer.
Could the helper take the constructor and the expected table, so it runs for both? Something like:
assertSelected(
(l, r) -> new LogicAndColumnTransformer(returnType, l, r),
new Boolean[] {null, false, null, false, false, false, null, false, true});
assertSelected(
(l, r) -> new LogicOrColumnTransformer(returnType, l, r),
new Boolean[] {null, null, true, null, false, true, true, true, true});The OR table passes on the current code (full and partial selection), and fails if OR picks up the same initializeColumnCache(rightColumn) bug.
|
Thanks @JackieTien97! Addressed all three comments in 3316fa6: stronger NULL/selection assertions, AND/OR coverage, and both SQL regressions. All 7 unit tests and the targeted integration test pass. |
Tree-model CASE can take the wrong branch when a later WHEN contains nullable AND. For example, when
s1is NULL ands2is 1,CASE WHEN s1 > 100 THEN 1 WHEN s1 > 0 AND s2 > 0 THEN 2 ELSE 3 ENDreturns 2 instead of 3.LogicAndColumnTransformer.evaluateWithSelection()builds the correct three-valued AND result but caches the right operand. Cache the built result instead. This is a one-line production change and requires no cache-lifecycle or selection API changes.Regression coverage includes all nine NULL/false/true operand combinations with full and partial selection masks, plus the CASE SQL example with NULL, true, false, and earlier-branch rows.
Closes #18864.
Validation:
verifyfrom master8effe5d6ce, selecting onlyLogicBinaryColumnTransformerTest(5 tests) andIoTDBCaseWhenThenIT#testNullableAndInLaterCaseBranch(1 test, rebuilt 1C1D nodes): passed.