Repository navigation
branch-4.2: [fix](nereids) Allow sync MV rewrite across strict-table SQL modes - #69476
Merged
yiguolei merged 1 commit intoOct 23, 2026
Merged
Conversation
…pache#69212) Nereids can fail to rewrite an aggregation query using a compatible synchronous materialized-view rollup when the querying session and the rollup creation session differ only in the `STRICT_TRANS_TABLES` and/or `STRICT_ALL_TABLES` SQL-mode flags. For example: - Rollup creation session: `ONLY_FULL_GROUP_BY` - Query execution session: `ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,STRICT_ALL_TABLES` Neither strict-table flag currently affects the SQL expression semantics relevant to synchronous MV rewriting. However, the existing session-variable compatibility check treats these configurations as different. Consequently, an otherwise eligible synchronous rollup may fail materialized-view rewriting, causing the optimizer to miss a potentially more efficient execution plan. The issue originates from the interaction between persisted session variables and session-variable guards in the Nereids materialized-view rewrite pipeline. 1. `sql_mode` is marked as a result-affecting session variable and is persisted as part of the synchronous materialized view's creation metadata. 2. `SessionVarGuardRewriter.checkSessionVariablesMatch()` performs an exact comparison of the current and persisted session-variable maps. 3. Differences in either strict-table SQL-mode bit cause this comparison to return `false`. 4. `InitMaterializationContextHook.createSyncMvContexts()` uses this compatibility result when constructing the synchronous materialization context. 5. When the session variables are considered incompatible, expressions implementing `NeedSessionVarGuard`, including `SUM`, may be wrapped in `SessionVarGuardExpr`. 6. The additional expression wrapper can prevent the aggregate-rollup matcher from recognizing an otherwise compatible aggregation expression. This results in a materialized-view rewrite failure caused by SQL-mode differences that do not affect the relevant query semantics. This PR introduces a synchronous-MV-specific session-variable compatibility check: `SessionVarGuardRewriter.checkSessionVariablesMatchForSyncMv()` The new comparison: - Allows differences exclusively in `STRICT_TRANS_TABLES` and/or `STRICT_ALL_TABLES`. - Preserves strict comparison of all other SQL-mode bits. - Preserves strict comparison of other result-affecting session variables. - Rejects incompatible or missing session-variable entries. - Does not relax compatibility when differing SQL-mode values cannot be parsed. - Preserves existing behavior for older materialized views without persisted session-variable metadata. The implementation compares SQL modes using their persisted numeric bitmask representations, excluding only: - `SqlModeHelper.MODE_STRICT_TRANS_TABLES` - `SqlModeHelper.MODE_STRICT_ALL_TABLES` Both flags are treated symmetrically because neither currently affects the SQL expression semantics used for synchronous MV rewriting. The change is restricted to synchronous materialized-view context initialization in `InitMaterializationContextHook`. The existing general-purpose `checkSessionVariablesMatch()` method remains unchanged. For an otherwise compatible synchronous aggregate rollup: | Rollup creation SQL mode | Query SQL mode | Expected rewrite behavior | | --- | --- | --- | | `ONLY_FULL_GROUP_BY` | `ONLY_FULL_GROUP_BY` | Eligible | | `ONLY_FULL_GROUP_BY` | `ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES` | Eligible | | `ONLY_FULL_GROUP_BY` | `ONLY_FULL_GROUP_BY,STRICT_ALL_TABLES` | Eligible | | `ONLY_FULL_GROUP_BY` | `ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,STRICT_ALL_TABLES` | Eligible | | `ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,STRICT_ALL_TABLES` | `ONLY_FULL_GROUP_BY` | Eligible | | `ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES` | `ONLY_FULL_GROUP_BY,STRICT_ALL_TABLES` | Eligible | | `ONLY_FULL_GROUP_BY` | `ONLY_FULL_GROUP_BY,PIPES_AS_CONCAT` | Incompatible | | `ONLY_FULL_GROUP_BY` | `ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,STRICT_ALL_TABLES,PIPES_AS_CONCAT` | Incompatible | Other result-affecting session-variable differences, such as changes to `enable_decimal256`, continue to require the existing compatibility protections. Regression test: `regression-test/suites/query_p0/variables_persist/test_sync_rollup_strict_sql_mode.groovy` The test: 1. Creates an Aggregate Key table with `region_id`, `store_id`, and `sales`. 2. Creates a synchronous aggregate rollup under `ONLY_FULL_GROUP_BY`. 3. Waits for the rollup creation job to finish successfully. 4. Validates rollup rewrite eligibility using `EXPLAIN VERBOSE` under four SQL-mode configurations: - `ONLY_FULL_GROUP_BY` - `ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES` - `ONLY_FULL_GROUP_BY,STRICT_ALL_TABLES` - `ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,STRICT_ALL_TABLES` 5. Verifies identical aggregation results across all four configurations. The query used for validation is: ```sql SELECT region_id, SUM(sales) FROM test_sync_rollup_strict_sql_mode GROUP BY region_id ORDER BY region_id; ``` Expected results: | region_id | SUM(sales) | | --- | --- | | 1 | 300 | | 2 | 700 | The regression validates successful rewrite eligibility independently of whether the cost-based optimizer ultimately selects the rollup as the final physical execution plan. --------- Co-authored-by: Aranya Singh Chauhan <aranyasinghchauhan@Aranyas-MacBook-Air.local> (cherry picked from commit 3b8b5d8)
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
Author
|
run buildall |
Contributor
FE UT Coverage ReportIncrement line coverage |
Contributor
FE Regression Coverage ReportIncrement line coverage |
yiguolei
approved these changes
Oct 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Backport of #69212 to
branch-4.2.Original merged commit:
3b8b5d8f54ce69076f83e343d3838bbbf5c613d2This fix allows synchronous materialized-view rewriting when the query session and MV creation session differ only in the
STRICT_TRANS_TABLESand/orSTRICT_ALL_TABLESSQL-mode bits.Other SQL-mode bits and result-affecting session variables continue to be compared strictly.
Backport changes
Cherry-picked the original fix onto
branch-4.2.The regression test and expected output were placed under
nereids_p0/variables_persist, following the directory structure of the release branch.No intentional changes were made to the original fix's behavior.
Validation
branch-4.2VariablePersistTest)Release note
Fix synchronous materialized-view rollup rewriting when the query and materialized-view creation sessions differ only in the STRICT_TRANS_TABLES and/or STRICT_ALL_TABLES SQL-mode flags.