Skip to content

[Improvement] Decouple common-expression materialization from FilterAndProjectOperator #18862

Closed as not planned

Description

@ColinLeeo

Status

Implemented in PR #18826, in the cache ownership fix. This issue duplicates work already tracked in that PR and will not be tracked separately. It is being closed as redundant. The PR is still open, so this does not imply that the fix has already landed on master.

The shared public static FilterAndProjectOperator.appendCommonExpressionColumns() API has been removed. Query filtering and schema attribute UPDATE materialize their common columns locally using ColumnTransformer.getColumnWithSelection(). That API returns caller-owned selected results without overwriting or consuming the shared full-column cache, and explicitly documents original-row alignment and the selected-row contract. No separate utility extraction is needed to remove the coupling described here.

Guarded CAST, zero-match UPDATE, real non-leaf common expressions, and common-expression evaluation counts are covered by passing unit/SQL regressions. Mask allocation/scanning costs and measured performance are documented in the PR.

Original context (before the fix)

Description

PR #18826 shares appendCommonExpressionColumns() between query filtering and schema attribute updates. The helper currently lives as a public static method on FilterAndProjectOperator and embeds its satisfy() filter semantics, coupling schema code to a query operator implementation.

The helper also has lifecycle effects beyond appending to a Java list: it may evaluate an uncached transformer with a row-selection mask, replace descendant caches through selection evaluation, and call getColumn() on each common expression, consuming reference counts and possibly releasing cached columns. Existing cached columns are reused; the helper does not unconditionally reset the DAG.

Follow-up scope

Move common-column materialization into an appropriately scoped calc utility/service with an explicit contract for:

  • original input row indexes and full-length masked columns;
  • false/NULL filter results and zero matching rows;
  • cached-result reuse, evaluation, and reference-count consumption;
  • shared use by query operators and schema UPDATE;
  • allocation/scanning costs when common expressions are present.

Keep the guarded-cast and real non-leaf common-expression regression coverage from PR #18826. Decoupling the helper should not alter short-circuit behavior, cache ownership, or the filtered column layout. Documentation clarifying the existing side effects can be improved independently of this extraction.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions