Skip to content

fix: TableNode FILTERS section shows duplicate predicates when same table is referenced multiple times in one statement #44

Description

@cooker-code

Bug Description

When the same table is referenced multiple times within a single SQL statement (e.g., a table appearing in two separate subqueries in the same CTE), the FILTERS section of the corresponding TableNode in the lineage graph shows every predicate twice (or N times for N references).

Root Cause

The Rust core appends each subquery occurrence's WHERE predicates to the flat node.filters array. For a table referenced twice in the same statement with identical filters, this produces 8 entries instead of 4:

"filters": [
  { "expression": "dt = '\${date}'" },
  { "expression": "device_id IS NOT NULL" },
  { "expression": "device_id <> ''" },
  { "expression": "device_id <> '00000000-0000-0000-0000-000000000000'" },
  { "expression": "dt = '\${date}'" },         // duplicate
  { "expression": "device_id IS NOT NULL" },   // duplicate
  { "expression": "device_id <> ''" },         // duplicate
  { "expression": "device_id <> '00000000-0000-0000-0000-000000000000'" }  // duplicate
]

The frontend had no deduplication before displaying node.filters, so all 8 entries were rendered.

Reproduction

SQL with the same physical table used in two subqueries within one CTE:

with new_device as (
  select a.device_id, ...
  from (
    select * from dw_dwd.dwd_eng_device_active_da  -- occurrence 1
    where dt = '${date}' and device_id is not null ...
  ) a
  left join (
    select device_id, ...
    from dw_dwd.dwd_eng_device_active_da            -- occurrence 2
    where dt = '${date}' and device_id is not null ...
  ) t1 on a.device_id = t1.device_id
)

The OccurrenceCycler correctly shows ◀ 1/2 ▶, but the FILTERS section listed 8 predicates instead of 4.

Fix (this commit)

  • Added deduplicateFilters() in graphBuilders.ts and graphBuilder.worker.ts to collapse node.filters to unique expressions (preserving insertion order) before populating TableNodeData.
  • Added per-occurrence filter storage (occurrenceFilters[][] in node metadata via mergeNodesForNavigation) for the cross-statement multi-occurrence path, so TableNode can display per-occurrence filters when cycling occurrences.
  • TableNodeData gains occurrenceFilters?: FilterPredicate[][].
  • TableNode subscribes to focusedOccurrenceIndex and picks the appropriate occurrence's filters when occurrenceFilters is populated.

Files Changed

  • packages/react/src/utils/graphBuilders.ts
  • packages/react/src/workers/graphBuilder.worker.ts
  • packages/react/src/utils/nodeOccurrences.ts
  • packages/react/src/components/TableNode.tsx
  • packages/react/src/types.ts

Activity

  1. cooker-code commented on May 18, 2026

    @cooker-code
    Author

    Screenshot — Bug Reproduction (before fix)

    The dwd_eng_device_active_da table is referenced twice in the same SQL statement (two subqueries in one CTE). The FILTERS section displays 8 entries instead of 4 because the backend appends each occurrence's predicates without deduplication:

    FILTERS
    dt = '${date}'
    device_id IS NOT NULL
    device_id <> ''
    device_id <> '00000000-0000-0000-0000-000000000000'
    dt = '${date}'                                        ← duplicate from 2nd occurrence
    device_id IS NOT NULL                                 ← duplicate
    device_id <> ''                                       ← duplicate
    device_id <> '00000000-0000-0000-0000-000000000000'   ← duplicate
    

    The OccurrenceCycler correctly shows ◀ 1/2 ▶ (two occurrences), but the FILTERS section was accumulating predicates from both references.

    After fix: FILTERS correctly shows 4 unique predicates.

  2. cooker-code commented on May 18, 2026

    @cooker-code
    Author

    Fix has been implemented and merged in cooker-code/flowscope#2. The deduplication is applied at the buildTableNodeData layer in both graphBuilders.ts and graphBuilder.worker.ts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions