Skip to content

fix: use CREATE OR REPLACE for view-to-view replaces in safer mode - #1733

Open
shantanutheone wants to merge 1 commit into
databricks:mainfrom
shantanutheone:fix-safer-view-replace-rename
Open

shantanutheone wants to merge 1 commit into
databricks:mainfrom
shantanutheone:fix-safer-view-replace-rename

Conversation

@shantanutheone

@shantanutheone shantanutheone commented Oct 17, 2026 •

Copy link
Copy Markdown

Resolves #1714

Description

With use_materialization_v2 and use_safer_relation_operations on, a view model replacing an existing view goes through safely_replace, which renames the old view to __dbt_backup before swapping in the staged one. Per the error in #1714, that rename fails with UNRESOLVED_COLUMN when an upstream model in the same run dropped or renamed a column the old view selects, and every retry fails the same way until the view is dropped by hand.

A view holds no data and CREATE OR REPLACE VIEW is atomic (if the new SQL is invalid the old view stays as it was), so the backup doesn't buy anything for a view-to-view replace. This PR takes the first option suggested in the issue: in databricks__get_replace_sql, a replaceable view replacing a view now uses get_replace_view_sql whether or not safer relation operations are on. Tables, materialized views and type changes (table to view etc.) keep their current paths.

Changes:

  • macros/relations/replace.sql: move the view-to-view CREATE OR REPLACE branch above the safe_replace check
  • unit tests: test_view_to_view_uses_create_or_replace now covers safer mode too (UC and HMS), and I removed test_hive_metastore_view_to_view_with_safe_replace_backs_up_view, which asserted the old backup behavior
  • functional test: TestSaferOpsReplacesViewAfterUpstreamColumnRename builds a table and a view on it, renames the column in both, runs again and checks the view returns the new column
  • docs/flow/replace_flow.md updated to match

Testing:

  • hatch run unit passes (1466 passed, 4 skipped), and pre-commit passes on the changed files
  • with the macro change reverted, exactly the two new safer-mode unit cases fail
  • I don't have a Databricks workspace, so I could not run the functional test or reproduce the error myself. It would need /integration-test.

Checklist

  • I have run this code in development and it appears to resolve the stated issue
  • This PR includes tests, or tests are not required/relevant for this PR
  • I have updated the CHANGELOG.md and added information about my change to the "dbt-databricks next" section.
  • [Optional] I have run the dbt-databricks-pr-ready project skill for this PR and addressed its merge-readiness feedback

With use_safer_relation_operations on, a view replacing a view went
through safely_replace, which renames the old view to a __dbt_backup
name first. As reported in databricks#1714, Databricks resolves the old view's
query again on that rename, so if an upstream model in the same run
dropped or renamed a column the old view selects, the rename fails
with UNRESOLVED_COLUMN. That is the usual reason a view gets rebuilt,
and every retry fails the same way until the view is dropped by hand.

CREATE OR REPLACE VIEW is atomic and a view holds no data, so the
backup protects nothing. View-to-view replaces now use it whether or
not safer relation operations are enabled. Tables, materialized views
and type changes keep their current paths.

Resolves databricks#1714

Signed-off-by: Shantanu <shaantanutripathi@gmail.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

use_safer_relation_operations: replacing a view fails with UNRESOLVED_COLUMN when its upstream columns changed in the same run

1 participant