Repository navigation
fix: use CREATE OR REPLACE for view-to-view replaces in safer mode - #1733
Open
shantanutheone wants to merge 1 commit into
Open
shantanutheone wants to merge 1 commit into
shantanutheone wants to merge 1 commit into
Conversation
shantanutheone
requested review from
jprakash-db,
saishreeeee and
sd-db
as code owners
October 17, 2026 21:14
With use_safer_relation_operations on, a view replacing a view went through safely_replace, which renames the old view to a __dbt_backup name first. As reported in databricks#1714, Databricks resolves the old view's query again on that rename, so if an upstream model in the same run dropped or renamed a column the old view selects, the rename fails with UNRESOLVED_COLUMN. That is the usual reason a view gets rebuilt, and every retry fails the same way until the view is dropped by hand. CREATE OR REPLACE VIEW is atomic and a view holds no data, so the backup protects nothing. View-to-view replaces now use it whether or not safer relation operations are enabled. Tables, materialized views and type changes keep their current paths. Resolves databricks#1714 Signed-off-by: Shantanu <shaantanutripathi@gmail.com>
shantanutheone
force-pushed
the
fix-safer-view-replace-rename
branch
from
October 17, 2026 21:15
27f03f2 to
8ac8f0a
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resolves #1714
Description
With
use_materialization_v2anduse_safer_relation_operationson, a view model replacing an existing view goes throughsafely_replace, which renames the old view to__dbt_backupbefore swapping in the staged one. Per the error in #1714, that rename fails withUNRESOLVED_COLUMNwhen an upstream model in the same run dropped or renamed a column the old view selects, and every retry fails the same way until the view is dropped by hand.A view holds no data and
CREATE OR REPLACE VIEWis atomic (if the new SQL is invalid the old view stays as it was), so the backup doesn't buy anything for a view-to-view replace. This PR takes the first option suggested in the issue: indatabricks__get_replace_sql, a replaceable view replacing a view now usesget_replace_view_sqlwhether or not safer relation operations are on. Tables, materialized views and type changes (table to view etc.) keep their current paths.Changes:
macros/relations/replace.sql: move the view-to-viewCREATE OR REPLACEbranch above thesafe_replacechecktest_view_to_view_uses_create_or_replacenow covers safer mode too (UC and HMS), and I removedtest_hive_metastore_view_to_view_with_safe_replace_backs_up_view, which asserted the old backup behaviorTestSaferOpsReplacesViewAfterUpstreamColumnRenamebuilds a table and a view on it, renames the column in both, runs again and checks the view returns the new columndocs/flow/replace_flow.mdupdated to matchTesting:
hatch run unitpasses (1466 passed, 4 skipped), andpre-commitpasses on the changed files/integration-test.Checklist
CHANGELOG.mdand added information about my change to the "dbt-databricks next" section.dbt-databricks-pr-readyproject skill for this PR and addressed its merge-readiness feedback