Repository navigation
fix: let dbt clone handle materialized views and streaming tables with row filters or column masks - #1744
Open
sd-db wants to merge 7 commits into
Open
fix: let dbt clone handle materialized views and streaming tables with row filters or column masks#1744sd-db wants to merge 7 commits into
sd-db wants to merge 7 commits into
Conversation
sd-db
added a commit
that referenced
this pull request
Oct 25, 2026
Co-authored-by: Isaac <no-reply@databricks.com>
Coverage report
This PR does not seem to contain any modification to coverable code. |
sd-db
changed the base branch from
1.13.latest
to
fix/1349-clone-view-contract
October 25, 2026 20:32
This was referenced Oct 26, 2026
saishreeeee
approved these changes
Oct 27, 2026
sd-db
added a commit
to sd-db/dbt-core
that referenced
this pull request
Oct 28, 2026
…g tables with row filters or column masks Port of databricks/dbt-databricks#1744. dbt clone builds a view over a deferred materialized view or streaming table, and that object keeps enforcing its own row filter and column masks. Under clone, validate_view_model now skips the mask and row-filter rejection for materialized_view and streaming_table models. Plain views still reject both. Co-authored-by: Isaac <no-reply@databricks.com>
Clone cannot shallow-clone a materialized view or streaming table, so it creates a view over the deferred object. validate_view_model rejected the model's row_filter/column_mask config there because views can't carry policies, so clone failed even when nothing changed. Under clone, skip that rejection for MV/ST models; the deferred object enforces its own policies. Plain view models still reject masks and row filters under clone. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
sd-db
force-pushed
the
fix/clone-view-skip-mask-filter-checks
branch
from
October 28, 2026 06:50
15ad66e to
9f0bc9c
Compare
Co-authored-by: Isaac <no-reply@databricks.com>
sd-db
added a commit
that referenced
this pull request
Oct 28, 2026
Co-authored-by: Isaac <no-reply@databricks.com>
### Description `TestViewLifecycleIgnoresMaterializationFlag` and `TestViewLifecycleMaterializationFlagOn` ran `dbt --warn-error run`, so any warning failed them, including warnings unrelated to the view lifecycle. On the lowest-direct dependency floor, dbt-core 1.11.2 reports `view_update_via_alter` as an unrecognized config key (`CustomKeyInConfigDeprecation`), and some auth paths emit warnings too. The `--warn-error` was only defense in depth, so this PR removes it. All server-state assertions (column comments and grants) stay. This PR also moves the 1.13.0 release date in `CHANGELOG.md` to Oct 29, 2026. Supersedes #1754. Verified locally against a SQL warehouse (`databricks_uc_sql_endpoint`), `tests/functional/adapter/views/test_views.py`: - dbt-core 1.11.2 (lowest-direct lock): 29 passed, 7 skipped - dbt-core 1.12.6 (`uv.lock`): 29 passed, 7 skipped ### Checklist - [x] I have run this code in development and it appears to resolve the stated issue - [x] This PR includes tests, or tests are not required/relevant for this PR - [ ] I have updated the `CHANGELOG.md` and added information about my change to the "dbt-databricks next" section. (test-only, no changelog entry) This pull request and its description were written by Isaac. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
This reverts commit 5ff4c36. Co-authored-by: Isaac <no-reply@databricks.com>
1 of 3 tasks
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
dbt clonefails for a materialized view with arow_filter, and for a streaming table with arow_filterorcolumn_mask. This happens even when nothing changed since the deferred state:Clone can't shallow-clone an MV or ST, so it builds a view over the deferred object (
select * from <prod MV/ST>) through the view materialization.validate_view_modelthen rejects the model's own mask or filter config, because views can't carry those policies. The cloned view doesn't need them: the deferred MV or ST enforces its own policies when the view reads from it.Under
clone,validate_view_modelnow skips the mask and row-filter rejection formaterialized_viewandstreaming_tablemodels. Plain view models still reject masks and row filters under clone, so a policy declared on a view is never silently dropped. The contract skip from #1742 is unchanged.The cloned view reflects the production object's policies, not the branch's config. That includes the case where the branch changed the model's materialization.
docs/flow/view_flow.mddocuments this.Tables with masks or row filters still fail, because Databricks refuses to shallow-clone them (
ROW_LEVEL_SECURITY_TABLE_CLONE_SOURCE_NOT_SUPPORTED/COLUMN_MASKS_TABLE_CLONE_SOURCE_NOT_SUPPORTED). They're out of scope here: falling back to a view for those needs separate verification of how policies apply to other users reading the view.Tests:
tests/functional/adapter/dbt_clone/:TestCloneRowFilteredMaterializedView,TestCloneRowFilteredStreamingTableandTestCloneColumnMaskedStreamingTable. Each builds the governed object, clones it, and asserts that the target is aVIEWwhose rows are filtered or masked by the production object's policy. With the fix reverted, all three fail with the errors above.run/view,clone/view,clone/materialized view andclone/streaming table.TestUpdateViewViaAlterValidation, bothTestViewConstraintsColumnsEqualclasses and the row-filter and column-mask suites on the UC SQL warehouse: 45 passed, 2 skipped. All macro unit tests pass (347).Checklist
CHANGELOG.mdand added information about my change to the "dbt-databricks next" section.dbt-databricks-pr-readyproject skill for this PR and addressed its merge-readiness feedbackThis pull request and its description were written by Isaac.