Repository navigation
feat: accept use_materialization_v2 as a model config - #1718
Merged
Merged
Conversation
sd-db
marked this pull request as ready for review
October 13, 2026 22:00
Coverage reportClick to see where and how coverage changed
This report was generated by python-coverage-comment-action |
||||||||||||||||||||||||
jprakash-db
approved these changes
Oct 19, 2026
tauhid621
added a commit
to dbt-labs/dbt
that referenced
this pull request
Oct 26, 2026
…16879) (#16892) Backport only the use_materialization_v2 entries from the dbt-fusion schema sync in #16879: +use_materialization_v2 in ProjectModelConfig and use_materialization_v2 in the model, seed, snapshot, source, data test, unit test, and function configs. The blocks match #16879 byte for byte. dbt-databricks is making use_materialization_v2 a per-model config (databricks/dbt-databricks#1718). Without these entries, Databricks projects that set it get a spurious CustomKeyInConfigDeprecation warning. The rest of #16879 is left out. It does not cherry-pick cleanly here because earlier syncs (#16421, #16525) were never backported, and the newer schema also drops or changes existing keys, which could cause new warnings in a patch release. This change only adds keys. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
tauhid621
added a commit
to dbt-labs/dbt
that referenced
this pull request
Oct 26, 2026
…16879) (#16893) Backport only the use_materialization_v2 entries from the dbt-fusion schema sync in #16879: +use_materialization_v2 in ProjectModelConfig and use_materialization_v2 in the model, seed, snapshot, source, data test, unit test, and function configs. The blocks match #16879 byte for byte. dbt-databricks is making use_materialization_v2 a per-model config (databricks/dbt-databricks#1718). Without these entries, Databricks projects that set it get a spurious CustomKeyInConfigDeprecation warning. The rest of #16879 is left out. It does not cherry-pick cleanly here because earlier syncs (#16421, #16525) were never backported, and the newer schema also drops or changes existing keys, which could cause new warnings in a patch release. This change only adds keys. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Let a project or an individual table, incremental, or view model choose its materialization path. A new use_materialization_v2() macro returns the model's use_materialization_v2 config when set, coercing quoted string values, and otherwise falls back to the behavior flag. The three materializations branch on it instead of reading the flag directly. Co-authored-by: Isaac <no-reply@databricks.com>
The path switch is a macro-only change, and both materialization paths are already covered by the existing V1 and V2 functional suites, so drop the dedicated functional tests and keep the resolver unit tests. Link the changelog entry to its PR. Co-authored-by: Isaac <no-reply@databricks.com>
…mental View materialization is being unified into a single implementation for 1.13.0, so the model config no longer selects a View path. Restore view.sql to read only the behavior flag, and scope the changelog and flow docs to table and incremental models, describing the CTAS versus create-then-insert tradeoff. Co-authored-by: Isaac <no-reply@databricks.com>
sd-db
force-pushed
the
sd-db/use-materialization-v2-model-config
branch
from
October 28, 2026 06:52
f1452c3 to
1226cd4
Compare
Co-authored-by: Isaac <no-reply@databricks.com>
…onfig" This reverts commit cbffe80. Co-authored-by: Isaac <no-reply@databricks.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
use_materialization_v2can only be set project-wide today, as a behavior flag underflags:. For table and incremental models, it chooses between two materialization paths with a real tradeoff:CREATE OR REPLACE TABLE … AS SELECT. Readers never see an empty table, but column masks can't be applied at create time.A project that needs create-time masks on a few sensitive tables has to give up atomic refresh everywhere, or the other way round. This makes
use_materialization_v2a model config for table and incremental models, matching the dbt Fusion engine change in dbt-labs/dbt#16661:DatabricksConfig.use_materialization_v2: Optional[bool]. It can be set in{{ config(...) }}, in properties YAML, or per folder with+use_materialization_v2indbt_project.yml, and inherits like other model configs.use_materialization_v2()macro returns the model's setting when set. If the setting is unset ornull, it returns theuse_materialization_v2behavior flag."false"arrive in Jinja as strings, and'false'is truthy. The macro compares them case-insensitively totrue, as Fusion does when it parses the config.tableandincrementalmaterializations branch on the macro instead of reading the flag directly.Views use one shared implementation in 1.13.0 and ignore this config. That unification is separate work, and this PR doesn't change
view.sql.view_update_via_alterstays a separate opt-in setting. Seeds, snapshots, materialized views, streaming tables, metric views, and clones don't use the flag and are unchanged.Projects that don't set the model config behave exactly as before.
Until dbt Core picks up the updated config schema from Fusion, dbt Core may warn that
use_materialization_v2is an unrecognized config key. The setting still takes effect.Testing
tests/unit/macros/materializations/test_use_materialization_v2.py): an unset ornullsetting follows the flag both ways.true/falseand the strings"true"/"False"override it. This is the regression coverage for the change. Both table and incremental paths are already covered by the existing V1 and V2 functional suites.+use_materialization_v2: true, table and incremental models were built by V2:DESCRIBE HISTORYshowed aWRITEand no… AS SELECT. With the flag on andconfig(use_materialization_v2=false), they were built by the default path's CTAS. Models that didn't set the config followed the flag. These checks were not committed as functional tests.Checklist
CHANGELOG.mdand added information about my change to the "dbt-databricks next" section.This pull request and its description were written by Isaac.