Skip to content

fix: escape relation string literals in metadata queries (#1747) - #1758

Open
MsMansiDhruv wants to merge 1 commit into
databricks:mainfrom
MsMansiDhruv:fix/escape-relation-string-literals
Open

MsMansiDhruv wants to merge 1 commit into
databricks:mainfrom
MsMansiDhruv:fix/escape-relation-string-literals

Conversation

@MsMansiDhruv

Copy link
Copy Markdown

Resolves #1747

Description

Several metadata macros interpolate relation components (catalog, schema, table) directly into SQL string literals (= '{{ relation.<part>|lower }}') without escaping special characters. In Databricks Unity Catalog, relation identifiers can contain quotes and backslashes when backtick-quoted (e.g. `orders'eu` or `cat\log`). For such relations, the unescaped string literals produce invalid SQL syntax and cause query execution errors when querying information_schema views or running metadata-dependent operations (such as table tags, row filters, column masks, constraints, and catalog generation).

This PR:

  1. Adds a shared helper macro escape_relation_literal(value) in macros/relations/escape.sql that escapes backslashes first (\ -> \\) and then single quotes (' -> \'), matching the comment escaping convention used in macros/relations/components/comment.sql.
  2. Updates all metadata macros that build string literal predicates on relation names:
    • macros/relations/tags.sql: fetch_tags_sql
    • macros/relations/components/row_filter.sql: fetch_row_filters_sql
    • macros/relations/components/column_mask.sql: fetch_column_masks_sql
    • macros/relations/components/column_tags.sql: fetch_column_tags_sql
    • macros/relations/components/constraints.sql: fetch_non_null_constraint_columns_sql, fetch_primary_key_constraints_sql, fetch_foreign_key_constraints_sql
    • macros/adapters/catalog.sql: databricks__get_catalog_schemas_where_clause_sql, databricks__get_catalog_relations_where_clause_sql
    • macros/adapters/metadata.sql: get_relation_last_modified_sql, get_view_description_sql, get_uc_tables_sql
  3. Adds unit tests for escape_relation_literal as well as tests covering relation names with single quotes (') and backslashes (\) for each modified macro test suite.
  4. Updates CHANGELOG.md.

Checklist

  • I have run this code in development and it appears to resolve the stated issue
  • This PR includes tests, or tests are not required/relevant for this PR
  • I have updated the CHANGELOG.md and added information about my change to the "dbt-databricks next" section.

@sd-db

sd-db commented Oct 29, 2026

Copy link
Copy Markdown
Collaborator

@MsMansiDhruv can you look at the failing test for code quality. Thx !

@MsMansiDhruv
MsMansiDhruv force-pushed the fix/escape-relation-string-literals branch from b93993e to 5257e50 Compare October 30, 2026 03:11
@MsMansiDhruv

Copy link
Copy Markdown
Author

@MsMansiDhruv can you look at the failing test for code quality. Thx !

Hey @sd-db , I applied ruff format across the new unit test files to resolve the failing test. Unit test cases run locally. Will await on the test results if something fails.

@MsMansiDhruv
MsMansiDhruv force-pushed the fix/escape-relation-string-literals branch from 5257e50 to 231dfd5 Compare October 30, 2026 07:31
@MsMansiDhruv

Copy link
Copy Markdown
Author

Hi @sd-db,
Looking into the latest Code Quality failure in CI, it seems the runner hit a cache restore timeout during Setup Python Dependencies, which triggered the fallback setup-jfrog-pypi step.
Because this PR is submitted from a fork, GitHub restricts the OIDC token (ACTIONS_ID_TOKEN_REQUEST_TOKEN), causing the setup script to exit before hatch run code-quality actually ran. All the Python 3.10–3.13 unit tests, package builds, and local ruff format --check / ruff check pass cleanly.

Could you please trigger a quick re-run on the Code Quality job from your end? Thanks!

@github-actions

Copy link
Copy Markdown

Coverage report

This PR does not seem to contain any modification to coverable code.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Metadata queries break on catalog, schema or table names containing a quote or backslash

2 participants