Skip to content

fix(csharp/src/Drivers/BigQuery): harden metadata query identifier and pattern handling - #4757

Open
CurtHagenlocher wants to merge 1 commit into
apache:mainfrom
CurtHagenlocher:fix/bigquery-metadata-identifier
Open

fix(csharp/src/Drivers/BigQuery): harden metadata query identifier and pattern handling#4757
CurtHagenlocher wants to merge 1 commit into
apache:mainfrom
CurtHagenlocher:fix/bigquery-metadata-identifier

Conversation

@CurtHagenlocher

Copy link
Copy Markdown
Contributor

The metadata queries in GetObjects and GetTableSchema built INFORMATION_SCHEMA SQL by interpolating caller-supplied catalog, dataset, table, and column values and patterns directly into the query text, relying on a regex allowlist. That allowlist was in tension with the fact that some legitimate BigQuery identifier and pattern values (for example, table names containing Unicode letters, spaces, or the '%' wildcard) don't satisfy it.

This change tightens identifier handling and removes the interpolation:

  • Every caller-supplied value across all six INFORMATION_SCHEMA queries is now bound as a query parameter rather than interpolated into the SQL text.
  • The catalog and dataset that name the view can't be parameters (BigQuery doesn't support binding identifiers), so those are still validated against an allowlist, now anchored with \A/\z instead of ^/$ (since .NET's $ also matches before a trailing newline). Rejected values are no longer echoed into the exception message, since a rejected value could itself contain characters - including newlines and other control characters - that might forge or split log/error output if interpolated there.
  • Search patterns (used for catalog, dataset, table, and column name filters) are now handled correctly: BigQuery's LIKE evaluates '%' and '_' as wildcards once the pattern is bound as a parameter, whereas the old allowlist rejected any pattern containing '%', and separately rejected Unicode letters/spaces that BigQuery permits in identifiers.
  • Catalog and dataset patterns, which are matched client-side via PatternToRegEx, are aligned with the table/column patterns, which are matched server-side via LIKE: every literal character is now escaped (previously '.' matched any character and '[' threw), and matching is now case-sensitive in both paths, matching BigQuery's own case-sensitive dataset and table names.
  • EscapeLikePattern doubles backslashes so that user-supplied '' characters remain literal under BigQuery's LIKE escaping, per the ADBC search pattern semantics (which give special meaning only to '%' and '_', with no other escaping supported).

Breaking changes, all of them cases that were previously incorrect:

  • Catalog and dataset patterns now match case-sensitively.
  • Regex metacharacters in a catalog or dataset pattern are literal.
  • A catalog or dataset containing a character outside [a-zA-Z0-9_-] is rejected rather than truncated past the first invalid character.

…d pattern handling

The metadata queries in GetObjects and GetTableSchema built INFORMATION_SCHEMA
SQL by interpolating caller-supplied catalog, dataset, table, and column
values and patterns directly into the query text, relying on a regex
allowlist (^[a-zA-Z0-9_-]+, unanchored) to filter input. That allowlist did
not fully constrain the input, and was also in tension with the fact that
some legitimate BigQuery identifier and pattern values (for example, table
names containing Unicode letters, spaces, or the '%' wildcard) don't
satisfy it.

This change tightens identifier handling and removes the interpolation:

- Every caller-supplied value across all six INFORMATION_SCHEMA queries is
  now bound as a query parameter rather than interpolated into the SQL
  text.
- The catalog and dataset that name the view can't be parameters (BigQuery
  doesn't support binding identifiers), so those are still validated
  against an allowlist, now anchored with \A/\z instead of ^/$ (since
  .NET's $ also matches before a trailing newline). Rejected values are no
  longer echoed into the exception message, since a rejected value could
  itself contain characters - including newlines and other control
  characters - that might forge or split log/error output if interpolated
  there.
- Search patterns (used for catalog, dataset, table, and column name
  filters) are now handled correctly: BigQuery's LIKE evaluates '%' and
  '_' as wildcards once the pattern is bound as a parameter, whereas the
  old allowlist rejected any pattern containing '%', and separately
  rejected Unicode letters/spaces that BigQuery permits in identifiers.
- Catalog and dataset patterns, which are matched client-side via
  PatternToRegEx, are aligned with the table/column patterns, which are
  matched server-side via LIKE: every literal character is now escaped
  (previously '.' matched any character and '[' threw), and matching is
  now case-sensitive in both paths, matching BigQuery's own case-sensitive
  dataset and table names.
- EscapeLikePattern doubles backslashes so that user-supplied '\'
  characters remain literal under BigQuery's LIKE escaping, per the ADBC
  search pattern semantics (which give special meaning only to '%' and
  '_', with no other escaping supported).

Breaking changes, all of them cases that were previously incorrect:

- Catalog and dataset patterns now match case-sensitively.
- Regex metacharacters in a catalog or dataset pattern are literal.
- A catalog or dataset containing a character outside [a-zA-Z0-9_-] is
  rejected rather than truncated past the first invalid character.

This mirrors a fix from CurtHagenlocher/bigquery#297, which found and
fixed the same issue in a downstream fork of this driver.

Co-Authored-By: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 02844248-1442-4b59-9cc7-fffc21b698fa
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant