Skip to content

Document Python BigQuery CDC record format - #40021

Open
mukeshbhandarkar wants to merge 1 commit into
apache:masterfrom
mukeshbhandarkar:docs/39768-bigquery-cdc-format
Open

Document Python BigQuery CDC record format#40021
mukeshbhandarkar wants to merge 1 commit into
apache:masterfrom
mukeshbhandarkar:docs/39768-bigquery-cdc-format

Conversation

@mukeshbhandarkar

Copy link
Copy Markdown

What does this PR do?

Documents the Python BigQuery Change Data Capture (CDC) input format for WriteToBigQuery.

The new documentation explains:

  • the required STORAGE_WRITE_API, use_at_least_once, and use_cdc_writes settings
  • primary key requirements
  • the expected row_mutation_info / record PCollection structure
  • UPSERT and DELETE mutation types
  • change_sequence_number
  • how Beam maps mutation metadata to BigQuery _CHANGE_TYPE and _CHANGE_SEQUENCE_NUMBER
  • dictionary input requirements

The use_cdc_writes Python API documentation is also clarified.

Fixes #39768

Testing

  • git diff --check
  • Python 3.12 compilation
  • Verified Hugo shortcode delimiters
  • Confirmed only the intended documentation/API docstring files changed

I did not run ./gradlew :website:testWebsite locally because it requires building the website environment and related dependencies that are not available in my local setup.

@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment assign set of reviewers

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature Request]: document BigQuery CDC record format

1 participant