Skip to content

feat: source Debian 13 DocumentDB extension packages from PGDG - #456

Merged
guanzhousongmicrosoft merged 1 commit into
documentdb:mainfrom
guanzhousongmicrosoft:guanzhou/documentdb-deb13-from-pgdg
Sep 11, 2026
Merged

guanzhousongmicrosoft merged 1 commit into
documentdb:mainfrom
guanzhousongmicrosoft:guanzhou/documentdb-deb13-from-pgdg

Conversation

@guanzhousongmicrosoft

@guanzhousongmicrosoft guanzhousongmicrosoft commented Sep 10, 2026

Copy link
Copy Markdown
Member

Summary

Upstream stopped publishing Debian 13 .deb release assets after 0.115, which broke the database
image build's package download. #455
solves that by building the extension from pinned upstream source on native amd64/arm64 runners.

This PR takes the other option: PGDG already builds and publishes these packages.
apt.postgresql.org ships postgresql-18-documentdb for trixie (Debian 13), for both
architectures, for every released DocumentDB version — including 0.116-0, and 0.113-0 /
0.114-0 behind it. So there is nothing to build.

I diffed the PGDG package against the upstream 0.113-0 release asset it replaces. It is a strict
superset: byte-identical documentdb.control (default_version = '0.113-0', same requires), same
pg_documentdb.so / pg_documentdb_core.so layout, plus the distributed extension
(pg_documentdb_distributed) and Debian packaging metadata. The one behavioural difference is an
extra dependency, postgresql-18-rum, which is in trixie-pgdg and resolves automatically.

Built image size is unchanged (280MB, measured against the same base for 0.113-0 from both
sources).

Changes

.github/workflows/build_documentdb_images.yml

  • resolve postgresql-18-documentdb in the trixie-pgdg APT index, pinning a single package
    version and its SHA256 for both architectures so the multi-arch manifest cannot mix builds

  • fail fast, before any image build, if the requested version is not published for both arches

  • download from the pinned pool URL and validate: checksum, package name, version, architecture,
    and the default_version declared in documentdb.control

    PGDG appends a packaging revision to the upstream version (0.116-0 becomes
    0.116-0-1.pgdg13+1), so the Debian version string alone is not a reliable check, and a URL
    cannot simply be constructed from the release version — hence resolving the index.

  • replace the documentdb_extension_github_repo input with documentdb_apt_base_url /
    documentdb_apt_suite, so a fork can point at its own APT repository

.github/dockerfiles/Dockerfile_extension

  • install the .deb with apt-get install rather than dpkg -i, so the package's own
    dependencies resolve (postgresql-18-rum), then assert the installed version matches the file

    This keeps the existing DEB_PACKAGE_REL_PATH contract, so the source-built packages that
    test-build-and-package.yml feeds into this Dockerfile for E2E keep working unchanged.

Docs — CHANGELOG, docs/designs/image-management.md, docs/developer-guides/testing-with-fork-images.md, AGENTS.md.

Trade-off vs #455

#455 (build from source) this PR (PGDG)
Extension provenance this repo's CI, from a pinned upstream commit PGDG, GPG-signed, same builders as every other postgresql-18-* package already in the image
Runs on unreleased/forked DocumentDB source yes no — needs a published package
CI to maintain source checkout, version assertion, build_packages.sh invocation with feature detection index resolution + download
Build time ~5 min ~3 min

The thing this PR gives up is building the extension from arbitrary source. That is worth naming:
if the release pipeline ever needs to ship an extension build that PGDG does not carry, #455's
approach is the one that works. For shipping released versions — which is what this workflow does —
PGDG removes the need to own an extension build at all.

Validation

  • actionlint clean (no new findings against main's three pre-existing SC2086 infos)

  • resolver exercised against the live PGDG index for 0.113-0 and 0.116-0 (both resolve, both
    arches agree) and for a nonexistent 0.199-0 (fails with an actionable message)

  • full workflow run on a fork for 0.116.0 — all 8 jobs green, including cosign sign + verify:
    https://github.com/guanzhousongmicrosoft/documentdb-kubernetes-operator/actions/runs/34511217725

    Both arches resolved 0.116-0-1.pgdg13+1, checksums verified, and the build logs show
    Setting up postgresql-18-rumSetting up postgresql-18-documentdb (0.116-0-1.pgdg13+1).

  • local Dockerfile_extension builds from the PGDG package and from the upstream 0.113-0 release
    asset both succeed, confirming the apt-get install change is backward compatible with the
    source-built packages used by E2E

  • runtime smoke test against the candidate image built by that run: extracted the published
    documentdb:0.116.0-build-34511217725-1-8722a1b extension image, mounted it into
    ghcr.io/cloudnative-pg/postgresql:18.4-minimal-trixie the way CNPG's ImageVolume mode does
    (extension_control_path / dynamic_library_path / LD_LIBRARY_PATH), and:

    CREATE EXTENSION documentdb CASCADE;
     -- documentdb 0.116-0, documentdb_core 0.116-0, pg_cron 1.6,
     -- postgis 3.6.4, tsm_system_rows 1.0, vector 0.8.6
    SELECT documentdb_api.insert_one('smokedb','coll','{"a":1,"name":"pgdg"}');  -- ok
    SELECT document FROM documentdb_api.collection('smokedb','coll');            -- returns the doc
    

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The workflow relies on Packages.gz without verifying signed APT repository metadata, undermining the intended PGDG GPG-signing/provenance guarantees for the pinned checksum.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR updates the database image build pipeline to source Debian 13 / PostgreSQL 18 DocumentDB extension packages from the PGDG APT repository, avoiding the need to build .deb assets from upstream sources, and aligns repository documentation with that new provenance.

Changes:

  • Updated the DocumentDB image build workflow to resolve, pin, download, and validate postgresql-18-documentdb from PGDG (trixie-pgdg) for both amd64/arm64.
  • Updated the extension image Dockerfile to install the .deb via apt-get (so dependencies like postgresql-18-rum resolve) and assert the installed version matches the file.
  • Updated docs (design docs, fork-testing guide, changelog, AGENTS) to reflect PGDG as the extension package source.
File summaries
File Description
.github/workflows/build_documentdb_images.yml Resolves and pins the PGDG package per-arch, validates checksum/package metadata/control default_version, then builds/signs multi-arch images.
.github/dockerfiles/Dockerfile_extension Switches .deb installation to apt-get install to resolve dependencies and verifies installed version matches the .deb.
docs/developer-guides/testing-with-fork-images.md Updates fork-testing instructions to reflect PGDG-based extension resolution and new override knobs.
docs/designs/image-management.md Updates image provenance/design documentation to describe PGDG sourcing and the new workflow steps.
CHANGELOG.md Records the PGDG sourcing change and the new pin/validation behavior.
AGENTS.md Updates repo briefing to reflect that extension packages come from PGDG (trixie-pgdg).
Review details
  • Files reviewed: 6/6 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread .github/workflows/build_documentdb_images.yml
Upstream stopped publishing Debian 13 .deb release assets after 0.115, so
the database image build could no longer download the extension package it
needs. Rather than build the extension from source, consume the packages
PGDG already builds: apt.postgresql.org publishes postgresql-18-documentdb
for trixie on both amd64 and arm64, for every released DocumentDB version.

The PGDG package is a superset of the upstream release asset (same
documentdb.control and default_version, plus the distributed extension) and
carries one extra dependency, postgresql-18-rum.

- resolve postgresql-18-documentdb in the trixie-pgdg APT index, pinning a
  single package version and its SHA256 for both architectures so the
  multi-arch manifest cannot mix builds
- fail fast, before any image build, when the requested version is not
  published for both architectures
- validate the downloaded .deb: checksum, package name, version,
  architecture, and the default_version declared in documentdb.control
  (PGDG appends a packaging revision, so the Debian version alone is not a
  reliable check)
- install the .deb with apt rather than dpkg -i so the package's own
  dependencies resolve, and assert the installed version afterwards
- make the APT repository overridable via documentdb_apt_base_url and
  documentdb_apt_suite, replacing documentdb_extension_github_repo
- update the changelog, image-management design, fork-testing guide, and
  AGENTS.md
@guanzhousongmicrosoft
guanzhousongmicrosoft force-pushed the guanzhou/documentdb-deb13-from-pgdg branch from 8722a1b to 5a0925d Compare September 10, 2026 18:04
@documentdb-triage-tool documentdb-triage-tool Bot added CI/CD documentation Improvements or additions to documentation enhancement New feature or request labels Sep 10, 2026
@documentdb-triage-tool

Copy link
Copy Markdown

🤖 Auto-triaged by documentdb-triage-tool.

Applied: documentation, CI/CD, enhancement
Project fields suggested: Component docs · Priority P1 · Effort L · Status Needs Review
Confidence: 0.88 (mixed)

Reasoning

component from path globs (docs, ci); effort from diff stats (170+46 LOC, 6 files); LLM: Fixes a broken database image build by sourcing Debian 13 DocumentDB packages from PGDG instead of the defunct upstream release assets, unblocking multi-arch image publishing.

If a label is wrong, remove it manually and ping @patty-chow so the rules can be tuned. The bot will not re-label items that already have component labels.

@guanzhousongmicrosoft
guanzhousongmicrosoft merged commit af470e9 into documentdb:main Sep 11, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI/CD documentation Improvements or additions to documentation enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants