build: mirror postgres_scanner instead of pinning a nightly - #1179
Merged
Conversation
Test Impact PlanDeterministic summary of how this PR changes tests, CI runners, and coverage-risk signals. Summary
Signals
Coverage risk: neutral or increased No coverage-reduction warnings detected. |
The stable DuckDB 1.5.5 postgres_scanner predates duckdb-postgres 71b85668, which fixes inconsistent snapshots across scan connections, so we take a nightly build. The nightly URL is mutable, and pinning a content hash against it has hard-failed the build five times between 2026-08-03 and 2026-09-11. Each break needed a human to re-pin, and warm Docker layer caches hid it until an unrelated branch built cold. Mirror the artifact instead, which the matrix comment already named as the real fix. PostHog/duckdb-postgres now holds the nightly captured at a3516c0, a descendant of 71b85668, under an immutable release tag. This makes postgres_scanner identical in shape to httpfs and ducklake, which already come from tagged PostHog releases: one tag per extension, no repository URL and no per-arch checksums to carry through five files. The checksums are recorded in the release notes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WCY5Jf2BQPCVKJTZU1TpEe
fuziontech
force-pushed
the
james/mirror-postgres-scanner
branch
from
September 11, 2026 22:52
5734ef4 to
e5cf728
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
postgres_scannernightly to PostHog/duckdb-postgresGoals
71b85668that made us take a nightly in the first placeNon-goals
41223e5, which predates71b85668, so it reintroduces the bug build: pin postgres scanner snapshot fix #1033 avoided.Why
The nightly URL is mutable, so a content pin against it is a treadmill. It has hard-failed the build five times since 2026-08-03 (
afc9a0f7→7ff4913f→574bd831→fdeb8a14, and now again), each needing a human to re-pin. Warm Docker layer caches hide the break until some unrelated branch builds cold, which is how it surfaced this time.The matrix comment already named the fix:
That is this PR. The mirror holds the nightly captured at duckdb-postgres
a3516c0;gh api .../compare/71b85668...a3516c0reports ahead_by 8, behind_by 0, so the required snapshot fix is present.Because a release asset URL does not move, the repository URL and both checksums stop earning their keep. That makes
postgres_scanneridentical in shape tohttpfsandducklake, which already come from tagged PostHog releases: one tag per extension instead of three values duplicated across five files. The checksums are recorded in the release notes.Production risk
Low, but this does change what the worker image bundles. The bundled scanner moves from revision
4053b61toa3516c0; both descend from71b85668. The build fails closed if the tag is missing (curl -fsSL404 writes nothing, and the[ -s ... ]check rejects an empty file).The mirror is a new dependency on a PostHog-owned repo. It is the same trust we already place in
PostHog/duckdb-httpfsandPostHog/ducklake.Start here
Dockerfile— the scanner fetch now sits beside httpfs and ducklake and looks the same..github/workflows/container-image-worker-cd.yml—pg_scanner_tagreplaces three matrix values.Dockerfile.workerand the other two workflows are the same substitution.Test plan
docker build --target builder -f Dockerfile .on arm64: the extension step fetches all four extensions and passes the size checkpostgres_scanner.duckdb_extensionin the built image reportsa3516c0/v1.5.5/linux_arm64and hashes tob2e464b1..., byte-identical to the mirrored assetNote
Needs #1178 (MinIO moved to quay.io) for the container-based lanes to go green; the two are independent and can land in either order.
🤖 Generated with Claude Code
https://claude.ai/code/session_01WCY5Jf2BQPCVKJTZU1TpEe