Skip to content

Fix DuckLake worker catalog loading without metadata password files - #38

Closed
bill-ph wants to merge 1 commit into
masterfrom
codex/ducklake-worker-password-validation
Closed

Fix DuckLake worker catalog loading without metadata password files#38
bill-ph wants to merge 1 commit into
masterfrom
codex/ducklake-worker-password-validation

Conversation

@bill-ph

@bill-ph bill-ph commented Sep 12, 2026

Copy link
Copy Markdown

Description

A DuckLake catalog can be created successfully on the coordinator while a worker's Kubernetes Secret volume has not yet received the metadata password file. The first distributed query then fails with HTTP 500 during worker catalog initialization, even though workers do not access the metadata database.

Validate password-file existence only on coordinators. Preserve the missing-file error text used by the provisioner's pending-state classifier, password-source mutual exclusion, and file reads on every new metadata connection.

Additional context and related issues

Addresses the separate worker catalog-loading failure exposed by PostHog/duckgres#1181. Duckgres needs to adopt an image containing this change before its E2E lane can exercise the fix.

The new distributed regression reproduced the original HTTP 500 before the change. With the fix, real INSERT and SELECT statements succeed both on the initial worker and on a replacement worker, with coordinator task scheduling disabled and the worker password file absent throughout. Additional checks cover coordinator rejection, password-source mutual exclusion, and recovery after deleting and restoring the coordinator password file.

Validation:

  • TZ=UTC ./mvnw -pl plugin/trino-ducklake test: 197 checks passed, no failures or skips. UTC is needed by two existing native DuckDB timestamp-string assertions.
  • ./mvnw -pl plugin/trino-ducklake -am validate: passed, including formatting and checkstyle across the connector and its dependencies.

Release notes

(x) Release notes are required, with the following suggested text:

## DuckLake connector
* Fix distributed queries failing when a worker has not received the metadata password file. Only coordinators require this file.

@github-actions github-actions Bot added the docs label Sep 12, 2026
@bill-ph bill-ph closed this Sep 12, 2026
@bill-ph
bill-ph deleted the codex/ducklake-worker-password-validation branch September 12, 2026 02:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant