Skip to content

deployment-scoped artifacts and reporting epic - #551

Open
dannash100 wants to merge 30 commits into
mainfrom
epic/deployment-artefacts
Open

dannash100 wants to merge 30 commits into
mainfrom
epic/deployment-artefacts

Conversation

@dannash100

@dannash100 dannash100 commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Epic branch for deployment-scoped artifacts. Implementation PRs merge here rather than into main.

  • Merges feat(platform): D2: spec deployment-scoped artefacts #529's spec onto current main. That branch sat 247 commits behind, and two of its links pointed at files main has since renamed (device-identity.md, servers/products.md).
  • Settles the review points still open on it: the digest is verified as the bytes arrive as well as when they are served, and artefact is spelled artifact to match the code.
  • First implementation branch is feat/group-scoped-artifacts.

🦸 Review Hero

  • Run Review Hero

@dannash100 dannash100 changed the title feat(platform): D2: deployment-scoped artifacts feat(platform): D2: deployment-scoped artifacts and reporting Sep 6, 2026
@dannash100 dannash100 changed the title feat(platform): D2: deployment-scoped artifacts and reporting deployment-scoped artifacts and reporting epic Sep 6, 2026
@dannash100

dannash100 commented Sep 7, 2026

Copy link
Copy Markdown
Contributor Author

🤖 # What the reporting-schema pipeline needs from the dbt repos

Canopy works out which pairs of group and Tamanu version need a reporting schema,
hands pgro a restored replica migrated to that version, and takes the built schema
back as a group-scoped artifact it offers that group's servers. bestool applies the
difference. The build in the middle is the piece nobody owns.

How pgro runs it

A Kubernetes Job against the freshly migrated replica, from an image canopy names per
group (builder_image on the worklist). That image is the deliverable, and nothing
fills the field on either side yet. One attempt, no retry, 30 minute deadline, no
service account token.

Variable What it is
TAMANU_DL_DB_URL Host of the restored replica
TAMANU_DL_DB_USER, TAMANU_DL_DB_PASSWORD, TAMANU_DL_DB_DATABASE Its credentials and database
TAMANU_VERSION The version the replica was migrated to, and the version to build for
TAMANU_DEPLOYMENT The group the snapshot came from
SCHEMA_CALLBACK_URL Where to POST the built SQL

The first four match the replica target already in each deployment repo's
config/profiles.yml.

What the build needs

  1. Version as an input. get_deployment_version()
    (scripts/utils/dbt_utils.py:82) reads it from dbt_project.yml, so a build is for
    whatever version the checkout sits at. Canopy names the version, often not the
    repo's.

  2. Deployment as an input. get_deployment_name() reads it the same way, and it
    drives behaviour: generate_reporting_schema_script
    (scripts/utils/report_utils.py:299) excludes restricted nodes unless
    has_sensitive_facility is set.

  3. No release cut. Following from those two, the build has to run at an arbitrary
    version without create_version_branch.

  4. POST to SCHEMA_CALLBACK_URL. Raw SQL body, 32 MiB ceiling. pgro cannot read a
    file out of a finished container and a Job's termination message caps at 4 KiB. The
    URL's last segment is the build's token, so keep it out of the build's logs.

  5. Stamp the version. COMMENT ON SCHEMA reporting IS '<version>'; It is parsed as
    a version, so v2.60.0 and 2.60.0 are the same stamp and anything over 64
    characters reads as unstamped. bestool will not re-apply a schema that applied
    without one, so an unstamped schema applies once and the check then sits failing.

  6. No BEGIN, COMMIT or ROLLBACK. bestool applies the SQL as one batch, which
    is what leaves a server its old schema when a statement fails partway. The generator
    emits none today, so this is a line to hold.

Separately

publish-artifacts.yml collapses v2.60.2 into v2.60.x, so a patch release
overwrites the previous bundle and leaves nothing to grade currency against. Canopy
refuses a reporting schema registered against a range for the same reason. The
deployment repos also have no version bump path: neither create-version-branch.yml
nor tamanu-upgrade.yml.

@review-hero

review-hero Bot commented Sep 8, 2026

Copy link
Copy Markdown

🦸 Review Hero Summary
9 agents reviewed this PR | 0 critical | 0 suggestions | 0 nitpicks | Filtering: consensus 3 voters

No issues found. Looks good!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant