diff --git a/.workhorse/specs/platform/artifacts.md b/.workhorse/specs/platform/artifacts.md index d9e6d7ed..6dd5f487 100644 --- a/.workhorse/specs/platform/artifacts.md +++ b/.workhorse/specs/platform/artifacts.md @@ -65,7 +65,9 @@ Canopy passes a group-scoped artifact's bytes only to a caller it is offered to. ## Registration -A registration names the version or range, the type, the platform, and the group where the artifact has one, and carries either the location of an unscoped artifact or the bytes of a group-scoped one. +Registering an unscoped artifact and registering a group's are separate: they name different things, carry different bodies, and are authorised differently, so each is its own path rather than one path that changes shape on a parameter. +An unscoped registration names the version or range, the type and the platform, and carries the artifact's location. +A group-scoped one names the group as well, carries the artifact's bytes, and names an exact version Canopy already holds rather than drafting one. The group is named on the registration rather than inferred from the caller. A releaser device registers unscoped artifacts, and carries no authorisation for any group. diff --git a/.workhorse/specs/public-server/reporting-schemas.md b/.workhorse/specs/public-server/reporting-schemas.md index bc6acada..ed894657 100644 --- a/.workhorse/specs/public-server/reporting-schemas.md +++ b/.workhorse/specs/public-server/reporting-schemas.md @@ -21,7 +21,7 @@ A **schema builder** produces a reporting schema from a database Canopy has rest It is a restore consumer (see [RST](restore-replicas.md)): a build operates on a replica, so the builder is dispatched, credentialled, and reports over the replica pathways and authorisations, and it advertises an intent carrying `reporting-schema`. How the builder produces a schema is the builder's own. -An **operator** declares which groups have a builder, reads which schema each application runs, and asks for the builds the derivation does not produce. +An **operator** declares which groups have a builder, marks the declaration that publishes a group's schema, reads which schema each application runs, and asks for the builds the derivation does not produce. The **device of a machine a Tamanu application runs on** fetches the schema Canopy offers that application and applies it (see [DID](machine-identity.md)). @@ -30,8 +30,8 @@ Canopy owns which pairs exist, the replica a build is given, the artifact that r ## Pairs A reporting schema is unique per pair of group and Tamanu version, and Canopy holds zero or one per pair. -The pairs are, for each group covered by an enabled declaration of a `reporting-schema` intent, each version a Tamanu application of the group reports running and the version its open plan moves it to (see [UPG](../private-server/upgrade-plans.md)). -That declaration is what covers a group: it names the group, is enabled or disabled, and is audited (see [RST](restore-replicas.md)). +The pairs are, for each group covered by an enabled declaration marked as publishing its schema, each version a Tamanu application of the group reports running and the version its open plan moves it to (see [UPG](../private-server/upgrade-plans.md)). +That declaration is what covers a group: it names the group, is enabled or disabled, carries the operator's mark, and is audited (see [RST](restore-replicas.md)). Only a published version is in a pair, since a version's migrations reach a builder as its published artifacts (see [ART](../platform/artifacts.md)) and an unpublished one has none. A pair with no schema is built, and a pair with one is settled. @@ -50,8 +50,10 @@ The replica is migrated to the named version before the build reads it, and is n The builder obtains read credentials for the restore per run as any consumer does, and no storage credential of any kind for what it publishes (see [RST](restore-replicas.md)). In the run it reports, the builder registers the **reporting schema** as an artifact of the exact version being built for, scoped to the group, of type `reporting-schema` on platform `any`, carrying a digest and the bytes themselves, which Canopy holds and serves (see [ART](../platform/artifacts.md)). -It may register further artifacts beside the schema for the same version and group, under types of its choosing, which Canopy offers as it offers any artifact. -The builder is authorised to register artifacts for a group its enabled `reporting-schema` declaration covers and for no other, and is the one device other than a releaser that registers artifacts (see [ART](../platform/artifacts.md)). +The schema is the only type it may register: what is published for a group is offered to every machine in it and fetched, so the authorisation stays defined with the artifact it was written for. +The builder is authorised to register artifacts for a group whose enabled declaration an operator has marked as publishing its reporting schema, and for no other, and is the one device other than a releaser that registers artifacts (see [ART](../platform/artifacts.md)). +The mark is the operator's alone, and is the whole of the authorisation: a consumer registers the set of semantics it advertises itself, so they shape what Canopy dispatches to it and grant it nothing, and what is published for a group is offered to every machine in it and applied. +Only a group-wide, non-redacting declaration of an intent carrying `reporting-schema` can carry the mark, which is the same declaration a build is dispatched for, so Canopy asks for no build it would refuse the result of. A schema is published for the exact version and never for a range, since it follows from the migrations that version applies, and one built against a patch is not the schema another patch of the same minor describes. @@ -80,6 +82,6 @@ The device compares what its application runs with what it is offered, applies t A failed build raises a reporting-schema check on the group's central Tamanu application, carrying the failure description (see [CHK](../monitoring/checks.md)). The check is a warning rather than a failure, and does not escalate: the application is up and its reports return rows, and a schema that cannot be built for the version its group is moving to is for whoever maintains the reports rather than whoever is on call. A replica that failed to restore or come up is the restore's own health rather than a build failure, and is dispatched again as any unhealthy restore is. -The check recovers when the pair is built, and an operator asking for the build is what clears it. +The check recovers when the pair is built, and nothing else clears it: an operator asking for the build puts the pair back on the worklist, and the warning stands until a build lands, since asking changes nothing about whether the group's applications can be offered a schema. Pairs are presented per group, showing which have a schema, which are being built, and which failed, so whether a group's applications can be offered the schema for the version they run or are moving to is answered in one place. diff --git a/.workhorse/specs/public-server/restore-replicas.md b/.workhorse/specs/public-server/restore-replicas.md index 0cbc2e2c..9052e6af 100644 --- a/.workhorse/specs/public-server/restore-replicas.md +++ b/.workhorse/specs/public-server/restore-replicas.md @@ -100,6 +100,7 @@ The recognised semantics are: `once` for such an intent is keyed to the snapshot and the target version together (see [Pre-upgrade migration testing](#pre-upgrade-migration-testing)). - **reporting-schema** — the intent builds a Tamanu reporting schema from the replica it restores and registers it as an artifact (see [RPT](reporting-schemas.md)). It carries `migrate` alongside, and its entries name the version of the pair of group and Tamanu version being built for, on a central server of the group, rather than the server's candidate. + Canopy dispatches a build only for a declaration an operator has marked as publishing its group's schema, and accepts a published schema only from one (see [RPT](reporting-schemas.md)). `once` for such an intent is keyed to the group and the version rather than the snapshot, so a newer snapshot does not rebuild a schema the pair already has, and a failed build settles the pair. A settled pair is reinstated when the version's artifacts change or an operator asks for the build (see [RPT](reporting-schemas.md)). - **redact** — the intent can de-identify the restored data before serving it. @@ -129,6 +130,7 @@ Each declaration carries: - a human-readable **name**, distinct from every other declaration assigned to the same consumer; - **parameter values** for the intent's schema, defaulted where the schema provides one; - whether the replica **redacts**, offered only for an intent carrying `redact` (see [Redaction](#redaction)); +- whether the declaration **publishes its group's reporting schema**, offered only for a group-wide, non-redacting declaration of an intent carrying `reporting-schema` (see [RPT](reporting-schemas.md)); - an **overdue bound**: the maximum time a replica may go without meeting its intent's health expectation before Canopy considers it overdue, interpreted per the intent's semantics (see [Alerting](#alerting)); - whether the declaration is **enabled**. diff --git a/crates/canopy-api/src/generated.rs b/crates/canopy-api/src/generated.rs index c4aa56cc..123b033c 100644 --- a/crates/canopy-api/src/generated.rs +++ b/crates/canopy-api/src/generated.rs @@ -7,7 +7,7 @@ pub const OPENAPI_VERSION: &str = "1.0.0"; /// BLAKE3 digest of that document, so a document that changed without the /// version moving with it can be told from one that did not. -pub const OPENAPI_BLAKE3: &str = "626c2115b5bedb1e43372e734c98bcbf018e1c8f5ccb81683f2d1b13c4ddcee1"; +pub const OPENAPI_BLAKE3: &str = "3cfc9a09c64793448410328da3f666a868f4ba77d17b0f6b9031e7c9b669be3b"; /// Error types. pub mod error { @@ -1440,7 +1440,7 @@ opts into and the settings it accepts per replica.*/ /// "$ref": "#/components/schemas/BTreeMap" /// }, /// "semantics": { -/// "description": "Behaviours this intent opts into. Recognised values are `check` (a\nhealth report is expected for each replica), `once` (a given snapshot\nis only ever dispatched to a replica once, rather than repeatedly\nuntil overdue), and `url` (a replica's health report includes a link\nto it). Unrecognised values are stored but have no effect.", +/// "description": "Behaviours this intent opts into. Recognised values are `check` (a\nhealth report is expected for each replica), `once` (a given snapshot\nis only ever dispatched to a replica once, rather than repeatedly until\noverdue), `url` (a replica's health report includes a link to it),\n`migrate` (Canopy names a target version and the replica applies that\nversion's migrations), `redact` (the replica de-identifies the restored\ndata before serving it), and `reporting-schema` (the replica builds a\nTamanu reporting schema and registers it for the group). Unrecognised\nvalues are stored but have no effect, so a consumer may advertise ahead\nof Canopy support.", /// "type": "array", /// "items": { /// "type": "string" @@ -1466,9 +1466,14 @@ parameter name.*/ pub params: ::std::option::Option, /**Behaviours this intent opts into. Recognised values are `check` (a health report is expected for each replica), `once` (a given snapshot -is only ever dispatched to a replica once, rather than repeatedly -until overdue), and `url` (a replica's health report includes a link -to it). Unrecognised values are stored but have no effect.*/ +is only ever dispatched to a replica once, rather than repeatedly until +overdue), `url` (a replica's health report includes a link to it), +`migrate` (Canopy names a target version and the replica applies that +version's migrations), `redact` (the replica de-identifies the restored +data before serving it), and `reporting-schema` (the replica builds a +Tamanu reporting schema and registers it for the group). Unrecognised +values are stored but have no effect, so a consumer may advertise ahead +of Canopy support.*/ #[serde(default, skip_serializing_if = "::std::vec::Vec::is_empty")] pub semantics: ::std::vec::Vec<::std::string::String>, } @@ -2423,6 +2428,78 @@ run: if progress reports already carried it, that value stands.*/ #[serde(rename = "type")] pub type_: ::std::string::String, } +///What a reporting-schema build reports beyond its replica's restore health. +/// +///
JSON schema +/// +/// ```json +///{ +/// "description": "What a reporting-schema build reports beyond its replica's restore health.", +/// "type": "object", +/// "required": [ +/// "built" +/// ], +/// "properties": { +/// "artifacts": { +/// "description": "The artifacts the build registered, of which the schema is one.", +/// "type": "array", +/// "items": { +/// "type": "string", +/// "format": "uuid" +/// } +/// }, +/// "built": { +/// "description": "Whether a schema came out of the build.", +/// "type": "boolean" +/// }, +/// "error": { +/// "description": "What went wrong, where the build failed.", +/// "type": [ +/// "string", +/// "null" +/// ] +/// }, +/// "target_version": { +/// "description": "The version the schema was built for, as semver, echoed from the\nworklist entry's `target_version`.", +/// "type": [ +/// "string", +/// "null" +/// ] +/// }, +/// "target_version_id": { +/// "description": "The same version as the identifier, echoed from `target_version_id`.\nAccepted for a consumer that reports the identifier; omit it when\n`target_version` is sent.", +/// "type": [ +/// "string", +/// "null" +/// ], +/// "format": "uuid" +/// } +/// } +///} +/// ``` +///
+#[derive(::serde::Deserialize, ::serde::Serialize, Clone, Debug)] +#[derive(::bon::Builder)] +#[non_exhaustive] +pub struct ReportingSchemaArgs { + ///The artifacts the build registered, of which the schema is one. + #[serde(default, skip_serializing_if = "::std::vec::Vec::is_empty")] + pub artifacts: ::std::vec::Vec<::uuid::Uuid>, + ///Whether a schema came out of the build. + pub built: bool, + ///What went wrong, where the build failed. + #[serde(default, skip_serializing_if = "::std::option::Option::is_none")] + pub error: ::std::option::Option<::std::string::String>, + /**The version the schema was built for, as semver, echoed from the +worklist entry's `target_version`.*/ + #[serde(default, skip_serializing_if = "::std::option::Option::is_none")] + pub target_version: ::std::option::Option<::std::string::String>, + /**The same version as the identifier, echoed from `target_version_id`. +Accepted for a consumer that reports the identifier; omit it when +`target_version` is sent.*/ + #[serde(default, skip_serializing_if = "::std::option::Option::is_none")] + pub target_version_id: ::std::option::Option<::uuid::Uuid>, +} ///A request to certify a key for a name. /// ///
JSON schema @@ -3361,6 +3438,17 @@ impl ::std::fmt::Display for UrlField { /// "type": "string", /// "format": "uuid" /// }, +/// "reporting_schema": { +/// "oneOf": [ +/// { +/// "type": "null" +/// }, +/// { +/// "description": "What a reporting-schema build produced, where the replica was restored\nfor one. Absent on any other report.", +/// "$ref": "#/components/schemas/ReportingSchemaArgs" +/// } +/// ] +/// }, /// "run_id": { /// "description": "This must be the run-uuid the client minted for this run.\nThe field is optional only so older clients don't break; it WILL be made\nmandatory in future.", /// "type": [ @@ -3469,6 +3557,8 @@ checks. A replica only counts as verified when the outcome is type, and intent, so a report that named no declaration could not be attributed to one of them.*/ pub replica_id: ::uuid::Uuid, + #[serde(default, skip_serializing_if = "::std::option::Option::is_none")] + pub reporting_schema: ::std::option::Option, /**This must be the run-uuid the client minted for this run. The field is optional only so older clients don't break; it WILL be made mandatory in future.*/ @@ -3970,6 +4060,25 @@ impl crate::CanopyClient { pub async fn applications_self(&self) -> crate::Result { self.call_json(::http::Method::GET, "/applications/self", None::<&()>).await } + /// Register a reporting schema for one group, carrying its bytes. + /// + /// Requires a device certificate whose restore declaration for the named group + /// advertises that it builds reporting schemas. The bytes travel on this + /// connection and Canopy holds them, so the builder is issued no credential to + /// any store. The path names the group the artifact is for, the exact version + /// it was built against, and the artifact's type and target platform. + /// + /// The version must be one Canopy already holds: a build is dispatched for a + /// group and version Canopy knows about, so a version that does not exist is + /// refused rather than drafted. A range pattern is refused for the same reason: + /// a schema follows the migrations one exact version applies. + /// + /// Returns the created artifact record. + /// + /// `POST /artifacts/groups/{group}/{version}/{artifact_type}/{platform}` + pub async fn artifacts_groups(&self, group: &str, version: &str, artifact_type: &str, platform: &str) -> crate::Result { + self.call_json(::http::Method::POST, &format!("/artifacts/groups/{}/{}/{}/{}", group, version, artifact_type, platform), None::<&()>).await + } /// Register a downloadable artifact for a version or version range. /// /// Requires a device certificate with the releaser role (or admin). The diff --git a/crates/commons-types/src/backup.rs b/crates/commons-types/src/backup.rs index 3f253529..1ccf2a6c 100644 --- a/crates/commons-types/src/backup.rs +++ b/crates/commons-types/src/backup.rs @@ -418,6 +418,13 @@ pub mod semantics { /// worklist entry, withholds an entry from a server whose product has no /// manifest, and holds the replica to the redaction outcome reported back. pub const REDACT: &str = "redact"; + /// The intent builds a Tamanu reporting schema from the replica it restores + /// and registers it as a group-scoped artifact: Canopy names the pair's + /// version on the worklist entry, restores a machine of the group running a + /// central Tamanu application, and keys `once` to the group and the version + /// rather than the snapshot. + // spec: RPT + pub const REPORTING_SCHEMA: &str = "reporting-schema"; } /// The parameters Canopy owns on behalf of the `redact` semantic. @@ -498,9 +505,14 @@ pub struct IntentDescriptor { pub description: Option, /// Behaviours this intent opts into. Recognised values are `check` (a /// health report is expected for each replica), `once` (a given snapshot - /// is only ever dispatched to a replica once, rather than repeatedly - /// until overdue), and `url` (a replica's health report includes a link - /// to it). Unrecognised values are stored but have no effect. + /// is only ever dispatched to a replica once, rather than repeatedly until + /// overdue), `url` (a replica's health report includes a link to it), + /// `migrate` (Canopy names a target version and the replica applies that + /// version's migrations), `redact` (the replica de-identifies the restored + /// data before serving it), and `reporting-schema` (the replica builds a + /// Tamanu reporting schema and registers it for the group). Unrecognised + /// values are stored but have no effect, so a consumer may advertise ahead + /// of Canopy support. #[serde(default)] pub semantics: Vec, /// Configurable parameters this intent accepts per replica, keyed by diff --git a/crates/database/src/applications.rs b/crates/database/src/applications.rs index cd67afd5..f89ff05c 100644 --- a/crates/database/src/applications.rs +++ b/crates/database/src/applications.rs @@ -800,6 +800,15 @@ impl Application { .map_err(AppError::from) } + /// What to call this application to an operator: the name it was given, + /// else the host it answers on, else its id. + pub fn label(&self) -> String { + self.name + .clone() + .or_else(|| self.host.as_ref().map(|h| h.0.to_string())) + .unwrap_or_else(|| self.id.to_string()) + } + /// All live (non-archived) applications in a group, ordered by name. Used to /// expand a group-wide restore-replica declaration into per-server entries. pub async fn list_live_in_group( diff --git a/crates/database/src/artifacts.rs b/crates/database/src/artifacts.rs index 69a145b3..c18a9713 100644 --- a/crates/database/src/artifacts.rs +++ b/crates/database/src/artifacts.rs @@ -83,6 +83,9 @@ pub struct Artifact { pub digest: Option>, /// The run that produced this artifact, where the registration named one. pub run_id: Option, + /// When this artifact was last registered. + #[diesel(deserialize_as = jiff_diesel::Timestamp, serialize_as = jiff_diesel::Timestamp)] + pub updated_at: jiff::Timestamp, } #[derive(Debug, Deserialize, Insertable)] @@ -106,6 +109,10 @@ pub struct NewArtifact { pub run_id: Option, } +/// Cap on the bytes Canopy will hold for one artifact. A reporting schema is a +/// SQL file, and anything approaching this is not one. +pub const MAX_HELD_ARTIFACT_BYTES: usize = 32 * 1024 * 1024; + /// The digest Canopy records and verifies bytes against. pub fn digest_of(bytes: &[u8]) -> Vec { Sha256::digest(bytes).to_vec() @@ -406,6 +413,51 @@ impl Artifact { pattern_rank(pattern_b).cmp(&pattern_rank(pattern_a)) } + /// When any artifact a build reads was last registered for each of these + /// versions, in one query however many versions are asked about. + /// + /// A schema built from a superseded release of a version is not the schema + /// that version describes, so this is what a build is held against. Only + /// the unscoped artifacts count: a group-scoped one is a build's own output, + /// and registering it would put every group's pair for the version back on + /// the worklist, including the pair that just produced it. A range artifact + /// counts for every version it covers, since that is how one is resolved + /// for a build. + // spec: RPT#pairs + pub async fn newest_change_for_versions( + db: &mut AsyncPgConnection, + versions: &[Version], + ranges: &RangeChanges, + ) -> Result> { + use crate::schema::artifacts::dsl; + + let ids: Vec = versions.iter().map(|v| v.id).collect(); + let exact: Vec<(Option, Option)> = dsl::artifacts + .filter(dsl::version_id.eq_any(&ids)) + .filter(dsl::group_id.is_null()) + .group_by(dsl::version_id) + .select((dsl::version_id, diesel::dsl::max(dsl::updated_at))) + .load(db) + .await + .map_err(AppError::from)?; + + let mut newest: std::collections::HashMap = exact + .into_iter() + .filter_map(|(id, at)| Some((id?, at?.into()))) + .collect(); + + for (range, at) in &ranges.0 { + for version in versions.iter().filter(|v| range.satisfies(&v.as_semver())) { + newest + .entry(version.id) + .and_modify(|held| *held = (*held).max(*at)) + .or_insert(*at); + } + } + + Ok(newest) + } + /// The id an artifact of this identity is already registered under, where one /// is. An artifact rests under its id, so a re-registration that reuses it /// puts the new bytes where the old ones were. @@ -645,3 +697,44 @@ impl Artifact { }) } } + +/// When each unscoped range artifact last changed, with its pattern parsed. +/// +/// A range covers versions rather than naming one, so which of them it answers +/// for is decided in memory. Loaded once and handed to each version it is asked +/// about: the patterns do not vary by group, and a worklist poll asks the same +/// question of every group it covers. +// spec: RPT#pairs +pub struct RangeChanges(Vec<(node_semver::Range, jiff::Timestamp)>); + +impl RangeChanges { + /// One row per distinct pattern rather than per artifact: the answer only + /// needs the newest change under each, and every row returned costs a + /// semver parse. + pub async fn load(db: &mut AsyncPgConnection) -> Result { + use crate::schema::artifacts::dsl; + + let rows: Vec<(Option, Option)> = dsl::artifacts + .filter(dsl::version_id.is_null()) + .filter(dsl::group_id.is_null()) + .group_by(dsl::version_range_pattern) + .select(( + dsl::version_range_pattern, + diesel::dsl::max(dsl::updated_at), + )) + .load(db) + .await + .map_err(AppError::from)?; + + Ok(Self( + rows.into_iter() + .filter_map(|(pattern, at)| { + // An unparseable pattern matches nothing rather than + // everything, as it does where the artifact is offered. + let range = node_semver::Range::parse(pattern?).ok()?; + Some((range, at?.into())) + }) + .collect(), + )) + } +} diff --git a/crates/database/src/backup.rs b/crates/database/src/backup.rs index ef2b242c..920e2be7 100644 --- a/crates/database/src/backup.rs +++ b/crates/database/src/backup.rs @@ -29,6 +29,9 @@ pub async fn sweep(db: &mut AsyncPgConnection) -> Result { let mut filed = staleness::sweep(db, &rows).await?; filed += reconcile::sweep(db, &rows).await?; filed += crate::restore::sweep_restore_checks(db).await?; + // Not counted in `filed`: this one files per group rather than per event, + // and its own instances are pairs. + crate::reporting_schemas::sweep(db).await?; // Not an event, but the same cadence: a plan closes once its group reports // the target, and this sweep is what notices. crate::upgrade_plans::close_met_plans(db).await?; diff --git a/crates/database/src/backup/refs.rs b/crates/database/src/backup/refs.rs index b9c32595..7c01c381 100644 --- a/crates/database/src/backup/refs.rs +++ b/crates/database/src/backup/refs.rs @@ -122,6 +122,13 @@ pub const MIGRATION_TEST: &str = "migration-test"; /// redacting replicas as instances. pub const REDACTION: &str = "redaction"; +/// A reporting schema could not be built for a pair of this group and a Tamanu +/// version it runs or is moving to. Application-scoped, `Warning`, does not +/// escalate. One check on the group's central application with its unbuilt +/// pairs as instances. +// spec: RPT#alerting +pub const REPORTING_SCHEMA: &str = "reporting-schema"; + // --- shipped documentation (seeded into the catalog on first filing) --- pub const STALENESS_DOC: &str = "## Description @@ -302,6 +309,18 @@ One of this server's managed restore replicas reported a failed restorability ch Read the detail for the replicas named: restore errors point at the snapshot or credentials, staleness at the consumer itself. To handle one replica differently from the rest, write a rule or silence against its `check.replica_key` rather than the check as a whole."; +pub const REPORTING_SCHEMA_DOC: &str = "## Description + +A reporting schema is built for each pair of a group and a Tamanu version it runs or is moving to, from a replica of the group's own data migrated to that version. This check says a build for one of those pairs failed. Reports on the servers themselves keep working against whatever schema they already have; what is missing is the schema for a version, so reports written against it have nothing to read from. The version is in the detail rather than the check name, so a release doesn't spawn a catalog entry of its own. + +## Results + +- **warn**: a build for one of this group's pairs failed. The server is up and its reports return rows, so this is for whoever maintains the reports rather than whoever is on call. + +## Solve + +Read the failure in the report detail. A build failing against a fixed version and configuration fails the same way every time, so the pair stays settled until the version's artifacts change or an operator asks for the build again."; + pub const MIGRATION_TEST_DOC: &str = "## Description A candidate version's schema migrations were applied to a restore replica of this server's data and one of them failed, or the candidate has gone untried past the replica's overdue bound. The server itself is unaffected: it is still running the version it was, and the finding is about a version it has not taken. The version under test is in the detail rather than the check name, so a release doesn't spawn a catalog entry of its own. diff --git a/crates/database/src/backup/staleness.rs b/crates/database/src/backup/staleness.rs index 98f04cbe..5ea04946 100644 --- a/crates/database/src/backup/staleness.rs +++ b/crates/database/src/backup/staleness.rs @@ -634,6 +634,29 @@ pub(crate) async fn open_server_issue_active( Ok(n > 0) } +/// The applications an active `(canopy, ref)` issue is open against, with the +/// group each belongs to. +/// +/// One question of the whole fleet: a sweep that files per group otherwise asks +/// it per group to learn whether it has anything to recover. +pub(crate) async fn applications_with_open_issue( + db: &mut AsyncPgConnection, + r#ref: &str, +) -> Result)>> { + use crate::schema::{applications, issues}; + + issues::table + .inner_join(applications::table.on(applications::id.nullable().eq(issues::application_id))) + .filter(issues::source.eq(refs::CANOPY_SOURCE)) + .filter(issues::ref_.eq(r#ref)) + .filter(issues::active.eq(true)) + .filter(issues::resolved_at.is_null()) + .select((applications::id, applications::group_id)) + .load(db) + .await + .map_err(Into::into) +} + /// Whether a machine-scoped `(canopy, ref)` check last *observed* something /// other than a pass. /// diff --git a/crates/database/src/lib.rs b/crates/database/src/lib.rs index ec7f5a8d..9d749544 100644 --- a/crates/database/src/lib.rs +++ b/crates/database/src/lib.rs @@ -29,6 +29,7 @@ pub mod partitions; pub mod pg_duration; pub mod recovery_vault; pub mod reported_detail; +pub mod reporting_schemas; pub mod restore; pub mod schema; pub mod self_alerts; diff --git a/crates/database/src/reporting_schemas.rs b/crates/database/src/reporting_schemas.rs new file mode 100644 index 00000000..eddc55fc --- /dev/null +++ b/crates/database/src/reporting_schemas.rs @@ -0,0 +1,663 @@ +//! Reporting-schema builds: which pairs of group and Tamanu version have a +//! schema, which have been tried, and which an operator has asked for again. +//! +//! spec: RPT + +use commons_errors::{AppError, Result}; +use diesel::prelude::*; +use diesel_async::{AsyncPgConnection, RunQueryDsl}; +use jiff::Timestamp; +use serde::{Deserialize, Serialize}; +use uuid::Uuid; + +use crate::{ + restore::{BackupRestoreCheck, NewBackupRestoreCheck}, + versions::Version, +}; +use commons_types::backup::RunOutcome; + +/// A build of one pair, hanging off the restore report that carries the +/// replica's own health. +#[derive(Debug, Clone, Serialize, Queryable, Selectable, utoipa::ToSchema)] +#[diesel(table_name = crate::schema::reporting_schema_builds)] +#[diesel(check_for_backend(diesel::pg::Pg))] +pub struct ReportingSchemaBuild { + /// The restore report this build was reported with. + pub check_id: i64, + /// The group the schema was built for. + pub group_id: Uuid, + /// The Tamanu version the schema was built for. + pub version_id: Uuid, + /// The central application whose snapshot the replica was restored from. + pub application_id: Option, + /// Whether a schema came out of it. + pub built: bool, + /// What went wrong, where it did not. + pub error: Option, + /// The artifacts this build registered, of which the schema is one. + pub artifact_ids: Vec>, + /// When the build was recorded, which is what a later artifact change is + /// compared against. + #[diesel(deserialize_as = jiff_diesel::Timestamp, serialize_as = jiff_diesel::Timestamp)] + pub built_at: Timestamp, +} + +#[derive(Debug, Clone)] +pub struct NewReportingSchemaBuild { + pub group_id: Uuid, + pub version_id: Uuid, + pub application_id: Option, + pub built: bool, + pub error: Option, + pub artifact_ids: Vec, +} + +impl ReportingSchemaBuild { + /// Record a build: the replica's restore report first, then the build that + /// rode on it. + // spec: RPT#what-a-build-reports + pub async fn record( + db: &mut AsyncPgConnection, + report: NewBackupRestoreCheck, + build: NewReportingSchemaBuild, + ) -> Result { + let restore_failed = report.outcome != RunOutcome::Success; + let began_at = match report.run_id { + Some(run) => run_started_at(db, run).await?.unwrap_or(report.observed_at), + None => report.observed_at, + }; + + let check_id = BackupRestoreCheck::record_report(db, report).await?; + + // A replica that failed to restore says nothing about whether the pair + // can be built: the build never ran. Restore-health already raises on + // that, and recording no build leaves the pair unsettled so it is + // dispatched again, which is what an unhealthy restore should do. + if restore_failed { + return Ok(check_id); + } + + diesel::insert_into(crate::schema::reporting_schema_builds::table) + .values(( + crate::schema::reporting_schema_builds::check_id.eq(check_id), + crate::schema::reporting_schema_builds::group_id.eq(build.group_id), + crate::schema::reporting_schema_builds::version_id.eq(build.version_id), + crate::schema::reporting_schema_builds::application_id.eq(build.application_id), + crate::schema::reporting_schema_builds::built.eq(build.built), + crate::schema::reporting_schema_builds::error.eq(build.error), + crate::schema::reporting_schema_builds::artifact_ids.eq(build + .artifact_ids + .into_iter() + .map(Some) + .collect::>()), + )) + .execute(db) + .await?; + + // An operator's ask is answered once the build it asked for lands, + // whichever way it went. A build takes half an hour, and an ask entered + // while it ran is for whatever changed after it began, so what answers + // that one is the next build rather than this. + ReportingSchemaRequest::clear(db, build.group_id, build.version_id, began_at).await?; + + Ok(check_id) + } + + /// The most recent build of a pair, if it has been tried. + pub async fn latest_for_pair( + db: &mut AsyncPgConnection, + group: Uuid, + version: Uuid, + ) -> Result> { + use crate::schema::{backup_restore_checks, reporting_schema_builds}; + + reporting_schema_builds::table + .inner_join( + backup_restore_checks::table + .on(backup_restore_checks::id.eq(reporting_schema_builds::check_id)), + ) + .filter(reporting_schema_builds::group_id.eq(group)) + .filter(reporting_schema_builds::version_id.eq(version)) + .order_by(backup_restore_checks::reported_at.desc()) + .select(Self::as_select()) + .first(db) + .await + .optional() + .map_err(AppError::from) + } + + /// The most recent build of each of a group's pairs, by version. + /// + /// One query rather than one per version: this backs both the operator page + /// and the sweep, which walk every version a group runs. + pub async fn latest_by_version_for_group( + db: &mut AsyncPgConnection, + group: Uuid, + ) -> Result> { + use crate::schema::{backup_restore_checks, reporting_schema_builds}; + + let builds: Vec = reporting_schema_builds::table + .inner_join( + backup_restore_checks::table + .on(backup_restore_checks::id.eq(reporting_schema_builds::check_id)), + ) + .filter(reporting_schema_builds::group_id.eq(group)) + .distinct_on(reporting_schema_builds::version_id) + .order_by(( + reporting_schema_builds::version_id, + backup_restore_checks::reported_at.desc(), + )) + .select(Self::as_select()) + .load(db) + .await + .map_err(AppError::from)?; + + Ok(builds.into_iter().map(|b| (b.version_id, b)).collect()) + } + + /// Whether a pair is settled: it has been built or has failed, and either + /// way is not dispatched again until the version's artifacts change or an + /// operator asks. + // spec: RPT#pairs + pub async fn is_settled( + db: &mut AsyncPgConnection, + group: Uuid, + version: Uuid, + ) -> Result { + let row = Version::get_by_id(db, version).await?; + let ranges = crate::artifacts::RangeChanges::load(db).await?; + let settlement = + Settlement::for_group(db, group, std::slice::from_ref(&row), &ranges).await?; + Ok(settlement.settled(version)) + } +} + +/// Where every pair of a group stands, answered from memory. +/// +/// The worklist asks this of each of a group's versions on every poll, and +/// every restore consumer polls on a schedule, so the three lookups it takes +/// are made once for the group rather than once per pair. +// spec: RPT#pairs +pub struct Settlement { + requested: std::collections::HashSet, + builds: std::collections::HashMap, + changed: std::collections::HashMap, +} + +impl Settlement { + pub async fn for_group( + db: &mut AsyncPgConnection, + group: Uuid, + versions: &[Version], + ranges: &crate::artifacts::RangeChanges, + ) -> Result { + Ok(Self { + requested: ReportingSchemaRequest::pending_for_group(db, group).await?, + builds: ReportingSchemaBuild::latest_by_version_for_group(db, group).await?, + changed: crate::artifacts::Artifact::newest_change_for_versions(db, versions, ranges) + .await?, + }) + } + + pub fn settled(&self, version: Uuid) -> bool { + if self.requested.contains(&version) { + return false; + } + + let Some(build) = self.builds.get(&version) else { + return false; + }; + + // A schema built from a superseded release of the version is not the + // schema that version describes, so an artifact registered since the + // build puts the pair back on the worklist. + match self.changed.get(&version) { + Some(at) => *at <= build.built_at, + None => true, + } + } +} + +/// When the run behind a report began, read from the first credential it was +/// issued. +/// +/// A run reports once it is over, so its own timestamp is the far end of a +/// window half an hour wide, and what it started before is the question an ask +/// made inside that window turns on. +async fn run_started_at(db: &mut AsyncPgConnection, run: Uuid) -> Result> { + use crate::schema::backup_credential_issuances::dsl; + + let issued: Option = dsl::backup_credential_issuances + .filter(dsl::run_id.eq(Some(run))) + .select(diesel::dsl::min(dsl::issued_at)) + .first(db) + .await + .map_err(AppError::from)?; + + Ok(issued.map(Into::into)) +} + +/// An operator asking for a pair's build. +#[derive(Debug, Clone, Serialize, Deserialize, Queryable, Selectable, utoipa::ToSchema)] +#[diesel(table_name = crate::schema::reporting_schema_requests)] +#[diesel(check_for_backend(diesel::pg::Pg))] +pub struct ReportingSchemaRequest { + pub group_id: Uuid, + pub version_id: Uuid, + #[diesel(deserialize_as = jiff_diesel::Timestamp, serialize_as = jiff_diesel::Timestamp)] + pub requested_at: Timestamp, + pub requested_by: Option, +} + +impl ReportingSchemaRequest { + /// Enqueue, or refresh, an ask for a pair. + // spec: RPT#pairs + pub async fn enqueue( + db: &mut AsyncPgConnection, + group: Uuid, + version: Uuid, + requested_by: Option<&str>, + ) -> Result<()> { + use crate::schema::reporting_schema_requests::dsl; + + diesel::insert_into(dsl::reporting_schema_requests) + .values(( + dsl::group_id.eq(group), + dsl::version_id.eq(version), + dsl::requested_by.eq(requested_by), + )) + .on_conflict((dsl::group_id, dsl::version_id)) + .do_update() + .set(( + dsl::requested_at.eq(diesel::dsl::now), + dsl::requested_by.eq(requested_by), + )) + .execute(db) + .await + .map_err(AppError::from)?; + + Ok(()) + } + + /// Which of a group's versions have an ask pending, in one query. + pub async fn pending_for_group( + db: &mut AsyncPgConnection, + group: Uuid, + ) -> Result> { + use crate::schema::reporting_schema_requests::dsl; + + let versions: Vec = dsl::reporting_schema_requests + .filter(dsl::group_id.eq(group)) + .select(dsl::version_id) + .load(db) + .await + .map_err(AppError::from)?; + + Ok(versions.into_iter().collect()) + } + + /// Clear a pair's ask, where it was made before `answered_at`. + async fn clear( + db: &mut AsyncPgConnection, + group: Uuid, + version: Uuid, + answered_at: Timestamp, + ) -> Result<()> { + use crate::schema::reporting_schema_requests::dsl; + + diesel::delete( + dsl::reporting_schema_requests + .filter(dsl::group_id.eq(group)) + .filter(dsl::version_id.eq(version)) + .filter(dsl::requested_at.lt(jiff_diesel::Timestamp::from(answered_at))), + ) + .execute(db) + .await + .map_err(AppError::from)?; + + Ok(()) + } +} + +/// Where a pair stands, for the operator view. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, utoipa::ToSchema)] +#[serde(rename_all = "lowercase")] +pub enum PairState { + /// No build has been recorded, so the pair is on the worklist. + Awaiting, + /// A build produced a schema. + Built, + /// A build ran and produced none. + Failed, +} + +/// One pair of group and Tamanu version, and where it stands. +#[derive(Debug, Clone, Serialize, utoipa::ToSchema)] +pub struct Pair { + /// The group this pair is for. + pub group_id: Uuid, + /// The Tamanu version this pair is for. + pub version_id: Uuid, + /// That version as semver, for display. + pub version: String, + /// Whether the pair has a schema, failed to build one, or is awaiting one. + pub state: PairState, + /// What went wrong, where a build failed. + pub error: Option, + /// Whether an operator has asked for this pair to be built again. + pub requested: bool, + /// The group's Tamanu applications reporting this version, by name. Empty + /// where the pair comes from the open plan rather than from something + /// running it. + pub applications: Vec, +} + +/// The pairs of a group: every published version its Tamanu applications report +/// running, plus the version its open plan moves it to. +/// +/// A group no enabled declaration covers has no pairs. Canopy owes it no +/// schema, so listing versions against it would offer an operator a build +/// nothing will pick up. +// spec: RPT#pairs +pub async fn pairs_for_group(db: &mut AsyncPgConnection, group: Uuid) -> Result> { + if !groups_building_schemas(db).await?.contains(&group) { + return Ok(Vec::new()); + } + + let members = crate::applications::Application::list_live_in_group(db, group).await?; + pairs_of_members(db, group, &members).await +} + +/// The pairs of a group already known to have a builder, from members already +/// in hand. +// spec: RPT#pairs +async fn pairs_of_members( + db: &mut AsyncPgConnection, + group: Uuid, + members: &[crate::applications::Application], +) -> Result> { + let versions = versions_and_applications(db, group, members).await?; + let builds = ReportingSchemaBuild::latest_by_version_for_group(db, group).await?; + let requests = ReportingSchemaRequest::pending_for_group(db, group).await?; + + let mut pairs = Vec::with_capacity(versions.len()); + for (version, applications) in versions { + let (state, error) = match builds.get(&version.id) { + None => (PairState::Awaiting, None), + Some(build) if build.built => (PairState::Built, None), + Some(build) => (PairState::Failed, build.error.clone()), + }; + + pairs.push(Pair { + group_id: group, + version_id: version.id, + applications, + version: version.as_semver().to_string(), + state, + error, + requested: requests.contains(&version.id), + }); + } + + Ok(pairs) +} + +/// Every published version a group's Tamanu applications report running, plus +/// the version its open plan moves it to. +/// +/// A reported version Canopy holds no release row for is not a pair: a build +/// needs that version's migrations, which reach a builder as its published +/// artifacts. +// spec: RPT#pairs +pub async fn versions_for_group(db: &mut AsyncPgConnection, group: Uuid) -> Result> { + let members = crate::applications::Application::list_live_in_group(db, group).await?; + versions_of_members(db, group, &members).await +} + +/// The versions of a group's pairs, from members already in hand. +pub async fn versions_of_members( + db: &mut AsyncPgConnection, + group: Uuid, + members: &[crate::applications::Application], +) -> Result> { + Ok(versions_and_applications(db, group, members) + .await? + .into_iter() + .map(|(version, _)| version) + .collect()) +} + +/// A group's pairs, and which of its Tamanu applications report each one. +/// +/// The applications are carried alongside the version rather than joined back +/// on a stringified semver: a `Version` row holds major, minor and patch alone, +/// so a reported `2.60.0-rc1` resolves to the 2.60.0 row and would never match +/// its own key, and the pair would read as an upgrade plan while a server runs +/// it. +// spec: RPT#pairs +async fn versions_and_applications( + db: &mut AsyncPgConnection, + group: Uuid, + applications: &[crate::applications::Application], +) -> Result)>> { + use commons_types::version::VersionStatus; + + let tamanu: Vec<&crate::applications::Application> = applications + .iter() + .filter(|a| a.r#type.software() == "tamanu") + .collect(); + + let ids: Vec = tamanu.iter().map(|a| a.id).collect(); + let reported = crate::reported_detail::ReportedDetail::last_versions(db, &ids).await?; + + let mut wanted: Vec = reported.values().cloned().collect(); + wanted.sort_by(|a, b| a.0.cmp(&b.0)); + wanted.dedup_by(|a, b| a.0 == b.0); + let released = Version::get_by_versions(db, &wanted).await?; + + let mut pairs: Vec<(Version, Vec)> = Vec::new(); + for application in tamanu { + let Some(shown) = reported.get(&application.id) else { + continue; + }; + // A version Canopy holds no release row for is not a pair: a build needs + // that version's migrations, which reach a builder as published artifacts. + let Some(version) = released.iter().find(|v| { + (v.major, v.minor, v.patch) + == ( + shown.0.major as i32, + shown.0.minor as i32, + shown.0.patch as i32, + ) + }) else { + continue; + }; + if version.status != VersionStatus::Published { + continue; + } + + // A pair is unique per group and version, so two applications on one + // version are one pair carrying both names. + match pairs.iter_mut().find(|(v, _)| v.id == version.id) { + Some((_, names)) => names.push(application.label()), + None => pairs.push((version.clone(), vec![application.label()])), + } + } + + // A plan moving a group to a version something already runs adds no pair. + // Dispatch counts a restore and a migrate per entry, so a duplicate here is + // paid for rather than merely untidy. + if let Some(target) = crate::upgrade_plans::planned_target(db, group).await? + && !pairs.iter().any(|(v, _)| v.id == target.id) + { + pairs.push((target, Vec::new())); + } + + for (_, names) in &mut pairs { + names.sort(); + } + pairs.sort_by_key(|(v, _)| (v.major, v.minor, v.patch)); + + Ok(pairs) +} + +/// File the reporting-schema check for every group that has a builder. +/// +/// One check per group, on its central application, with each of the group's +/// pairs as an instance. The version is in the instance detail rather than the +/// check name, so a release does not spawn a catalog entry of its own. +// spec: RPT#alerting +pub async fn sweep(db: &mut AsyncPgConnection) -> Result<()> { + use crate::{ + applications::Application, + backup::{refs, staleness::applications_with_open_issue}, + issues::{CheckInstance, GradedInstance, Scope}, + restore::{RestoreCheck, file_restore_check}, + server_groups::ServerGroup, + }; + use commons_types::status::CheckResult; + + // Which groups have a builder is one question of the whole fleet rather than + // one per group: asking per group walked every group's declarations and + // every declaration's consumer, once a minute, for groups that have none. + let builders = groups_building_schemas(db).await?; + + // A group that has stopped building still has whatever this check filed + // while it did, and filing no instances is what says the finding is gone. + // Asked of the fleet at once, it is also what keeps this to the groups the + // sweep has something to say about: every other group is walked, its + // members loaded and its issues probed, once a minute, to file nothing. + // spec: RPT#alerting + let open = applications_with_open_issue(db, refs::REPORTING_SCHEMA).await?; + let walk: std::collections::HashSet = builders + .iter() + .copied() + .chain(open.iter().filter_map(|(_, group)| *group)) + .collect(); + + for group in ServerGroup::list_all(db).await? { + if !walk.contains(&group.id) { + continue; + } + + let members = Application::list_live_in_group(db, group.id).await?; + // The check files on the group's central. A group that has lost it + // keeps the finding open against whichever application it was filed on, + // which is the only scope a recovery reaches it through. + let Some(central) = ServerGroup::canonical_central(&members) + .map(|a| a.id) + .or_else(|| { + open.iter() + .find(|(_, g)| *g == Some(group.id)) + .map(|(application, _)| *application) + }) + else { + continue; + }; + + let pairs = if builders.contains(&group.id) { + pairs_of_members(db, group.id, &members).await? + } else { + Vec::new() + }; + + let instances: Vec = pairs + .iter() + .filter(|p| p.state != PairState::Awaiting) + .map(|pair| { + let mut detail = serde_json::json!({ "version": pair.version }); + if pair.state != PairState::Built { + detail["why"] = pair + .error + .clone() + .unwrap_or_else(|| format!("no schema could be built for {}", pair.version)) + .into(); + } + + CheckInstance { + label: pair.version.clone(), + observed: match pair.state { + PairState::Built => CheckResult::Passed, + _ => CheckResult::Warning, + }, + detail: Some(detail), + } + }) + .collect(); + + let name = group.name.clone(); + let total = pairs.len(); + file_restore_check( + db, + Scope::Application(central), + RestoreCheck { + r#ref: refs::REPORTING_SCHEMA, + documentation: refs::REPORTING_SCHEMA_DOC, + title: "reporting schema not built", + gone: &format!("No reporting schema is owed for {}", group.name), + }, + instances, + &move |degraded: &[GradedInstance]| match degraded { + [] => format!("Reporting schemas are built for every version {name} runs"), + [one] => format!( + "No reporting schema for {name} on {}: {}", + one.label, + one.detail + .as_ref() + .and_then(|d| d.get("why")) + .and_then(|v| v.as_str()) + .unwrap_or("the build failed") + ), + many => format!( + "No reporting schema for {} of {total} versions {name} runs: {}", + many.len(), + many.iter() + .map(|i| i.label.as_str()) + .collect::>() + .join(", ") + ), + }, + ) + .await?; + } + + Ok(()) +} + +/// The groups an enabled declaration builds schemas for. +/// +/// The same conditions `RestoreReplica::authorizes_schema_artifacts` asks of one +/// consumer and one group, asked of the fleet at once: dispatching builds a +/// group would then refuse to accept is the divergence worth not having. +async fn groups_building_schemas( + db: &mut AsyncPgConnection, +) -> Result> { + use crate::schema::{restore_consumer_capabilities, restore_replicas}; + use diesel::dsl::sql; + use diesel::sql_types::Bool; + + let groups: Vec = restore_replicas::table + .inner_join( + restore_consumer_capabilities::table.on( + restore_consumer_capabilities::consumer_device_id + .eq(restore_replicas::consumer_device_id) + .and(restore_consumer_capabilities::intent.eq(restore_replicas::intent)), + ), + ) + .filter(restore_replicas::enabled.eq(true)) + .filter(restore_replicas::publishes_schemas.eq(true)) + // Dispatch builds no schema from a redacting or machine-scoped + // declaration, and one nothing is dispatched for publishes nothing. + .filter(restore_replicas::redacts.eq(false)) + .filter(restore_replicas::machine_id.is_null()) + .filter(sql::( + "restore_consumer_capabilities.semantics @> '[\"reporting-schema\"]'::jsonb", + )) + .select(restore_replicas::group_id) + .distinct() + .load(db) + .await + .map_err(AppError::from)?; + + Ok(groups.into_iter().collect()) +} diff --git a/crates/database/src/restore.rs b/crates/database/src/restore.rs index c67d32d7..b04cbe1f 100644 --- a/crates/database/src/restore.rs +++ b/crates/database/src/restore.rs @@ -81,6 +81,12 @@ pub struct RestoreReplica { /// the whole of the operator's say in it, and it answers on its own /// whether a replica that came up unmasked is a finding. pub redacts: bool, + /// Whether this declaration's consumer may publish the group's reporting + /// schema. Only an operator sets it: what a consumer advertises is the + /// consumer's own claim, and every machine in the group runs what is + /// published for it. + // spec: RPT#the-build-contract + pub publishes_schemas: bool, /// Whether this declaration is currently active. When disabled, it /// produces no work and grants no access, but is kept for reference. pub enabled: bool, @@ -107,6 +113,7 @@ pub struct NewRestoreReplica { pub overdue_after: Option, pub params: serde_json::Value, pub redacts: bool, + pub publishes_schemas: bool, pub created_by: Option, } @@ -124,6 +131,7 @@ pub struct RestoreReplicaUpdate { pub overdue_after: Option, pub params: serde_json::Value, pub redacts: bool, + pub publishes_schemas: bool, pub enabled: bool, } @@ -151,6 +159,14 @@ fn unique_violation(info: &dyn diesel::result::DatabaseErrorInformation) -> AppE Some("restore_replicas_consumer_name") | None => { AppError::Conflict("this consumer already has a restore replica with that name".into()) } + // What a builder registers is offered to every machine in the group and + // replaces what was registered before it, so two publishers overwrite + // each other and which schema a machine ends up on is whichever + // reported last. + // spec: RPT#the-build-contract + Some("restore_replicas_one_schema_publisher") => AppError::Conflict( + "another enabled declaration already publishes this group's reporting schema".into(), + ), Some(other) => { AppError::Conflict(format!("this declaration collides with another ({other})")) } @@ -261,6 +277,7 @@ impl RestoreReplica { dsl::overdue_after.eq(update.overdue_after), dsl::params.eq(update.params), dsl::redacts.eq(update.redacts), + dsl::publishes_schemas.eq(update.publishes_schemas), dsl::enabled.eq(update.enabled), )) .returning(Self::as_select()) @@ -311,6 +328,100 @@ impl RestoreReplica { .await } + /// Whether a consumer may register group-scoped artifacts for this group: + /// an operator has marked an enabled declaration of theirs covering the + /// group as publishing its schema, and that declaration is one a build is + /// actually dispatched for. + /// + /// The operator's flag is what grants this, not the semantics the consumer + /// advertises: a device registers its own capability set, so a semantic is + /// a claim the claimant controls, and what is published here is offered to + /// every machine in the group and run. The advertised semantic still has to + /// be there, since a consumer that cannot build a schema has no business + /// publishing one, but it grants nothing on its own. + // spec: ART#registration, RPT#the-build-contract + pub async fn authorizes_schema_artifacts( + db: &mut AsyncPgConnection, + consumer_device_id: Uuid, + group_id: Uuid, + ) -> Result { + let building: Vec = + RestoreConsumerCapability::list_for_consumer(db, consumer_device_id) + .await? + .into_iter() + .filter(|d| d.has_semantic(semantics::REPORTING_SCHEMA)) + .map(|d| d.intent) + .collect(); + + if building.is_empty() { + return Ok(false); + } + + use crate::schema::restore_replicas::dsl; + let n: i64 = dsl::restore_replicas + .filter(dsl::consumer_device_id.eq(consumer_device_id)) + .filter(dsl::group_id.eq(group_id)) + .filter(dsl::intent.eq_any(building.iter().map(|i| i.0.clone()).collect::>())) + .filter(dsl::enabled.eq(true)) + .filter(dsl::publishes_schemas.eq(true)) + // Dispatch builds no schema from a redacting or machine-scoped + // declaration, and one nothing is dispatched for publishes nothing. + .filter(dsl::redacts.eq(false)) + .filter(dsl::machine_id.is_null()) + .count() + .get_result(db) + .await + .map_err(AppError::from)?; + + Ok(n > 0) + } + + /// Whether a run id is already recorded against a different consumer or + /// group. + /// + /// A run id is minted by the device performing the run, so one Canopy has + /// not seen is ordinary: an artifact is registered mid-restore, before the + /// report of that restore lands. One already recorded for somebody else is + /// a claim on their run, and provenance a party can forge for itself is + /// worth nothing to the operator reading it. + pub async fn run_claimed_elsewhere( + db: &mut AsyncPgConnection, + run: Uuid, + consumer_device_id: Uuid, + group_id: Uuid, + ) -> Result { + use crate::schema::{backup_restore_checks, backup_runs}; + + let checks: i64 = backup_restore_checks::table + .filter(backup_restore_checks::run_id.eq(Some(run))) + .filter( + backup_restore_checks::consumer_device_id + .ne(consumer_device_id) + .or(backup_restore_checks::group_id.ne(group_id)), + ) + .count() + .get_result(db) + .await + .map_err(AppError::from)?; + if checks > 0 { + return Ok(true); + } + + let runs: i64 = backup_runs::table + .filter(backup_runs::id.eq(run)) + .filter( + backup_runs::device_id + .ne(consumer_device_id) + .or(backup_runs::group_id.ne(group_id)), + ) + .count() + .get_result(db) + .await + .map_err(AppError::from)?; + + Ok(runs > 0) + } + /// Whether an enabled declaration covers `(consumer, group, type)` — the /// authorization check for issuing restore credentials. A server-scoped or /// a group-wide declaration both satisfy it. @@ -1655,11 +1766,11 @@ async fn file_migration( /// The fixed parts of one restore check: what it is called, the documentation it /// ships with, its headline when degraded, and what it says once a server has no /// instances of it left. -struct RestoreCheck<'a> { - r#ref: &'a str, - documentation: &'a str, - title: &'a str, - gone: &'a str, +pub(crate) struct RestoreCheck<'a> { + pub(crate) r#ref: &'a str, + pub(crate) documentation: &'a str, + pub(crate) title: &'a str, + pub(crate) gone: &'a str, } /// File one of a server's restore checks from its instances, and say whether it @@ -1671,7 +1782,7 @@ struct RestoreCheck<'a> { /// instances is recovered on its own — with no instances there is nothing left /// to grade, so it is filed as the plain passing check it has become rather /// than left open with nothing that could ever clear it. -async fn file_restore_check( +pub(crate) async fn file_restore_check( db: &mut AsyncPgConnection, scope: Scope, check: RestoreCheck<'_>, diff --git a/crates/database/src/schema.rs b/crates/database/src/schema.rs index 494bf8f1..bb28fbe9 100644 --- a/crates/database/src/schema.rs +++ b/crates/database/src/schema.rs @@ -604,6 +604,28 @@ diesel::table! { } } +diesel::table! { + reporting_schema_builds (check_id) { + check_id -> Int8, + group_id -> Uuid, + version_id -> Uuid, + application_id -> Nullable, + built -> Bool, + error -> Nullable, + artifact_ids -> Array>, + built_at -> Timestamptz, + } +} + +diesel::table! { + reporting_schema_requests (group_id, version_id) { + group_id -> Uuid, + version_id -> Uuid, + requested_at -> Timestamptz, + requested_by -> Nullable, + } +} + diesel::table! { restore_consumer_capabilities (consumer_device_id, intent) { consumer_device_id -> Uuid, @@ -632,6 +654,7 @@ diesel::table! { updated_at -> Timestamptz, params -> Jsonb, redacts -> Bool, + publishes_schemas -> Bool, } } @@ -882,6 +905,12 @@ diesel::joinable!(migration_tests -> applications (application_id)); diesel::joinable!(migration_tests -> backup_restore_checks (check_id)); diesel::joinable!(migration_tests -> versions (target_version_id)); diesel::joinable!(migration_timings -> migration_tests (check_id)); +diesel::joinable!(reporting_schema_builds -> applications (application_id)); +diesel::joinable!(reporting_schema_builds -> backup_restore_checks (check_id)); +diesel::joinable!(reporting_schema_builds -> server_groups (group_id)); +diesel::joinable!(reporting_schema_builds -> versions (version_id)); +diesel::joinable!(reporting_schema_requests -> server_groups (group_id)); +diesel::joinable!(reporting_schema_requests -> versions (version_id)); diesel::joinable!(restore_consumer_capabilities -> devices (consumer_device_id)); diesel::joinable!(restore_replicas -> devices (consumer_device_id)); diesel::joinable!(restore_replicas -> machines (machine_id)); @@ -946,6 +975,8 @@ diesel::allow_tables_to_appear_in_same_query!( migration_tests, migration_timings, recovery_vault_writes, + reporting_schema_builds, + reporting_schema_requests, restore_consumer_capabilities, restore_replicas, scoped_check_policies, diff --git a/crates/database/src/server_groups.rs b/crates/database/src/server_groups.rs index bfbfdcb7..bf4a724a 100644 --- a/crates/database/src/server_groups.rs +++ b/crates/database/src/server_groups.rs @@ -191,6 +191,22 @@ impl ServerGroup { .collect()) } + /// The group's canonical central application: the highest-ranked one, and + /// the lowest id among equals so the choice is stable. + /// + /// There is no fallback. A group with no central has none, because a + /// group's version, and the database a reporting schema is built from, are + /// things its central has and nothing else stands in for. + // spec: APP#capabilities + pub fn canonical_central( + members: &[crate::applications::Application], + ) -> Option<&crate::applications::Application> { + members + .iter() + .filter(|s| s.r#type == ApplicationType::TamanuCentral) + .min_by_key(|s| (rank_priority(s.rank), s.id)) + } + pub async fn list_all(db: &mut AsyncPgConnection) -> Result> { use crate::schema::server_groups::dsl; dsl::server_groups @@ -518,11 +534,7 @@ impl ServerGroup { // version, because a group's version is a thing its central has // and nothing else stands in for it. // spec: APP#capabilities - let canonical = members - .iter() - .filter(|s| s.r#type == ApplicationType::TamanuCentral) - .min_by_key(|s| (rank_priority(s.rank), s.id)) - .map(|s| s.id); + let canonical = Self::canonical_central(&members).map(|s| s.id); let (version_application_id, effective_version) = match canonical { None => (None, None), diff --git a/crates/database/src/statuses.rs b/crates/database/src/statuses.rs index 2a338374..8d0e80d2 100644 --- a/crates/database/src/statuses.rs +++ b/crates/database/src/statuses.rs @@ -59,13 +59,6 @@ const GRACE_LOOKBACK_SQL: &str = "NOW() - INTERVAL '30 days'"; /// caller-supplied point in time rather than to `NOW()`. const GRACE_LOOKBACK: SignedDuration = SignedDuration::from_hours(24 * 30); -fn server_label(s: &Application) -> String { - s.name - .clone() - .or_else(|| s.host.as_ref().map(|h| h.0.to_string())) - .unwrap_or_else(|| s.id.to_string()) -} - fn machine_label(m: &crate::machines::Machine) -> String { m.name.clone().unwrap_or_else(|| m.id.to_string()) } @@ -381,7 +374,7 @@ impl Status { for server in &swept { let graded = grade_reachability( "Application", - &server_label(server), + &server.label(), server.alert_when_down_for.0, expected.get(&server.id).map(Vec::as_slice).unwrap_or(&[]), status_map.get(&server.id).copied(), diff --git a/crates/database/src/versions.rs b/crates/database/src/versions.rs index 01d87c9d..3e7f8b76 100644 --- a/crates/database/src/versions.rs +++ b/crates/database/src/versions.rs @@ -168,6 +168,58 @@ impl Version { .map_err(AppError::from) } + /// The release rows for these exact versions, in one query. A version with + /// no row is absent from the result rather than an error. + pub async fn get_by_versions( + db: &mut AsyncPgConnection, + wanted: &[VersionStr], + ) -> Result> { + use crate::schema::versions::dsl::*; + + if wanted.is_empty() { + return Ok(Vec::new()); + } + + // The SQL narrows on each component and the triple is matched here: one + // predicate per version builds a boxed OR chain as long as the fleet's + // version spread, while three set predicates leave Postgres a cross + // product small enough to sift in memory. + let mut majors: Vec = wanted.iter().map(|want| want.0.major as i32).collect(); + majors.sort_unstable(); + majors.dedup(); + + let mut minors: Vec = wanted.iter().map(|want| want.0.minor as i32).collect(); + minors.sort_unstable(); + minors.dedup(); + + let mut patches: Vec = wanted.iter().map(|want| want.0.patch as i32).collect(); + patches.sort_unstable(); + patches.dedup(); + + let rows: Vec = versions + .filter(major.eq_any(majors)) + .filter(minor.eq_any(minors)) + .filter(patch.eq_any(patches)) + .select(Version::as_select()) + .load(db) + .await + .map_err(AppError::from)?; + + Ok(rows + .into_iter() + .filter(|row| { + wanted.iter().any(|want| { + (row.major, row.minor, row.patch) + == ( + want.0.major as i32, + want.0.minor as i32, + want.0.patch as i32, + ) + }) + }) + .collect()) + } + pub async fn get_by_id(db: &mut AsyncPgConnection, version_id: Uuid) -> Result { use crate::schema::versions::dsl::*; diff --git a/crates/database/tests/it/main.rs b/crates/database/tests/it/main.rs index bde25560..95196042 100644 --- a/crates/database/tests/it/main.rs +++ b/crates/database/tests/it/main.rs @@ -49,6 +49,7 @@ mod reachability_silence_migration; mod reachability_sweep; mod recovery_vault; mod reported_detail; +mod reporting_schemas; mod restore; mod rotation_interlock; mod scope; diff --git a/crates/database/tests/it/reporting_schemas.rs b/crates/database/tests/it/reporting_schemas.rs new file mode 100644 index 00000000..db883650 --- /dev/null +++ b/crates/database/tests/it/reporting_schemas.rs @@ -0,0 +1,1008 @@ +//! Pair derivation and settling for reporting schemas. +//! +//! spec: RPT + +use commons_tests::db::TestDb; +use database::{ + diesel_async::AsyncPgConnection, + reporting_schemas::{ + NewReportingSchemaBuild, PairState, ReportingSchemaBuild, ReportingSchemaRequest, + pairs_for_group, versions_for_group, + }, + restore::NewBackupRestoreCheck, +}; +use diesel_async::SimpleAsyncConnection; +use uuid::Uuid; + +const GROUP: &str = "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa"; +const MACHINE: &str = "bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb"; +const CENTRAL: &str = "cccccccc-cccc-cccc-cccc-cccccccccccc"; +const FACILITY: &str = "dddddddd-dddd-dddd-dddd-dddddddddddd"; +const CONSUMER: &str = "eeeeeeee-eeee-eeee-eeee-eeeeeeeeeeee"; + +/// A group with a central on 2.60.0 and a facility on 2.59.0, both published. +async fn seed(conn: &mut AsyncPgConnection) -> (Uuid, Uuid) { + conn.batch_execute(&format!( + "INSERT INTO devices (id, role) VALUES ('{CONSUMER}', 'backup-restore'); + + INSERT INTO versions (id, major, minor, patch, changelog, status) VALUES + ('11111111-1111-1111-1111-111111111111', 2, 59, 0, '', 'published'), + ('22222222-2222-2222-2222-222222222222', 2, 60, 0, '', 'published'); + + INSERT INTO server_groups (id, name) VALUES ('{GROUP}', 'kamaka'); + + INSERT INTO machines (id, name, group_id) VALUES ('{MACHINE}', 'box', '{GROUP}'); + + INSERT INTO applications (id, type, name, host, machine_id, group_id) VALUES + ('{CENTRAL}', 'tamanu-central', 'central', 'https://c', '{MACHINE}', '{GROUP}'), + ('{FACILITY}', 'tamanu-facility', 'facility', 'https://f', '{MACHINE}', '{GROUP}'); + + INSERT INTO application_reported_detail (application_id, source, reported_at, version) VALUES + ('{CENTRAL}', 'tamanu', NOW(), '2.60.0'), + ('{FACILITY}', 'tamanu', NOW(), '2.59.0')", + )) + .await + .expect("seed"); + + ( + "11111111-1111-1111-1111-111111111111".parse().unwrap(), + "22222222-2222-2222-2222-222222222222".parse().unwrap(), + ) +} + +fn group() -> Uuid { + GROUP.parse().unwrap() +} + +/// A restore report for the pair, as a consumer would send one. +fn report_for( + outcome: commons_types::backup::RunOutcome, + healthy: bool, + error: Option, +) -> NewBackupRestoreCheck { + NewBackupRestoreCheck { + replica_id: None, + replica_name: None, + consumer_device_id: CONSUMER.parse().unwrap(), + group_id: group(), + machine_id: Some(MACHINE.parse().unwrap()), + r#type: "tamanu-postgres".parse().unwrap(), + intent: "reporting-schema".parse().unwrap(), + snapshot_id: Some("snap-1".to_owned()), + outcome, + error, + replica_healthy: healthy, + postgres_version: None, + observed_at: jiff::Timestamp::now(), + s3_sent_raw_bytes: None, + s3_sent_payload_bytes: None, + s3_received_raw_bytes: None, + s3_received_payload_bytes: None, + health_details: None, + run_id: None, + redaction_outcome: None, + redaction_manifest_version: None, + redaction_columns_masked: None, + redaction_columns_skipped: None, + redaction_error: None, + } +} + +/// Record a build against a throwaway restore report for the pair. +async fn record_build(conn: &mut AsyncPgConnection, version: Uuid, built: bool) { + record_build_for_run(conn, version, built, None).await; +} + +/// The same, for a build reported as a named run. +async fn record_build_for_run( + conn: &mut AsyncPgConnection, + version: Uuid, + built: bool, + run_id: Option, +) { + let report = NewBackupRestoreCheck { + replica_id: None, + replica_name: None, + consumer_device_id: CONSUMER.parse().unwrap(), + group_id: group(), + machine_id: Some(MACHINE.parse().unwrap()), + r#type: "tamanu-postgres".parse().unwrap(), + intent: "reporting-schema".parse().unwrap(), + snapshot_id: Some("snap-1".to_owned()), + outcome: commons_types::backup::RunOutcome::Success, + error: None, + replica_healthy: true, + postgres_version: None, + observed_at: jiff::Timestamp::now(), + s3_sent_raw_bytes: None, + s3_sent_payload_bytes: None, + s3_received_raw_bytes: None, + s3_received_payload_bytes: None, + health_details: None, + run_id, + redaction_outcome: None, + redaction_manifest_version: None, + redaction_columns_masked: None, + redaction_columns_skipped: None, + redaction_error: None, + }; + + ReportingSchemaBuild::record( + conn, + report, + NewReportingSchemaBuild { + group_id: group(), + version_id: version, + application_id: Some(CENTRAL.parse().unwrap()), + built, + error: (!built).then(|| "views did not compile".to_owned()), + artifact_ids: vec![], + }, + ) + .await + .expect("record build"); +} + +/// The pairs are every version the group's Tamanu applications report running. +/// A facility mid-rollout is on a different version from its central, and both +/// are pairs, because a schema follows the version rather than the application. +#[tokio::test(flavor = "multi_thread")] +async fn a_facility_on_its_own_version_is_a_pair_of_its_own() { + TestDb::run(|mut conn, _url| async move { + let (older, newer) = seed(&mut conn).await; + + let versions = versions_for_group(&mut conn, group()) + .await + .expect("derive versions"); + let ids: Vec = versions.iter().map(|v| v.id).collect(); + + assert!(ids.contains(&older), "the facility's version is a pair"); + assert!(ids.contains(&newer), "the central's version is a pair"); + }) + .await; +} + +/// A pair with no build is awaiting one; a built pair is settled; a failed +/// build settles it as firmly, since a build against a fixed version and +/// configuration fails the same way every time. +#[tokio::test(flavor = "multi_thread")] +async fn a_failed_build_settles_the_pair() { + TestDb::run(|mut conn, _url| async move { + let (_older, newer) = seed(&mut conn).await; + declare_builder(&mut conn, true).await; + + assert!( + !ReportingSchemaBuild::is_settled(&mut conn, group(), newer) + .await + .unwrap(), + "an untried pair is on the worklist" + ); + + record_build(&mut conn, newer, false).await; + + assert!( + ReportingSchemaBuild::is_settled(&mut conn, group(), newer) + .await + .unwrap(), + "a failure settles it as firmly as a pass" + ); + + let pairs = pairs_for_group(&mut conn, group()).await.expect("pairs"); + let failed = pairs.iter().find(|p| p.version_id == newer).unwrap(); + assert_eq!(failed.state, PairState::Failed); + assert_eq!(failed.error.as_deref(), Some("views did not compile")); + }) + .await; +} + +/// An operator asking for a build reinstates a settled pair, and the ask is +/// answered once the build lands. +#[tokio::test(flavor = "multi_thread")] +async fn an_operator_ask_reinstates_a_settled_pair() { + TestDb::run(|mut conn, _url| async move { + let (_older, newer) = seed(&mut conn).await; + declare_builder(&mut conn, true).await; + + record_build(&mut conn, newer, true).await; + assert!( + ReportingSchemaBuild::is_settled(&mut conn, group(), newer) + .await + .unwrap() + ); + + ReportingSchemaRequest::enqueue(&mut conn, group(), newer, Some("someone@bes.au")) + .await + .expect("enqueue"); + + assert!( + !ReportingSchemaBuild::is_settled(&mut conn, group(), newer) + .await + .unwrap(), + "an ask puts the pair back on the worklist" + ); + + let pairs = pairs_for_group(&mut conn, group()).await.expect("pairs"); + assert!( + pairs + .iter() + .find(|p| p.version_id == newer) + .unwrap() + .requested + ); + + // The build that answers the ask clears it. + record_build(&mut conn, newer, true).await; + assert!( + ReportingSchemaBuild::is_settled(&mut conn, group(), newer) + .await + .unwrap(), + "the ask is answered once the build it asked for lands" + ); + }) + .await; +} + +/// A build runs for half an hour and reports at the end of it. An ask entered +/// while it ran is for whatever changed after it began, so the build it asked +/// for is the next one. +#[tokio::test(flavor = "multi_thread")] +async fn an_ask_made_while_the_build_ran_stands() { + TestDb::run(|mut conn, _url| async move { + let (_older, newer) = seed(&mut conn).await; + declare_builder(&mut conn, true).await; + + const RUN: &str = "ffffffff-ffff-ffff-ffff-ffffffffffff"; + conn.batch_execute(&format!( + "INSERT INTO backup_credential_issuances + (device_id, group_id, type, issued_at, expires_at, purpose, + sts_assumed_role, bucket, prefix, run_id) + VALUES ('{CONSUMER}', '{GROUP}', 'tamanu-postgres', + now() - interval '30 minutes', now(), 'restore', + 'arn:test', 'b', '', '{RUN}')" + )) + .await + .expect("issue the run its credentials"); + + ReportingSchemaRequest::enqueue(&mut conn, group(), newer, Some("someone@bes.au")) + .await + .expect("enqueue"); + + record_build_for_run(&mut conn, newer, true, Some(RUN.parse().unwrap())).await; + + assert!( + !ReportingSchemaBuild::is_settled(&mut conn, group(), newer) + .await + .unwrap(), + "the ask stands until a build that began after it lands" + ); + }) + .await; +} + +/// A replica that failed to restore says nothing about whether the pair can be +/// built, so it records no build and the pair stays on the worklist. +#[tokio::test(flavor = "multi_thread")] +async fn a_failed_restore_leaves_the_pair_unsettled() { + TestDb::run(|mut conn, _url| async move { + let (_older, newer) = seed(&mut conn).await; + + let report = NewBackupRestoreCheck { + replica_id: None, + replica_name: None, + consumer_device_id: CONSUMER.parse().unwrap(), + group_id: group(), + machine_id: Some(MACHINE.parse().unwrap()), + r#type: "tamanu-postgres".parse().unwrap(), + intent: "reporting-schema".parse().unwrap(), + snapshot_id: Some("snap-1".to_owned()), + outcome: commons_types::backup::RunOutcome::Failure, + error: Some("replica never came up".to_owned()), + replica_healthy: false, + postgres_version: None, + observed_at: jiff::Timestamp::now(), + s3_sent_raw_bytes: None, + s3_sent_payload_bytes: None, + s3_received_raw_bytes: None, + s3_received_payload_bytes: None, + health_details: None, + run_id: None, + redaction_outcome: None, + redaction_manifest_version: None, + redaction_columns_masked: None, + redaction_columns_skipped: None, + redaction_error: None, + }; + + ReportingSchemaBuild::record( + &mut conn, + report, + NewReportingSchemaBuild { + group_id: group(), + version_id: newer, + application_id: Some(CENTRAL.parse().unwrap()), + built: false, + error: None, + artifact_ids: vec![], + }, + ) + .await + .expect("record"); + + assert!( + !ReportingSchemaBuild::is_settled(&mut conn, group(), newer) + .await + .unwrap(), + "an unhealthy restore is dispatched again rather than settling the pair" + ); + }) + .await; +} + +/// A schema built from a superseded release of a version is not the schema that +/// version describes, so registering an artifact against the version puts the +/// pair back on the worklist without an operator asking. +#[tokio::test(flavor = "multi_thread")] +async fn a_new_artifact_for_the_version_reinstates_the_pair() { + TestDb::run(|mut conn, _url| async move { + let (_older, newer) = seed(&mut conn).await; + + record_build(&mut conn, newer, true).await; + assert!( + ReportingSchemaBuild::is_settled(&mut conn, group(), newer) + .await + .unwrap(), + "a built pair settles" + ); + + conn.batch_execute(&format!( + "INSERT INTO artifacts (version_id, artifact_type, platform, download_url) + VALUES ('{}', 'migrations', 'any', 'https://example.com/m.tar')", + newer + )) + .await + .expect("register an artifact for the version"); + + assert!( + !ReportingSchemaBuild::is_settled(&mut conn, group(), newer) + .await + .unwrap(), + "the version's artifacts changed, so the pair is built again" + ); + }) + .await; +} + +/// Canopy resolves a range artifact for every version it covers, so one +/// registered over the pair's version is a change the next build reads and +/// reinstates the pair the same way an exact one does. +#[tokio::test(flavor = "multi_thread")] +async fn a_range_artifact_covering_the_version_reinstates_the_pair() { + TestDb::run(|mut conn, _url| async move { + let (_older, newer) = seed(&mut conn).await; + + record_build(&mut conn, newer, true).await; + + conn.batch_execute( + "INSERT INTO artifacts + (version_range_pattern, artifact_type, platform, download_url) + VALUES ('2.60.x', 'migrations', 'any', 'https://example.com/m.tar')", + ) + .await + .expect("register a range artifact"); + + assert!( + !ReportingSchemaBuild::is_settled(&mut conn, group(), newer) + .await + .unwrap(), + "a range covering the version is one of its artifacts" + ); + }) + .await; +} + +/// A build's own output is not a change a build reads. Counted, a second +/// group's schema for the version unsettles the first group's pair, whose +/// rebuild unsettles the second, and neither pair ever settles: a restore and a +/// migrate per pass, forever. +#[tokio::test(flavor = "multi_thread")] +async fn a_group_s_own_schema_does_not_reinstate_the_pair() { + TestDb::run(|mut conn, _url| async move { + let (_older, newer) = seed(&mut conn).await; + let other = "12121212-1212-1212-1212-121212121212"; + + record_build(&mut conn, newer, true).await; + + conn.batch_execute(&format!( + "INSERT INTO server_groups (id, name) VALUES ('{other}', 'drifting'); + INSERT INTO artifacts + (version_id, artifact_type, platform, group_id, content_type, digest) + VALUES ('{newer}', 'reporting-schema', 'any', '{other}', + 'application/sql', sha256('CREATE VIEW ...'::bytea))", + )) + .await + .expect("another group registers its schema"); + + assert!( + ReportingSchemaBuild::is_settled(&mut conn, group(), newer) + .await + .unwrap(), + "a schema is another group's output, not a change to the version" + ); + }) + .await; +} + +/// A build records the artifacts it registered, so an operator can see what came +/// out of it rather than only that something did. +#[tokio::test(flavor = "multi_thread")] +async fn a_build_records_what_it_registered() { + TestDb::run(|mut conn, _url| async move { + let (_older, newer) = seed(&mut conn).await; + + let artifact: Uuid = "77777777-7777-7777-7777-777777777777".parse().unwrap(); + conn.batch_execute(&format!( + "INSERT INTO artifacts (id, version_id, artifact_type, platform, download_url) + VALUES ('{artifact}', '{newer}', 'reporting-schema', 'any', 'https://example.com/s.sql')" + )) + .await + .expect("seed artifact"); + + let report = report_for(commons_types::backup::RunOutcome::Success, true, None); + ReportingSchemaBuild::record( + &mut conn, + report, + NewReportingSchemaBuild { + group_id: group(), + version_id: newer, + application_id: Some(CENTRAL.parse().unwrap()), + built: true, + error: None, + artifact_ids: vec![artifact], + }, + ) + .await + .expect("record"); + + let build = ReportingSchemaBuild::latest_for_pair(&mut conn, group(), newer) + .await + .unwrap() + .expect("a build"); + assert_eq!(build.artifact_ids, vec![Some(artifact)]); + }) + .await; +} + +/// A declaration whose consumer advertises a schema-building intent, which is +/// what brings a group into the sweep at all. +async fn declare_builder(conn: &mut AsyncPgConnection, enabled: bool) { + conn.batch_execute(&format!( + "INSERT INTO restore_consumer_capabilities + (consumer_device_id, intent, description, semantics, params) + VALUES ('{CONSUMER}', 'reporting-schema', '', + '[\"check\",\"once\",\"migrate\",\"reporting-schema\"]'::jsonb, '[]'::jsonb); + + INSERT INTO restore_replicas + (consumer_device_id, group_id, type, intent, name, enabled, params, publishes_schemas) + VALUES ('{CONSUMER}', '{GROUP}', 'tamanu-postgres', 'reporting-schema', + 'kamaka-schemas', {enabled}, '{{}}'::jsonb, true)", + )) + .await + .expect("declare builder"); +} + +/// The reporting-schema issues standing against the group's central. +async fn schema_issues(conn: &mut AsyncPgConnection) -> Vec { + database::issues::Issue::list_by_source_ref( + conn, + database::statuses::CANOPY_SOURCE, + database::backup::refs::REPORTING_SCHEMA, + &[CENTRAL.parse().unwrap()], + ) + .await + .expect("list issues") +} + +/// A failed build files against the group's central application, carrying the +/// builder's own description, and grades a warning rather than a failure. +#[tokio::test(flavor = "multi_thread")] +async fn a_failed_build_warns_on_the_group_central() { + TestDb::run(|mut conn, _url| async move { + let (older, _newer) = seed(&mut conn).await; + declare_builder(&mut conn, true).await; + record_build(&mut conn, older, false).await; + + database::reporting_schemas::sweep(&mut conn) + .await + .expect("sweep"); + + let issues = schema_issues(&mut conn).await; + assert_eq!(issues.len(), 1, "one check per group, on its central"); + let issue = &issues[0]; + assert_eq!( + issue.effective_result, + Some(commons_types::status::CheckResult::Warning), + "a failed build is a warning, not a failure" + ); + assert!(issue.active); + assert!( + issue.message.contains("2.59.0") && issue.message.contains("views did not compile"), + "the builder's own description reaches the operator: {}", + issue.message + ); + }) + .await; +} + +/// The check does not escalate. A warning ceiling is what holds that: an +/// escalating flag is normalised away for anything below a failure, so pinning +/// the ceiling is what stops a schema nobody can build waking whoever is on +/// call for an application that is up and answering. +#[tokio::test(flavor = "multi_thread")] +async fn the_reporting_schema_check_cannot_escalate() { + TestDb::run(|mut conn, _url| async move { + let (older, _newer) = seed(&mut conn).await; + declare_builder(&mut conn, true).await; + record_build(&mut conn, older, false).await; + + database::reporting_schemas::sweep(&mut conn) + .await + .expect("sweep"); + + let policies = database::check_policies::CheckPolicy::get_across_namespaces( + &mut conn, + database::statuses::CANOPY_SOURCE, + database::backup::refs::REPORTING_SCHEMA, + ) + .await + .expect("read the policy"); + + assert!(!policies.is_empty(), "the filing seeds a policy"); + for policy in &policies { + assert_eq!( + policy.ceiling, + commons_types::status::CheckResult::Warning, + "a failed build tops out at a warning" + ); + assert!(!policy.escalates, "and so cannot escalate"); + } + + assert!( + !schema_issues(&mut conn).await[0].escalates, + "which the issue carries through" + ); + }) + .await; +} + +/// The check recovers when the pair that failed is built. +#[tokio::test(flavor = "multi_thread")] +async fn a_built_pair_grades_the_check_passed() { + TestDb::run(|mut conn, _url| async move { + let (older, _newer) = seed(&mut conn).await; + declare_builder(&mut conn, true).await; + + record_build(&mut conn, older, false).await; + database::reporting_schemas::sweep(&mut conn) + .await + .expect("sweep the failure"); + + record_build(&mut conn, older, true).await; + database::reporting_schemas::sweep(&mut conn) + .await + .expect("sweep the recovery"); + + let issues = schema_issues(&mut conn).await; + assert_eq!(issues.len(), 1); + assert_eq!( + issues[0].effective_result, + Some(commons_types::status::CheckResult::Passed), + "a built pair is not a finding" + ); + }) + .await; +} + +/// A group whose pairs are all built and has never had a finding gets no +/// passing row: a check filed for it seeds a catalog entry nothing ever graded. +#[tokio::test(flavor = "multi_thread")] +async fn a_group_that_never_failed_files_nothing() { + TestDb::run(|mut conn, _url| async move { + let (older, newer) = seed(&mut conn).await; + declare_builder(&mut conn, true).await; + record_build(&mut conn, older, true).await; + record_build(&mut conn, newer, true).await; + + database::reporting_schemas::sweep(&mut conn) + .await + .expect("sweep"); + + assert!(schema_issues(&mut conn).await.is_empty()); + }) + .await; +} + +/// A pair still awaiting its first build is not a failure: nothing has gone +/// wrong yet, and the worklist is what moves it along. +#[tokio::test(flavor = "multi_thread")] +async fn a_pair_awaiting_its_first_build_files_nothing() { + TestDb::run(|mut conn, _url| async move { + seed(&mut conn).await; + declare_builder(&mut conn, true).await; + + database::reporting_schemas::sweep(&mut conn) + .await + .expect("sweep"); + + assert!( + schema_issues(&mut conn).await.is_empty(), + "an unbuilt pair is not yet a finding" + ); + }) + .await; +} + +/// A group nothing builds schemas for owes none, so a disabled declaration +/// files nothing even where a build once failed. +#[tokio::test(flavor = "multi_thread")] +async fn a_disabled_declaration_takes_the_group_out_of_the_sweep() { + TestDb::run(|mut conn, _url| async move { + let (older, _newer) = seed(&mut conn).await; + declare_builder(&mut conn, false).await; + record_build(&mut conn, older, false).await; + + database::reporting_schemas::sweep(&mut conn) + .await + .expect("sweep"); + + assert!( + schema_issues(&mut conn).await.is_empty(), + "a group with no enabled builder is not owed a schema" + ); + }) + .await; +} + +/// An open check has to be closed when the group's last non-awaiting pair goes +/// away, or it stands forever against a group that owes nothing. +#[tokio::test(flavor = "multi_thread")] +async fn the_check_closes_once_the_group_owes_no_schema() { + TestDb::run(|mut conn, _url| async move { + let (older, _newer) = seed(&mut conn).await; + declare_builder(&mut conn, true).await; + record_build(&mut conn, older, false).await; + + database::reporting_schemas::sweep(&mut conn) + .await + .expect("sweep"); + assert_eq!( + schema_issues(&mut conn).await[0].effective_result, + Some(commons_types::status::CheckResult::Warning), + "the warning stands while the pair is failed" + ); + + conn.batch_execute("DELETE FROM reporting_schema_builds") + .await + .expect("drop the build"); + + database::reporting_schemas::sweep(&mut conn) + .await + .expect("sweep again"); + + let issues = schema_issues(&mut conn).await; + assert_eq!(issues.len(), 1, "the same check, regraded"); + assert_eq!( + issues[0].effective_result, + Some(commons_types::status::CheckResult::Passed), + "a group owed no schema is not a finding" + ); + assert!( + issues[0].message.contains("No reporting schema is owed"), + "the closing message says why: {}", + issues[0].message + ); + }) + .await; +} + +/// The check files on the group's central, so a group that has lost it has an +/// open finding nothing else regrades. +#[tokio::test(flavor = "multi_thread")] +async fn a_group_that_lost_its_central_still_recovers() { + TestDb::run(|mut conn, _url| async move { + let (older, _newer) = seed(&mut conn).await; + declare_builder(&mut conn, true).await; + record_build(&mut conn, older, false).await; + + database::reporting_schemas::sweep(&mut conn) + .await + .expect("sweep"); + assert_eq!( + schema_issues(&mut conn).await[0].effective_result, + Some(commons_types::status::CheckResult::Warning), + "the warning stands while the pair is failed" + ); + + conn.batch_execute(&format!( + "UPDATE applications SET deleted_at = NOW() WHERE id = '{CENTRAL}'; + DELETE FROM reporting_schema_builds" + )) + .await + .expect("retire the central"); + + database::reporting_schemas::sweep(&mut conn) + .await + .expect("sweep again"); + + let issues = schema_issues(&mut conn).await; + assert_eq!(issues.len(), 1, "the same check, regraded"); + assert_eq!( + issues[0].effective_result, + Some(commons_types::status::CheckResult::Passed), + "a finding open against a former central is still recovered" + ); + }) + .await; +} + +/// A group nothing builds schemas for is owed none, so it presents no pairs +/// even where its applications report published versions. Listing them would +/// offer an operator a build nothing will pick up. +#[tokio::test(flavor = "multi_thread")] +async fn a_group_with_no_builder_has_no_pairs() { + TestDb::run(|mut conn, _url| async move { + seed(&mut conn).await; + + assert!( + pairs_for_group(&mut conn, group()) + .await + .expect("pairs") + .is_empty(), + "no declaration covers the group, so it is owed no schema" + ); + + declare_builder(&mut conn, true).await; + + assert_eq!( + pairs_for_group(&mut conn, group()) + .await + .expect("pairs") + .len(), + 2, + "declaring a builder is what brings the pairs into being" + ); + }) + .await; +} + +/// A pair is unique per group and version. Two applications reporting one +/// version are one pair, so the worklist dispatches one restore rather than +/// one per application. +#[tokio::test(flavor = "multi_thread")] +async fn two_applications_on_one_version_are_one_pair() { + TestDb::run(|mut conn, _url| async move { + let (_older, newer) = seed(&mut conn).await; + + conn.batch_execute(&format!( + "UPDATE application_reported_detail SET version = '2.60.0' + WHERE application_id = '{FACILITY}'" + )) + .await + .expect("put the facility on the central's version"); + + let versions = versions_for_group(&mut conn, group()) + .await + .expect("derive versions"); + + assert_eq!( + versions.iter().filter(|v| v.id == newer).count(), + 1, + "one pair, not one per reporting application: {versions:?}" + ); + }) + .await; +} + +/// A build needs the version's migrations, which reach a builder as that +/// version's published artifacts. A version Canopy holds no published release +/// row for has none, so a server reporting one is not a pair however loudly it +/// reports it, and Canopy is not owed a schema it cannot build. +/// +/// spec: RPT#pairs +#[tokio::test(flavor = "multi_thread")] +async fn only_a_published_version_is_a_pair() { + TestDb::run(|mut conn, _url| async move { + let (older, newer) = seed(&mut conn).await; + + for status in ["draft", "yanked"] { + conn.batch_execute(&format!( + "UPDATE versions SET status = '{status}' WHERE id = '{older}'" + )) + .await + .expect("change the version's status"); + + let ids: Vec = versions_for_group(&mut conn, group()) + .await + .expect("derive versions") + .iter() + .map(|v| v.id) + .collect(); + + assert!(!ids.contains(&older), "a {status} version is not a pair"); + assert!(ids.contains(&newer), "the published one still is"); + } + }) + .await; +} + +/// A group is owed a schema for where it is going as well as where it is, so +/// an open plan's target is a pair before anything reports running it. A plan +/// that is no longer open is history and adds nothing: the group either got +/// there, in which case an application reports it, or it is not going. +/// +/// spec: RPT#pairs +#[tokio::test(flavor = "multi_thread")] +async fn an_open_plan_s_target_is_a_pair_and_a_closed_one_is_not() { + TestDb::run(|mut conn, _url| async move { + let (_older, newer) = seed(&mut conn).await; + + // Nothing reports the target: the group is on 2.59.0 throughout, and + // 2.60.0 is only where it is heading. + conn.batch_execute(&format!( + "UPDATE application_reported_detail SET version = '2.59.0'; + + INSERT INTO upgrade_plans (group_id, target_version_id, created_by) + VALUES ('{GROUP}', '{newer}', 'seed@bes.au')" + )) + .await + .expect("plan the upgrade"); + + let ids: Vec = versions_for_group(&mut conn, group()) + .await + .expect("derive versions") + .iter() + .map(|v| v.id) + .collect(); + assert!(ids.contains(&newer), "the plan's target is a pair: {ids:?}"); + + conn.batch_execute("UPDATE upgrade_plans SET met_at = NOW()") + .await + .expect("meet the plan"); + + let ids: Vec = versions_for_group(&mut conn, group()) + .await + .expect("derive versions") + .iter() + .map(|v| v.id) + .collect(); + assert!( + !ids.contains(&newer), + "a met plan is history, and nothing reports its target: {ids:?}" + ); + }) + .await; +} + +/// A settled pair stays settled when a newer snapshot arrives. Every other +/// `once` intent keys its settling to the snapshot and re-dispatches on a fresh +/// one; a schema follows the version and the group's configuration, so backing +/// the group up again is no reason to build it a second time. Keying this one +/// to the snapshot would rebuild every pair of every group on every backup. +/// +/// spec: RPT#pairs +#[tokio::test(flavor = "multi_thread")] +async fn a_newer_snapshot_does_not_unsettle_a_pair() { + TestDb::run(|mut conn, _url| async move { + let (_older, newer) = seed(&mut conn).await; + + record_build(&mut conn, newer, true).await; + + conn.batch_execute(&format!( + "INSERT INTO backup_runs + (id, device_id, machine_id, group_id, type, purpose, outcome, snapshot_id, reported_at) + VALUES (gen_random_uuid(), '{CONSUMER}', '{MACHINE}', '{GROUP}', 'tamanu-postgres', + 'backup', 'success', 'snap-later', NOW())" + )) + .await + .expect("a newer snapshot"); + + assert!( + ReportingSchemaBuild::is_settled(&mut conn, group(), newer) + .await + .unwrap(), + "the version and the configuration are unchanged, so nothing is owed" + ); + + // The pair does still come back for the one event that means the schema + // is stale, so the answer above is the rule rather than a function that + // has stopped moving. + ReportingSchemaRequest::enqueue(&mut conn, group(), newer, Some("ops@bes.au")) + .await + .expect("ask for a build"); + assert!( + !ReportingSchemaBuild::is_settled(&mut conn, group(), newer) + .await + .unwrap(), + "an operator asking still reinstates it" + ); + }) + .await; +} + +/// A pair names the applications on its version. One row stands for every +/// application running it, and a row that names none leaves an operator reading +/// a bare version string against a group of eight servers. +/// +/// spec: RPT#pairs +#[tokio::test(flavor = "multi_thread")] +async fn a_pair_names_the_applications_on_its_version() { + TestDb::run(|mut conn, _url| async move { + let (older, newer) = seed(&mut conn).await; + declare_builder(&mut conn, true).await; + + let pairs = pairs_for_group(&mut conn, group()).await.expect("pairs"); + let named = |version: Uuid| { + pairs + .iter() + .find(|p| p.version_id == version) + .map(|p| p.applications.clone()) + .expect("the pair") + }; + + assert_eq!(named(newer), vec!["central".to_owned()]); + assert_eq!(named(older), vec!["facility".to_owned()]); + + // Two applications on one version are one pair, and the row has to + // account for both of them rather than for whichever was read last. + conn.batch_execute(&format!( + "UPDATE application_reported_detail SET version = '2.60.0' + WHERE application_id = '{FACILITY}'" + )) + .await + .expect("move the facility onto the central's version"); + + let pairs = pairs_for_group(&mut conn, group()).await.expect("pairs"); + let both = pairs + .iter() + .find(|p| p.version_id == newer) + .expect("the pair"); + assert_eq!( + both.applications, + vec!["central".to_owned(), "facility".to_owned()] + ); + }) + .await; +} + +/// A version only an open plan contributes has nothing running it, so the pair +/// names no applications rather than borrowing the ones on another version. +/// +/// spec: RPT#pairs +#[tokio::test(flavor = "multi_thread")] +async fn a_planned_pair_names_no_applications() { + TestDb::run(|mut conn, _url| async move { + let (_older, newer) = seed(&mut conn).await; + declare_builder(&mut conn, true).await; + + conn.batch_execute(&format!( + "UPDATE application_reported_detail SET version = '2.59.0'; + + INSERT INTO upgrade_plans (group_id, target_version_id, created_by) + VALUES ('{GROUP}', '{newer}', 'seed@bes.au')" + )) + .await + .expect("plan the upgrade"); + + let pairs = pairs_for_group(&mut conn, group()).await.expect("pairs"); + let planned = pairs + .iter() + .find(|p| p.version_id == newer) + .expect("the plan's target is a pair"); + + assert!( + planned.applications.is_empty(), + "nothing runs it: {:?}", + planned.applications + ); + }) + .await; +} diff --git a/crates/database/tests/it/restore.rs b/crates/database/tests/it/restore.rs index 8d4d04cc..db4fb704 100644 --- a/crates/database/tests/it/restore.rs +++ b/crates/database/tests/it/restore.rs @@ -164,6 +164,7 @@ fn new_replica( overdue_after: None, params: serde_json::json!({}), redacts: false, + publishes_schemas: false, created_by: Some("op@example.com".into()), } } @@ -181,6 +182,7 @@ fn update_from(r: &RestoreReplica) -> RestoreReplicaUpdate { overdue_after: r.overdue_after, params: r.params.clone(), redacts: r.redacts, + publishes_schemas: r.publishes_schemas, enabled: r.enabled, } } diff --git a/crates/private-server/src/fns.rs b/crates/private-server/src/fns.rs index bfdb17b6..0942b831 100644 --- a/crates/private-server/src/fns.rs +++ b/crates/private-server/src/fns.rs @@ -16,6 +16,7 @@ pub mod machines; pub mod maintenance; pub mod mcp_tokens; pub mod migration_tests; +pub mod reporting_schemas; pub mod restore_replicas; pub mod self_alerts; pub mod server_groups; @@ -139,6 +140,7 @@ pub fn routes() -> OpenApiRouter { .nest("/issues", issues::routes()) .nest("/mcp_tokens", mcp_tokens::routes()) .nest("/migration_tests", migration_tests::routes()) + .nest("/reporting_schemas", reporting_schemas::routes()) .nest("/restore_replicas", restore_replicas::routes()) .nest("/self_alerts", self_alerts::routes()) .nest("/maintenance", maintenance::routes()) diff --git a/crates/private-server/src/fns/reporting_schemas.rs b/crates/private-server/src/fns/reporting_schemas.rs new file mode 100644 index 00000000..ed7b338d --- /dev/null +++ b/crates/private-server/src/fns/reporting_schemas.rs @@ -0,0 +1,106 @@ +use axum::Json; +use axum::extract::State; +use canopy_utoipa_axum::{router::OpenApiRouter, routes}; +use commons_errors::{AppError, ProblemDetailsSchema, Result}; +use commons_servers::tailscale_auth::{TailscaleAdmin, TailscaleUser}; +use database::reporting_schemas::{Pair, ReportingSchemaRequest}; +use serde::Deserialize; +use utoipa::ToSchema; +use uuid::Uuid; + +use crate::state::AppState; + +pub fn routes() -> OpenApiRouter { + OpenApiRouter::new() + .routes(routes!(for_group)) + .routes(routes!(build)) +} + +/// Request body for reading a group's pairs. +#[derive(Deserialize, ToSchema)] +pub struct PairsForGroupArgs { + /// The group to report on. + pub group_id: Uuid, +} + +/// Where each of a group's pairs of group and Tamanu version stands. +/// +/// One entry per published version the group's Tamanu applications report +/// running, plus the version its open plan moves it to, so whether a group's +/// applications can be offered the schema for the version they run or are +/// moving to is answered in one place. +// spec: RPT#alerting +#[utoipa::path( + post, + path = "/for_group", + operation_id = "reporting_schemas_for_group", + tag = "reporting_schemas", + security(("tailscale-admin" = [])), + request_body = PairsForGroupArgs, + responses( + (status = 200, description = "Pairs, one per version the group runs or is moving to.", body = Vec), + (status = 401, body = ProblemDetailsSchema), + (status = 403, body = ProblemDetailsSchema), + ), +)] +pub async fn for_group( + State(state): State, + _admin: TailscaleAdmin, + Json(args): Json, +) -> Result>> { + let mut conn = state.db_read.get().await?; + let pairs = database::reporting_schemas::pairs_for_group(&mut conn, args.group_id).await?; + Ok(Json(pairs)) +} + +/// Which pair to build. +#[derive(Deserialize, ToSchema)] +pub struct BuildPairArgs { + /// The group whose schema to build. + pub group_id: Uuid, + /// The Tamanu version to build it for. + pub version_id: Uuid, +} + +/// Ask for a pair's schema to be built. +/// +/// This is how a schema is refreshed after the group's configuration changes, +/// and how a settled pair is put back on the worklist: a build against a fixed +/// version and configuration fails the same way every time, so a failed pair +/// waits for this rather than retrying on its own. +// spec: RPT#pairs +#[utoipa::path( + post, + path = "/build", + operation_id = "reporting_schemas_build", + tag = "reporting_schemas", + security(("tailscale-admin" = [])), + request_body = BuildPairArgs, + responses( + (status = 200), + (status = 400, body = ProblemDetailsSchema), + (status = 401, body = ProblemDetailsSchema), + (status = 403, body = ProblemDetailsSchema), + ), +)] +pub async fn build( + State(state): State, + admin: TailscaleAdmin, + Json(args): Json, +) -> Result> { + let mut conn = state.db.get().await?; + let TailscaleAdmin(TailscaleUser { login, .. }) = admin; + + // An ask against a pair the group does not have is one nothing dispatches + // and nothing clears, so it would stand against the group for good. + let pairs = database::reporting_schemas::pairs_for_group(&mut conn, args.group_id).await?; + if !pairs.iter().any(|pair| pair.version_id == args.version_id) { + return Err(AppError::BadRequest( + "that group has no pair for that version".into(), + )); + } + + ReportingSchemaRequest::enqueue(&mut conn, args.group_id, args.version_id, Some(&login)) + .await?; + Ok(Json(())) +} diff --git a/crates/private-server/src/fns/restore_replicas.rs b/crates/private-server/src/fns/restore_replicas.rs index 2bc35f7b..124a8dde 100644 --- a/crates/private-server/src/fns/restore_replicas.rs +++ b/crates/private-server/src/fns/restore_replicas.rs @@ -94,6 +94,12 @@ pub struct RestoreReplicaView { /// True when the intent carries the `redact` semantic, so the declaration /// can be switched to redacting. pub can_redact: bool, + /// Whether this declaration's consumer may publish the group's reporting + /// schema. Only an operator sets it. + pub publishes_schemas: bool, + /// True when the intent carries the `reporting-schema` semantic, so the + /// declaration can be made the group's publisher. + pub can_publish_schemas: bool, /// Servers this declaration covers that cannot currently be redacted: /// either their product publishes no masking manifest, or the version /// they report has none published. Each is withheld from the worklist @@ -191,6 +197,12 @@ pub struct RestoreReplicasCreateArgs { /// to set. Defaults to false. #[serde(default)] pub redacts: bool, + /// Whether this consumer may publish the group's reporting schema. + /// Accepted only for a group-wide, non-redacting declaration whose intent + /// carries the `reporting-schema` semantic. Defaults to false, so a + /// consumer publishes only where an operator has said it may. + #[serde(default)] + pub publishes_schemas: bool, } /// Request to update an existing declaration. @@ -236,6 +248,11 @@ pub struct RestoreReplicasUpdateArgs { /// intent carrying the `redact` semantic. Defaults to false. #[serde(default)] pub redacts: bool, + /// Whether this consumer may publish the group's reporting schema. + /// Accepted only for a group-wide, non-redacting declaration whose intent + /// carries the `reporting-schema` semantic. Defaults to false. + #[serde(default)] + pub publishes_schemas: bool, /// Whether the declaration should be active. pub enabled: bool, } @@ -262,16 +279,36 @@ fn overdue_after_to_pg(overdue_after: Option<&str>) -> Result } /// Resolve human-unit strings in operator-supplied parameter values to their -/// raw stored form and validate them against the consumer's advertised schema -/// for `intent`. If the intent is not advertised (a gap) there is no schema to -/// resolve or check against, so the values are accepted as-is. +/// raw stored form, validate them against the consumer's advertised schema for +/// `intent`, and refuse a flag the declaration cannot carry. If the intent is +/// not advertised (a gap) there is no schema to resolve or check against, so +/// the values are accepted as-is. async fn normalized_params_for_intent( conn: &mut AsyncPgConnection, consumer_device_id: Uuid, intent: &RestoreIntent, params: &ParamValues, redacts: bool, + publishes_schemas: bool, + machine_id: Option, ) -> Result { + // What a build is dispatched for does not depend on the intent's descriptor, + // and the mark is unique per group: a declaration accepted with it here + // holds the group's only publisher slot while authorising nothing. + // spec: RPT#the-build-contract + if publishes_schemas { + if redacts { + return Err(AppError::BadRequest( + "a redacting declaration cannot publish a reporting schema".into(), + )); + } + if machine_id.is_some() { + return Err(AppError::BadRequest( + "a machine-scoped declaration cannot publish a reporting schema: a build is per group".into(), + )); + } + } + let descriptors = RestoreConsumerCapability::list_for_consumer(conn, consumer_device_id).await?; let Some(desc) = descriptors.iter().find(|d| &d.intent == intent) else { @@ -289,6 +326,16 @@ async fn normalized_params_for_intent( "intent {intent} cannot redact: it does not carry the `redact` semantic" ))); } + + // A schema is built per group from its canonical central, from data the + // masking manifest has not altered, so a declaration Canopy would never + // dispatch a build to cannot be the group's publisher either. + // spec: RPT#the-build-contract + if publishes_schemas && !desc.has_semantic(semantics::REPORTING_SCHEMA) { + return Err(AppError::BadRequest(format!( + "intent {intent} cannot publish a reporting schema: it does not carry the `reporting-schema` semantic" + ))); + } let params = if owns_masking { ¶ms .iter() @@ -393,6 +440,11 @@ async fn to_views( .get(&r.consumer_device_id) .and_then(|descs| descs.iter().find(|d| d.intent == r.intent)) .is_some_and(|d| d.has_semantic(semantics::REDACT)), + can_publish_schemas: caps + .get(&r.consumer_device_id) + .and_then(|descs| descs.iter().find(|d| d.intent == r.intent)) + .is_some_and(|d| d.has_semantic(semantics::REPORTING_SCHEMA)), + publishes_schemas: r.publishes_schemas, redacts: r.redacts, redaction_gaps: gaps.remove(&r.id).unwrap_or_default(), consumer_name: names.get(&r.consumer_device_id).cloned().flatten(), @@ -737,6 +789,8 @@ pub async fn create( &args.intent, &args.params, args.redacts, + args.publishes_schemas, + args.machine_id, ) .await?; let replica = RestoreReplica::create( @@ -751,6 +805,7 @@ pub async fn create( overdue_after: overdue_after_to_pg(args.overdue_after.as_deref())?, params: serde_json::to_value(¶ms).expect("params serialize"), redacts: args.redacts, + publishes_schemas: args.publishes_schemas, created_by: Some(admin.login), }, ) @@ -803,6 +858,8 @@ pub async fn update( &args.intent, &args.params, args.redacts, + args.publishes_schemas, + args.machine_id, ) .await?; let replica = RestoreReplica::update( @@ -818,6 +875,7 @@ pub async fn update( overdue_after: overdue_after_to_pg(args.overdue_after.as_deref())?, params: serde_json::to_value(¶ms).expect("params serialize"), redacts: args.redacts, + publishes_schemas: args.publishes_schemas, enabled: args.enabled, }, ) diff --git a/crates/private-server/src/fns/versions.rs b/crates/private-server/src/fns/versions.rs index cb38802a..20c0617e 100644 --- a/crates/private-server/src/fns/versions.rs +++ b/crates/private-server/src/fns/versions.rs @@ -10,7 +10,10 @@ use commons_servers::artifact_store; use commons_servers::tailscale_auth::{TailscaleAdmin, TailscaleUser}; use commons_types::version::{VersionStatus, VersionStr}; use database::{ - artifacts::{Artifact, NewArtifact, Scope, digest_of, parse_sri, parse_sri_opt, sri}, + artifacts::{ + Artifact, MAX_HELD_ARTIFACT_BYTES, NewArtifact, Scope, digest_of, parse_sri, parse_sri_opt, + sri, + }, server_groups::ServerGroup, version_known_issues::VersionKnownIssue, versions::Version, @@ -22,11 +25,6 @@ use uuid::Uuid; use crate::state::AppState; -/// Cap on the bytes Canopy will hold for one artifact. A reporting schema is a -/// SQL file; anything approaching this is not one, and the whole of it is held -/// in memory to be digested before it is stored. -const MAX_HELD_ARTIFACT_BYTES: usize = 32 * 1024 * 1024; - /// Header the SPA sets on an upload, which no cross-origin page can send /// without the browser preflighting the request first. const FETCH_HEADER: &str = "x-canopy-upload"; diff --git a/crates/private-server/src/openapi.rs b/crates/private-server/src/openapi.rs index 8fe8690e..be113d0e 100644 --- a/crates/private-server/src/openapi.rs +++ b/crates/private-server/src/openapi.rs @@ -31,6 +31,7 @@ use utoipa::{ (name = "mcp_tokens", description = "Bearer tokens for the public MCP mount."), (name = "upgrade_plans", description = "Where each group is going: the version it intends to move to, and when."), (name = "migration_tests", description = "Where each server stands against the version it would take next."), + (name = "reporting_schemas", description = "Which of a group's versions have a reporting schema built for them."), (name = "restore_replicas", description = "Managed restore replicas: capabilities, worklist, and health."), (name = "self_alerts", description = "Canopy's alerts about its own operation."), (name = "server_groups", description = "Application group management and group-level configuration."), diff --git a/crates/private-server/tests/it/main.rs b/crates/private-server/tests/it/main.rs index 7bb123bc..c45158bd 100644 --- a/crates/private-server/tests/it/main.rs +++ b/crates/private-server/tests/it/main.rs @@ -28,6 +28,7 @@ mod openapi_spec; mod operator_presence; mod private_statuses; mod provision_credential; +mod reporting_schemas; mod restore_replicas; mod server_version_distance; mod sql; diff --git a/crates/private-server/tests/it/reporting_schemas.rs b/crates/private-server/tests/it/reporting_schemas.rs new file mode 100644 index 00000000..e9b08fe5 --- /dev/null +++ b/crates/private-server/tests/it/reporting_schemas.rs @@ -0,0 +1,66 @@ +//! Asking for a pair's build. +//! +//! spec: RPT + +use commons_tests::diesel_async::{AsyncPgConnection, SimpleAsyncConnection}; +use uuid::Uuid; + +/// A group whose central reports 2.60.0, and a published 2.59.0 nothing runs. +async fn seed(conn: &mut AsyncPgConnection) -> (Uuid, Uuid, Uuid) { + let group = Uuid::new_v4(); + let machine = Uuid::new_v4(); + let central = Uuid::new_v4(); + let consumer = Uuid::new_v4(); + let ran = Uuid::new_v4(); + let unrun = Uuid::new_v4(); + + conn.batch_execute(&format!( + "INSERT INTO versions (id, major, minor, patch, changelog, status) VALUES + ('{ran}', 2, 60, 0, '', 'published'), + ('{unrun}', 2, 59, 0, '', 'published'); + + INSERT INTO server_groups (id, name) VALUES ('{group}', 'kamaka'); + INSERT INTO machines (id, group_id) VALUES ('{machine}', '{group}'); + INSERT INTO applications (id, type, name, host, machine_id, group_id) VALUES + ('{central}', 'tamanu-central', 'central', 'https://c', '{machine}', '{group}'); + INSERT INTO application_reported_detail (application_id, source, reported_at, version) + VALUES ('{central}', 'tamanu', NOW(), '2.60.0'); + + INSERT INTO devices (id, role) VALUES ('{consumer}', 'backup-restore'); + INSERT INTO restore_consumer_capabilities + (consumer_device_id, intent, description, semantics, params) + VALUES ('{consumer}', 'reporting-schema', '', + '[\"once\",\"migrate\",\"reporting-schema\"]'::jsonb, '[]'::jsonb); + INSERT INTO restore_replicas + (consumer_device_id, group_id, type, intent, name, enabled, params, publishes_schemas) + VALUES ('{consumer}', '{group}', 'tamanu-postgres', 'reporting-schema', 'builds', + true, '{{}}'::jsonb, true)" + )) + .await + .expect("seed"); + + (group, ran, unrun) +} + +/// A version the group neither runs nor is moving to is not one of its pairs, +/// and an ask against it would stand for good: nothing dispatches it and +/// nothing clears it. +#[tokio::test(flavor = "multi_thread")] +async fn an_ask_for_a_version_the_group_does_not_run_is_refused() { + commons_tests::server::run(async |mut conn, _public, private| { + let (group, ran, unrun) = seed(&mut conn).await; + + private + .post("/api/reporting_schemas/build") + .json(&serde_json::json!({ "group_id": group, "version_id": ran })) + .await + .assert_status_ok(); + + private + .post("/api/reporting_schemas/build") + .json(&serde_json::json!({ "group_id": group, "version_id": unrun })) + .await + .assert_status_bad_request(); + }) + .await; +} diff --git a/crates/private-server/tests/it/restore_replicas.rs b/crates/private-server/tests/it/restore_replicas.rs index 4b2d1920..4a31db38 100644 --- a/crates/private-server/tests/it/restore_replicas.rs +++ b/crates/private-server/tests/it/restore_replicas.rs @@ -284,6 +284,36 @@ async fn an_intent_that_cannot_redact_refuses_the_flag() { .await; } +/// The publisher mark is unique per group, and a machine-scoped or redacting +/// declaration is one no build is dispatched to. Accepting the mark on one +/// takes the group's only slot and leaves the operator unable to declare the +/// publisher that would work. +#[tokio::test(flavor = "multi_thread")] +async fn a_declaration_no_build_is_dispatched_to_cannot_publish() { + commons_tests::server::run(async |mut conn, _public, private| { + let group = insert_group(&mut conn).await; + let consumer = insert_consumer(&mut conn).await; + let server = insert_server(&mut conn, group).await; + + // The intent is one the consumer has not advertised, so the descriptor + // this would otherwise be checked against does not exist yet. + private + .post("/api/restore_replicas/create") + .json(&serde_json::json!({ + "consumer_device_id": consumer, + "group_id": group, + "machine_id": server, + "type": "tamanu-postgres", + "intent": "reporting-schema", + "name": "one-box-publisher", + "publishes_schemas": true, + })) + .await + .assert_status_bad_request(); + }) + .await; +} + /// A server whose product publishes no manifest is withheld from the /// worklist, so the operator is shown which of the declaration's replicas /// aren't being restored and why. diff --git a/crates/public-server/openapi.json b/crates/public-server/openapi.json index 108fa7cf..869ce8ce 100644 --- a/crates/public-server/openapi.json +++ b/crates/public-server/openapi.json @@ -105,6 +105,127 @@ ] } }, + "/artifacts/groups/{group}/{version}/{artifact_type}/{platform}": { + "post": { + "tags": [ + "artifacts" + ], + "summary": "Register a reporting schema for one group, carrying its bytes.", + "description": "Requires a device certificate whose restore declaration for the named group\nadvertises that it builds reporting schemas. The bytes travel on this\nconnection and Canopy holds them, so the builder is issued no credential to\nany store. The path names the group the artifact is for, the exact version\nit was built against, and the artifact's type and target platform.\n\nThe version must be one Canopy already holds: a build is dispatched for a\ngroup and version Canopy knows about, so a version that does not exist is\nrefused rather than drafted. A range pattern is refused for the same reason:\na schema follows the migrations one exact version applies.\n\nReturns the created artifact record.", + "operationId": "register_group_artifact", + "parameters": [ + { + "name": "group", + "in": "path", + "description": "Group the artifact is for.", + "required": true, + "schema": { + "type": "string", + "format": "uuid" + } + }, + { + "name": "version", + "in": "path", + "description": "Exact semver (e.g. `2.10.5`) the schema was built against.", + "required": true, + "schema": { + "type": "string" + } + }, + { + "name": "artifact_type", + "in": "path", + "description": "Must be `reporting-schema`: the authorisation is defined with that artifact.", + "required": true, + "schema": { + "type": "string" + } + }, + { + "name": "platform", + "in": "path", + "required": true, + "schema": { + "type": "string" + } + }, + { + "name": "run", + "in": "query", + "description": "The run that produced the artifact, where one produced it.", + "required": false, + "schema": { + "type": "string", + "format": "uuid" + } + } + ], + "requestBody": { + "description": "The artifact's bytes, which Canopy holds and records the digest of.", + "content": { + "application/octet-stream": { + "schema": { + "type": "array", + "items": { + "type": "integer", + "format": "int32", + "minimum": 0 + } + } + } + }, + "required": true + }, + "responses": { + "200": { + "description": "", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/Artifact" + } + } + } + }, + "400": { + "description": "", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ProblemDetailsSchema" + } + } + } + }, + "401": { + "description": "", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ProblemDetailsSchema" + } + } + } + }, + "403": { + "description": "", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ProblemDetailsSchema" + } + } + } + } + }, + "security": [ + { + "backup-restore-device": [] + } + ] + } + }, "/artifacts/{version}/{artifact_type}/{platform}": { "post": { "tags": [ @@ -2226,7 +2347,7 @@ "items": { "type": "string" }, - "description": "Behaviours this intent opts into. Recognised values are `check` (a\nhealth report is expected for each replica), `once` (a given snapshot\nis only ever dispatched to a replica once, rather than repeatedly\nuntil overdue), and `url` (a replica's health report includes a link\nto it). Unrecognised values are stored but have no effect." + "description": "Behaviours this intent opts into. Recognised values are `check` (a\nhealth report is expected for each replica), `once` (a given snapshot\nis only ever dispatched to a replica once, rather than repeatedly until\noverdue), `url` (a replica's health report includes a link to it),\n`migrate` (Canopy names a target version and the replica applies that\nversion's migrations), `redact` (the replica de-identifies the restored\ndata before serving it), and `reporting-schema` (the replica builds a\nTamanu reporting schema and registers it for the group). Unrecognised\nvalues are stored but have no effect, so a consumer may advertise ahead\nof Canopy support." } } }, @@ -2757,6 +2878,49 @@ } } }, + "ReportingSchemaArgs": { + "type": "object", + "description": "What a reporting-schema build reports beyond its replica's restore health.", + "required": [ + "built" + ], + "properties": { + "artifacts": { + "type": "array", + "items": { + "type": "string", + "format": "uuid" + }, + "description": "The artifacts the build registered, of which the schema is one." + }, + "built": { + "type": "boolean", + "description": "Whether a schema came out of the build." + }, + "error": { + "type": [ + "string", + "null" + ], + "description": "What went wrong, where the build failed." + }, + "target_version": { + "type": [ + "string", + "null" + ], + "description": "The version the schema was built for, as semver, echoed from the\nworklist entry's `target_version`." + }, + "target_version_id": { + "type": [ + "string", + "null" + ], + "format": "uuid", + "description": "The same version as the identifier, echoed from `target_version_id`.\nAccepted for a consumer that reports the identifier; omit it when\n`target_version` is sent." + } + } + }, "RequestCertificateArgs": { "type": "object", "description": "A request to certify a key for a name.", @@ -3163,6 +3327,17 @@ "format": "uuid", "description": "The declaration this report concerns, taken from the worklist entry's\n`replica_id`. Required: several replicas can share one group, machine,\ntype, and intent, so a report that named no declaration could not be\nattributed to one of them." }, + "reporting_schema": { + "oneOf": [ + { + "type": "null" + }, + { + "$ref": "#/components/schemas/ReportingSchemaArgs", + "description": "What a reporting-schema build produced, where the replica was restored\nfor one. Absent on any other report." + } + ] + }, "run_id": { "type": [ "string", diff --git a/crates/public-server/src/artifacts.rs b/crates/public-server/src/artifacts.rs index 8b64f32d..a5676c2a 100644 --- a/crates/public-server/src/artifacts.rs +++ b/crates/public-server/src/artifacts.rs @@ -1,15 +1,22 @@ use axum::{ Json, - extract::{Path, Query, State}, + extract::{DefaultBodyLimit, Path, Query, State}, }; use canopy_utoipa_axum::{router::OpenApiRouter, routes}; use commons_errors::{AppError, ProblemDetailsSchema, Result}; use commons_servers::device_auth::{AuthDevice, ReleaserDevice}; -use commons_types::version::{VersionStatus, VersionStr}; +use commons_types::{ + device::DeviceRole, + version::{VersionStatus, VersionStr}, +}; use database::{ Db, - artifacts::{Artifact as ArtifactRow, NewArtifact, Scope, location, parse_sri_opt, sri}, + artifacts::{ + Artifact as ArtifactRow, MAX_HELD_ARTIFACT_BYTES, NewArtifact, Scope, digest_of, location, + parse_sri_opt, sri, + }, machines::Machine, + restore::RestoreReplica, versions::{NewVersion, Version}, }; use diesel::SelectableHelper as _; @@ -91,8 +98,24 @@ pub(crate) async fn caller_scope( Ok(Scope::for_caller(machine.and_then(|m| m.group_id))) } +/// Body budget for a schema upload. Sized above the held-bytes cap so an +/// over-limit upload is the handler's structured refusal naming the limit, +/// rather than axum's plain-text 413. +const MAX_UPLOAD_BODY_BYTES: usize = MAX_HELD_ARTIFACT_BYTES + 64 * 1024; + +/// The artifact type a reporting-schema build publishes. +const REPORTING_SCHEMA_TYPE: &str = "reporting-schema"; + +/// The platform it publishes on. A schema follows the version's migrations +/// rather than anything about the machine reading it. +const SCHEMA_PLATFORM: &str = "any"; + pub fn routes() -> OpenApiRouter { - OpenApiRouter::new().routes(routes!(create)) + OpenApiRouter::new().routes(routes!(create)).merge( + OpenApiRouter::new() + .routes(routes!(register_for_group)) + .layer(DefaultBodyLimit::max(MAX_UPLOAD_BODY_BYTES)), + ) } /// Register a downloadable artifact for a version or version range. @@ -222,6 +245,189 @@ async fn create( Ok(Json(Artifact::offered(row, &base, &version))) } +/// Register a reporting schema for one group, carrying its bytes. +/// +/// Requires a device certificate whose restore declaration for the named group +/// advertises that it builds reporting schemas. The bytes travel on this +/// connection and Canopy holds them, so the builder is issued no credential to +/// any store. The path names the group the artifact is for, the exact version +/// it was built against, and the artifact's type and target platform. +/// +/// The version must be one Canopy already holds: a build is dispatched for a +/// group and version Canopy knows about, so a version that does not exist is +/// refused rather than drafted. A range pattern is refused for the same reason: +/// a schema follows the migrations one exact version applies. +/// +/// Returns the created artifact record. +#[utoipa::path( + post, + path = "/groups/{group}/{version}/{artifact_type}/{platform}", + operation_id = "register_group_artifact", + tag = "artifacts", + security(("backup-restore-device" = [])), + params( + ("group" = Uuid, Path, description = "Group the artifact is for."), + ("version" = String, Path, description = "Exact semver (e.g. `2.10.5`) the schema was built against."), + ("artifact_type" = String, Path, description = "Must be `reporting-schema`: the authorisation is defined with that artifact."), + ("platform" = String, Path), + ("run" = Option, Query, description = "The run that produced the artifact, where one produced it."), + ), + request_body(content = Vec, content_type = "application/octet-stream", description = "The artifact's bytes, which Canopy holds and records the digest of."), + responses( + (status = 200, body = Artifact), + (status = 400, body = ProblemDetailsSchema), + (status = 401, body = ProblemDetailsSchema), + (status = 403, body = ProblemDetailsSchema), + ), +)] +#[axum::debug_handler] +async fn register_for_group( + device: AuthDevice, + State(state): State, + Path((group, version, artifact_type, platform)): Path<(Uuid, String, String, String)>, + Query(named): Query, + headers: axum::http::HeaderMap, + body: axum::body::Bytes, +) -> Result> { + use node_semver::Version as SemverVersion; + + let mut db = state.db.get().await?; + let device_id = device.0.id; + + // What a schema builder is authorised for is the artifact its declaration + // names. Any other type or platform registered under it would displace the + // releaser's own for every machine in the group, and those machines fetch + // and run what they are offered. A schema is one artifact per version, so + // the platform it is published on is fixed too: left open, one builder + // registers a schema per platform and a group is offered every one of them. + // spec: ART#registration, RPT#the-build-contract + if artifact_type != REPORTING_SCHEMA_TYPE || platform != SCHEMA_PLATFORM { + return Err(AppError::BadRequest(format!( + "this registers a {REPORTING_SCHEMA_TYPE} on {SCHEMA_PLATFORM}, not a \ + {artifact_type} on {platform}" + ))); + } + + let authorised = device.0.role == DeviceRole::Admin + || RestoreReplica::authorizes_schema_artifacts(&mut db, device_id, group).await?; + if !authorised { + // Refused the same way whether the group exists or not, so the endpoint + // is not a directory of which groups have a builder. + return Err(AppError::AuthInsufficientPermissions { + required: "an enabled declaration building this group's artifacts".into(), + }); + } + + if body.len() > MAX_HELD_ARTIFACT_BYTES { + return Err(AppError::BadRequest(format!( + "artifact is larger than the {MAX_HELD_ARTIFACT_BYTES} byte limit" + ))); + } + if body.is_empty() { + return Err(AppError::BadRequest( + "a group-scoped artifact carries its bytes".into(), + )); + } + + // Provenance is what an operator reads to answer what produced the bytes, + // so a run already recorded for somebody else is not one this registration + // may name. + if let Some(run) = named.run + && RestoreReplica::run_claimed_elsewhere(&mut db, run, device_id, group).await? + { + return Err(AppError::BadRequest( + "the named run belongs to another consumer or group".into(), + )); + } + + // A schema follows the migrations one exact version applies, and Canopy + // resolves a range artifact for every version it covers. + // spec: RPT#the-build-contract + let semver = SemverVersion::parse(&version) + .map_err(|_| AppError::BadRequest("a reporting schema names an exact version".into()))?; + + // A build is dispatched for a pair whose version Canopy already holds, so + // this names one rather than drafting a release nobody has cut. + // spec: RPT#pairs + let version_row = match Version::get_by_version(&mut db, VersionStr(semver)).await { + Ok(version) => version, + Err(AppError::DatabaseQuery(diesel::result::Error::NotFound)) => { + return Err(AppError::BadRequest(format!( + "no version {version} to register a group-scoped artifact against" + ))); + } + Err(error) => return Err(error), + }; + + let content_type = headers + .get(axum::http::header::CONTENT_TYPE) + .and_then(|v| v.to_str().ok()) + .map(str::to_owned); + + // Canopy holds these bytes, so it records the digest of what it actually + // took in rather than one the registration claims for them. + // spec: ART#digests + let digest = digest_of(&body); + + let store = state + .artifacts + .as_ref() + .ok_or_else(commons_servers::artifact_store::unconfigured)?; + + let input = NewArtifact { + id: None, + version_id: Some(version_row.id), + platform, + artifact_type, + download_url: None, + device_id: Some(device_id), + version_range_pattern: None, + group_id: Some(group), + content_type, + digest: Some(digest), + run_id: named.run, + }; + + // Under the id the artifact already has where one is registered, so a rebuild + // lands where the build it replaces was. + // spec: ART#where-an-artifact-rests + let existing = ArtifactRow::id_for_identity(&mut db, &input).await?; + let artifact_id = existing.unwrap_or_else(Uuid::new_v4); + store.put(artifact_id, Vec::from(body)).await?; + + let row = match ArtifactRow::register( + &mut db, + NewArtifact { + id: Some(artifact_id), + ..input + }, + ) + .await + { + Ok(row) => row, + Err(refusal) => { + // Only bytes put under an id minted here are dropped: under one + // already registered they are the live artifact's. + if existing.is_none() + && let Err(err) = store.delete(artifact_id).await + { + tracing::error!(artifact = %artifact_id, "refused registration left its bytes: {err}"); + } + return Err(refusal); + } + }; + + let base = crate::versions::public_base_url(&headers); + Ok(Json(Artifact::offered(row, &base, &version))) +} + +/// What a group-scoped registration names beside the path. +#[derive(Debug, serde::Deserialize)] +struct GroupRegisterQuery { + /// The run that produced the artifact, where one produced it. + run: Option, +} + /// What a registration names beside the path. #[derive(Debug, serde::Deserialize)] struct RegisterQuery { diff --git a/crates/public-server/src/restore.rs b/crates/public-server/src/restore.rs index 4b1da674..c672f60a 100644 --- a/crates/public-server/src/restore.rs +++ b/crates/public-server/src/restore.rs @@ -28,6 +28,7 @@ use database::{ backups::{BackupRun, NewBackupCredentialIssuance, ServerGroupBackupConfig}, migration_tests::{self, MigrationTest, NewMigrationTest}, pg_duration::PgDuration, + reporting_schemas::{NewReportingSchemaBuild, ReportingSchemaBuild}, restore::{ BackupRestoreCheck, NewBackupRestoreCheck, RestoreConsumerCapability, RestoreReplica, }, @@ -182,6 +183,47 @@ pub struct WorklistEntry { pub target_version_id: Option, } +/// What dispatching a group's schema builds needs of the group itself. +/// +/// A build restores the group's canonical central and differs per pair only in +/// the version it migrates to, so this is the same for every declaration +/// covering the group. +// spec: RPT#the-build-contract +struct SchemaGroup { + machine_id: Uuid, + central_type: commons_types::server::app_type::ApplicationType, + versions: Vec, + settlement: database::reporting_schemas::Settlement, +} + +/// Resolve a group's central and pairs, or `None` where it has no central to +/// build from. +async fn resolve_schema_group( + conn: &mut database::diesel_async::AsyncPgConnection, + group_id: Uuid, + ranges: &database::artifacts::RangeChanges, +) -> Result> { + let members = database::applications::Application::list_live_in_group(conn, group_id).await?; + let Some(central) = database::server_groups::ServerGroup::canonical_central(&members) else { + return Ok(None); + }; + let central_type = central.r#type.clone(); + let machine = database::machines::Machine::get_by_id(conn, central.machine_id).await?; + + let versions = + database::reporting_schemas::versions_of_members(conn, group_id, &members).await?; + let settlement = + database::reporting_schemas::Settlement::for_group(conn, group_id, &versions, ranges) + .await?; + + Ok(Some(SchemaGroup { + machine_id: machine.id, + central_type, + versions, + settlement, + })) +} + /// Fetch the full set of replicas this device should maintain. /// /// Returns the device's complete desired state, computed fresh on every call: @@ -238,6 +280,16 @@ async fn worklist( // keys on. A group-wide and a machine-scoped declaration with different names // are two replicas of that machine, and both are dispatched. let mut seen: HashSet<(Uuid, String)> = HashSet::new(); + // A schema build is keyed on the pair, not the machine, so two declarations + // covering one group with schema-building intents would each emit the whole + // pair list: a restore and a migrate paid for twice per build. + let mut pairs: HashSet<(Uuid, Uuid)> = HashSet::new(); + // Everything a build's dispatch needs of a group: its canonical central, + // the versions its pairs cover, and where each pair stands. Resolved once + // per group rather than per declaration, and the absence of a central is + // cached too, since every restore consumer polls this on a schedule. + let mut schema_groups: HashMap> = HashMap::new(); + let mut range_changes: Option = None; // Per-group caches so a group referenced by several declarations is resolved // once: the latest produced snapshot per (machine, type), and the latest // healthy-verified snapshot per (machine, type, intent) for `once` suppression. @@ -290,11 +342,108 @@ async fn worklist( let once = descriptor.has_semantic(semantics::ONCE); let migrates = descriptor.has_semantic(semantics::MIGRATE); let owns_masking = descriptor.has_semantic(semantics::REDACT); + let builds_schema = descriptor.has_semantic(semantics::REPORTING_SCHEMA); let replica_values: ParamValues = serde_json::from_value(d.params.clone()).unwrap_or_default(); let params = resolve_params(&descriptor.params, &replica_values); let region = cfg.region.clone().unwrap_or_else(instance_default_region); + + // A build is dispatched per pair rather than per machine. The + // configuration a schema follows from is held centrally, so every pair + // of a group restores the same central's snapshot and differs only in + // the version it is migrated to. + // spec: RPT#the-build-contract + if builds_schema { + // A build nobody may publish the result of is a restore and a + // migrate spent for nothing, so the operator's flag gates dispatch + // as well as publishing. + // spec: RPT#the-build-contract + if !d.publishes_schemas { + continue; + } + + // Masking alters the configuration a schema follows from, so a + // redacting declaration builds nothing rather than building from a + // database that is no longer the group's. + if d.redacts { + continue; + } + + // A build restores the group's canonical central, so a declaration + // pinned to a machine names something this dispatch cannot honour. + // Retargeting it silently would build against a box the operator + // did not declare. + if d.machine_id.is_some() { + tracing::warn!( + replica = %d.id, + "a machine-scoped declaration builds no reporting schema; a build is per group" + ); + continue; + } + + // Sending the masking parameters unset is what tells a consumer not + // to redact, so an intent advertising both has to be told here as + // well rather than inheriting the defaults declared with it. + // spec: RST#the-masking-manifest + let params = if owns_masking { + masked_params(¶ms, None) + } else { + params.clone() + }; + + if !schema_groups.contains_key(&d.group_id) { + // The range artifacts a pair is held against are the same set for + // every group, so they are read once for the poll rather than + // once per group it covers. + if range_changes.is_none() { + range_changes = Some(database::artifacts::RangeChanges::load(&mut conn).await?); + } + let ranges = range_changes.as_ref().expect("loaded above"); + let resolved = resolve_schema_group(&mut conn, d.group_id, ranges).await?; + schema_groups.insert(d.group_id, resolved); + } + let Some(group) = &schema_groups[&d.group_id] else { + continue; + }; + + let latest = snapshots.get(&(group.machine_id, d.r#type.clone())); + + for version in &group.versions { + if !pairs.insert((d.group_id, version.id)) { + continue; + } + + if once && group.settlement.settled(version.id) { + continue; + } + + #[expect(deprecated, reason = "emitted for consumers on the earlier shape")] + out.push(WorklistEntry { + replica_id: d.id, + group_id: d.group_id, + machine_id: group.machine_id, + server_id: group.machine_id, + application_type: Some(group.central_type.clone()), + r#type: d.r#type.clone(), + intent: d.intent.clone(), + name: d.name.clone(), + overdue_after_seconds: d.overdue_after.map(|f| f.0.as_secs()), + params: params.clone(), + snapshot_id: latest.and_then(|r| r.snapshot_id.clone()), + snapshot_at: latest.map(|r| r.reported_at.to_string()), + storage: "s3".into(), + bucket: cfg.bucket.clone(), + prefix: cfg.prefix.clone(), + region: region.clone(), + target_version: Some(version.as_semver().to_string()), + target_version_id: Some(version.id), + }); + } + + continue; + } + for machine in machines { let key = (machine.id, d.name.clone()); if !seen.insert(key) { @@ -659,6 +808,10 @@ pub struct VerificationArgs { /// What the migrations did, for a report under a `migrate` intent. Omit for /// every other intent. pub migration: Option, + /// What a reporting-schema build produced, where the replica was restored + /// for one. Absent on any other report. + // spec: RPT#what-a-build-reports + pub reporting_schema: Option, /// What the masking manifest did, for a replica that redacts. Omit for a /// replica that doesn't. pub redaction: Option, @@ -747,6 +900,26 @@ pub struct MigrationArgs { pub timings: Vec, } +/// What a reporting-schema build reports beyond its replica's restore health. +// spec: RPT#what-a-build-reports +#[derive(Debug, Deserialize, ToSchema)] +pub struct ReportingSchemaArgs { + /// The version the schema was built for, as semver, echoed from the + /// worklist entry's `target_version`. + pub target_version: Option, + /// The same version as the identifier, echoed from `target_version_id`. + /// Accepted for a consumer that reports the identifier; omit it when + /// `target_version` is sent. + pub target_version_id: Option, + /// Whether a schema came out of the build. + pub built: bool, + /// What went wrong, where the build failed. + pub error: Option, + /// The artifacts the build registered, of which the schema is one. + #[serde(default)] + pub artifacts: Vec, +} + /// How long one migration took. #[derive(Debug, Deserialize, ToSchema)] pub struct MigrationTimingArgs { @@ -864,8 +1037,52 @@ async fn verification( redaction_error: args.redaction.as_ref().and_then(|r| r.error.clone()), }; - match args.migration { - Some(migration) => { + match (args.migration, args.reporting_schema) { + // A build rides the migrate pathway, so a report may carry both; the + // build is the one that settles the pair. + (_, Some(build)) => { + // A build report settles the pair: it stops the pair being + // dispatched again and clears an operator's ask. Nothing but a + // consumer authorised to publish the group's schema may say so, or + // a plain verify consumer settles a pair no schema was built for. + // spec: RPT#the-build-contract + if !RestoreReplica::authorizes_schema_artifacts( + &mut conn, + consumer_device_id, + args.group, + ) + .await? + { + return Err(AppError::AuthInsufficientPermissions { + required: "an enabled declaration building this group's schemas".into(), + }); + } + + let version_id = resolve_build_target(&mut conn, &build).await?; + // The build is held against the group's central application, which is + // the one whose database the schema followed from and the one the + // entry named. + // spec: RPT#alerting + let members = + database::applications::Application::list_live_in_group(&mut conn, args.group) + .await?; + let application_id = + database::server_groups::ServerGroup::canonical_central(&members).map(|a| a.id); + ReportingSchemaBuild::record( + &mut conn, + report, + NewReportingSchemaBuild { + group_id: args.group, + version_id, + application_id, + built: build.built, + error: build.error, + artifact_ids: build.artifacts, + }, + ) + .await?; + } + (Some(migration), None) => { let target_version_id = resolve_migration_target(&mut conn, &migration).await?; let application_id = resolve_migration_application(&mut conn, &migration, machine_id, target_version_id) @@ -877,7 +1094,7 @@ async fn verification( ) .await?; } - None => { + (None, None) => { BackupRestoreCheck::record_report(&mut conn, report).await?; } } @@ -908,6 +1125,26 @@ async fn resolve_migration_target( .ok_or_else(|| AppError::BadRequest("migration report names no target version".into())) } +/// Resolve the version a reporting-schema build is about. +/// +/// The semver is preferred, matching a migration report: it is what the entry +/// carried and what the builder actually built for. +async fn resolve_build_target( + conn: &mut AsyncPgConnection, + build: &ReportingSchemaArgs, +) -> Result { + if let Some(semver) = &build.target_version { + return Ok( + database::versions::Version::get_by_version(conn, semver.parse()?) + .await? + .id, + ); + } + build + .target_version_id + .ok_or_else(|| AppError::BadRequest("build report names no version".into())) +} + /// Resolve the application a migration report is about. /// /// The version under test is an application's candidate while the data is the diff --git a/crates/public-server/tests/it/artifact_scopes.rs b/crates/public-server/tests/it/artifact_scopes.rs index e3115060..958c8f3f 100644 --- a/crates/public-server/tests/it/artifact_scopes.rs +++ b/crates/public-server/tests/it/artifact_scopes.rs @@ -249,7 +249,7 @@ async fn a_releaser_cannot_register_for_a_group() { let response = public .post(&format!( - "/artifacts/2.60.0/reporting-schema/any?group={GROUP_A}" + "/artifacts/groups/{GROUP_A}/2.60.0/reporting-schema/any" )) .add_header("x-forwarded-client-cert", &format!("Cert={cert}")) .text("https://example.com/x.sql") @@ -641,27 +641,32 @@ async fn held_bytes_are_never_rendered_by_a_browser() { .await } -/// An operator device registers either kind, but the group-scoped path is not -/// reachable from this endpoint at all, so an admin naming a group is refused -/// exactly as a releaser is. Publishing into a group is the private server's. +/// An operator device registers either kind, and needs no declaration to name a +/// group: the authorisation a builder holds is what a component has instead of +/// being an operator, not a narrower form of it. // spec: ART#registration #[tokio::test(flavor = "multi_thread")] -async fn an_admin_device_cannot_register_for_a_group_here_either() { +async fn an_admin_device_registers_for_any_group() { commons_tests::server::run_with_device_auth( "admin", async |mut conn, cert, _device_id, public, _| { seed(&mut conn).await; - let refused = public + let scoped = public .post(&format!( - "/artifacts/2.60.0/reporting-schema/any?group={GROUP_A}" + "/artifacts/groups/{GROUP_A}/2.60.0/reporting-schema/any" )) .add_header("x-forwarded-client-cert", &format!("Cert={cert}")) - .text("https://example.com/x.sql") + .add_header("content-type", "application/sql") + .text("CREATE VIEW ...") .await; - assert_eq!(refused.status_code(), StatusCode::FORBIDDEN); + scoped.assert_status_ok(); + let scoped: serde_json::Value = scoped.json(); + assert!( + scoped["digest"].is_string(), + "the bytes are held, so Canopy digests them" + ); - // Unscoped, the same admin credential registers fine. let accepted = public .post("/artifacts/2.60.0/installer/windows") .add_header("x-forwarded-client-cert", &format!("Cert={cert}")) @@ -699,9 +704,9 @@ async fn a_registration_with_nothing_in_it_is_refused() { } let malformed = public - .post("/artifacts/2.60.0/installer/windows?group=not-a-uuid") + .post("/artifacts/groups/not-a-uuid/2.60.0/reporting-schema/any") .add_header("x-forwarded-client-cert", &format!("Cert={cert}")) - .text("https://example.com/x.exe") + .text("CREATE VIEW ...") .await; assert_eq!( malformed.status_code(), diff --git a/crates/public-server/tests/it/main.rs b/crates/public-server/tests/it/main.rs index d3d925e2..959851dc 100644 --- a/crates/public-server/tests/it/main.rs +++ b/crates/public-server/tests/it/main.rs @@ -19,6 +19,7 @@ mod mcp; mod names; mod openapi_spec; mod password; +mod reporting_schemas; mod restore; mod server_self; mod server_versions; diff --git a/crates/public-server/tests/it/reporting_schemas.rs b/crates/public-server/tests/it/reporting_schemas.rs new file mode 100644 index 00000000..59b5fbbf --- /dev/null +++ b/crates/public-server/tests/it/reporting_schemas.rs @@ -0,0 +1,1126 @@ +//! Dispatching reporting-schema builds, and who may publish what one produces. +//! +//! spec: RPT + +use axum::http::StatusCode; +use diesel_async::SimpleAsyncConnection; + +const GROUP: &str = "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa"; +const OTHER_GROUP: &str = "ffffffff-ffff-ffff-ffff-ffffffffffff"; +const MACHINE: &str = "bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb"; +const CENTRAL: &str = "cccccccc-cccc-cccc-cccc-cccccccccccc"; +const VERSION: &str = "22222222-2222-2222-2222-222222222222"; + +/// A group whose central runs 2.60.0, with a ready backup repo and a snapshot, +/// and a consumer device declared against it building reporting schemas. +async fn seed(conn: &mut database::diesel_async::AsyncPgConnection, consumer: uuid::Uuid) { + conn.batch_execute(&format!( + "INSERT INTO versions (id, major, minor, patch, changelog, status) + VALUES ('{VERSION}', 2, 60, 0, '', 'published'); + + INSERT INTO server_groups (id, name) VALUES + ('{GROUP}', 'kamaka'), ('{OTHER_GROUP}', 'drifting'); + + INSERT INTO machines (id, name, group_id) VALUES ('{MACHINE}', 'box', '{GROUP}'); + + INSERT INTO applications (id, type, name, host, machine_id, group_id) + VALUES ('{CENTRAL}', 'tamanu-central', 'central', 'https://c', '{MACHINE}', '{GROUP}'); + + INSERT INTO application_reported_detail (application_id, source, reported_at, version) + VALUES ('{CENTRAL}', 'tamanu', NOW(), '2.60.0'); + + INSERT INTO server_group_backup_config + (group_id, bucket, prefix, target_role_arn, maintenance_role_arn, repo_password_ref, status) + VALUES ('{GROUP}', 'b', 'p/', 'arn:t', 'arn:m', 'ref', 'ready'); + + INSERT INTO backup_runs + (id, device_id, machine_id, group_id, type, purpose, outcome, snapshot_id, reported_at) + VALUES (gen_random_uuid(), '{consumer}', '{MACHINE}', '{GROUP}', 'tamanu-postgres', 'backup', 'success', 'snap-1', NOW()); + + INSERT INTO restore_consumer_capabilities + (consumer_device_id, intent, description, semantics, params) + VALUES ('{consumer}', 'schema-build', 'builds schemas', + '[\"check\", \"once\", \"migrate\", \"reporting-schema\"]'::jsonb, '{{}}'::jsonb); + + INSERT INTO restore_replicas + (consumer_device_id, group_id, type, intent, name, enabled, publishes_schemas) + VALUES ('{consumer}', '{GROUP}', 'tamanu-postgres', 'schema-build', 'schemas', true, true)", + )) + .await + .expect("seed"); +} + +/// A build is dispatched per pair on the group's central machine, naming the +/// pair's version rather than the machine's own upgrade candidate. +#[tokio::test(flavor = "multi_thread")] +async fn a_build_is_dispatched_per_pair_on_the_central() { + commons_tests::server::run_with_device_auth( + "backup-restore", + async |mut conn, cert, device_id, public, _| { + seed(&mut conn, device_id).await; + + let response = public + .get("/restore-worklist") + .add_header("x-forwarded-client-cert", &format!("Cert={cert}")) + .await; + response.assert_status_ok(); + let entries: Vec = response.json(); + + let ours: Vec<&serde_json::Value> = entries + .iter() + .filter(|e| e["intent"] == "schema-build") + .collect(); + + assert_eq!(ours.len(), 1, "one entry for the group's one pair"); + assert_eq!(ours[0]["machine_id"], MACHINE, "restores the central's box"); + assert_eq!( + ours[0]["target_version"], "2.60.0", + "names the pair's version" + ); + assert_eq!(ours[0]["application_type"], "tamanu-central"); + }, + ) + .await +} + +/// A pair is dispatched once however many declarations cover its group. Each +/// entry costs a restore and a migrate, so a second declaration doubling the +/// list is paid for. +#[tokio::test(flavor = "multi_thread")] +async fn a_second_declaration_dispatches_no_second_build() { + commons_tests::server::run_with_device_auth( + "backup-restore", + async |mut conn, cert, device_id, public, _| { + seed(&mut conn, device_id).await; + + // A second publisher for the group is refused outright: what a + // builder registers replaces what was registered before it, so two + // would overwrite each other and which schema a machine ends up on + // would be whichever reported last. + // spec: RPT#the-build-contract + let second = conn + .batch_execute(&format!( + "INSERT INTO restore_replicas + (consumer_device_id, group_id, type, intent, name, enabled, publishes_schemas) + VALUES ('{device_id}', '{GROUP}', 'tamanu-postgres', 'schema-build', + 'schemas-weekly', true, true)" + )) + .await; + assert!(second.is_err(), "one publisher per group"); + + // A declaration of the same intent that does not publish is allowed, + // and dispatches nothing of its own. + conn.batch_execute(&format!( + "INSERT INTO restore_replicas + (consumer_device_id, group_id, type, intent, name, enabled, publishes_schemas) + VALUES ('{device_id}', '{GROUP}', 'tamanu-postgres', 'schema-build', + 'schemas-weekly', true, false)" + )) + .await + .expect("a non-publishing declaration"); + + let response = public + .get("/restore-worklist") + .add_header("x-forwarded-client-cert", &format!("Cert={cert}")) + .await; + response.assert_status_ok(); + let entries: Vec = response.json(); + + assert_eq!( + entries + .iter() + .filter(|e| e["intent"] == "schema-build") + .count(), + 1, + "one entry for the group's one pair" + ); + }, + ) + .await +} + +/// A build restores the group's canonical central, so a declaration pinned to a +/// machine names something this dispatch cannot honour. Retargeting it silently +/// would build against a box the operator did not declare. +#[tokio::test(flavor = "multi_thread")] +async fn a_machine_scoped_declaration_builds_no_schema() { + commons_tests::server::run_with_device_auth( + "backup-restore", + async |mut conn, cert, device_id, public, _| { + seed(&mut conn, device_id).await; + + conn.batch_execute(&format!( + "UPDATE restore_replicas SET machine_id = '{MACHINE}' + WHERE consumer_device_id = '{device_id}'" + )) + .await + .expect("pin the declaration to a machine"); + + let response = public + .get("/restore-worklist") + .add_header("x-forwarded-client-cert", &format!("Cert={cert}")) + .await; + response.assert_status_ok(); + let entries: Vec = response.json(); + + assert!( + entries.iter().all(|e| e["intent"] != "schema-build"), + "a build is per group, not per machine" + ); + }, + ) + .await +} + +/// `once` is keyed to the pair rather than the snapshot, so a pair that has been +/// built drops off the worklist and stays off while the snapshot moves on. +#[tokio::test(flavor = "multi_thread")] +async fn a_built_pair_drops_off_the_worklist() { + commons_tests::server::run_with_device_auth( + "backup-restore", + async |mut conn, cert, device_id, public, _| { + seed(&mut conn, device_id).await; + + // Record a build for the pair, riding a restore report as one does. + conn.batch_execute(&format!( + "INSERT INTO backup_restore_checks + (consumer_device_id, group_id, machine_id, type, intent, snapshot_id, + outcome, replica_healthy, observed_at, reported_at) + VALUES ('{device_id}', '{GROUP}', '{MACHINE}', 'tamanu-postgres', + 'schema-build', 'snap-1', 'success', true, NOW(), NOW()); + + INSERT INTO reporting_schema_builds (check_id, group_id, version_id, built) + SELECT id, '{GROUP}', '{VERSION}', true FROM backup_restore_checks + ORDER BY id DESC LIMIT 1", + )) + .await + .expect("record a build"); + + let response = public + .get("/restore-worklist") + .add_header("x-forwarded-client-cert", &format!("Cert={cert}")) + .await; + response.assert_status_ok(); + let entries: Vec = response.json(); + + assert!( + !entries.iter().any(|e| e["intent"] == "schema-build"), + "a built pair is settled and not dispatched again" + ); + + // A newer snapshot does not bring it back: the key is the pair. + conn.batch_execute(&format!( + "INSERT INTO backup_runs + (id, device_id, machine_id, group_id, type, purpose, outcome, snapshot_id, reported_at) + VALUES (gen_random_uuid(), '{device_id}', '{MACHINE}', '{GROUP}', 'tamanu-postgres', 'backup', 'success', 'snap-2', NOW())", + )) + .await + .expect("newer snapshot"); + + let response = public + .get("/restore-worklist") + .add_header("x-forwarded-client-cert", &format!("Cert={cert}")) + .await; + let entries: Vec = response.json(); + assert!( + !entries.iter().any(|e| e["intent"] == "schema-build"), + "a newer snapshot does not rebuild a schema the pair already has" + ); + }, + ) + .await +} + +/// Masking alters the configuration a schema follows from, so a declaration set +/// to redact builds nothing rather than building from a database that is no +/// longer the group's. +/// +/// spec: RPT#the-build-contract +#[tokio::test(flavor = "multi_thread")] +async fn a_redacting_declaration_builds_no_schema() { + commons_tests::server::run_with_device_auth( + "backup-restore", + async |mut conn, cert, device_id, public, _| { + seed(&mut conn, device_id).await; + + conn.batch_execute(&format!( + "UPDATE restore_replicas SET redacts = true + WHERE consumer_device_id = '{device_id}'" + )) + .await + .expect("set the declaration to redact"); + + let response = public + .get("/restore-worklist") + .add_header("x-forwarded-client-cert", &format!("Cert={cert}")) + .await; + response.assert_status_ok(); + let entries: Vec = response.json(); + + assert!( + !entries.iter().any(|e| e["intent"] == "schema-build"), + "a redacting declaration dispatches no build: {entries:?}" + ); + }, + ) + .await +} + +/// The configuration a schema follows from is held centrally, so every pair of +/// a group restores a central's snapshot. A group with no central has no +/// snapshot to build from, and dispatching against a facility would build a +/// schema from the wrong half of the deployment. +/// +/// spec: RPT#the-build-contract +#[tokio::test(flavor = "multi_thread")] +async fn a_group_with_no_central_dispatches_nothing() { + commons_tests::server::run_with_device_auth( + "backup-restore", + async |mut conn, cert, device_id, public, _| { + seed(&mut conn, device_id).await; + + conn.batch_execute(&format!( + "UPDATE applications SET type = 'tamanu-facility' WHERE id = '{CENTRAL}'" + )) + .await + .expect("leave the group with no central"); + + let response = public + .get("/restore-worklist") + .add_header("x-forwarded-client-cert", &format!("Cert={cert}")) + .await; + response.assert_status_ok(); + let entries: Vec = response.json(); + + assert!( + !entries.iter().any(|e| e["intent"] == "schema-build"), + "nothing to restore a central's snapshot from: {entries:?}" + ); + }, + ) + .await +} + +/// An intent may advertise `redact` alongside building schemas, and a schema is +/// built against the group's own data rather than a masked copy. Canopy owns +/// the masking parameters, and sending them unset is what tells a consumer not +/// to redact, so an entry carrying the defaults declared with the intent would +/// have the builder mask the very configuration it is reading. +/// +/// spec: RST#the-masking-manifest +#[tokio::test(flavor = "multi_thread")] +async fn a_schema_build_is_never_told_to_redact() { + commons_tests::server::run_with_device_auth( + "backup-restore", + async |mut conn, cert, device_id, public, _| { + seed(&mut conn, device_id).await; + + conn.batch_execute(&format!( + "UPDATE restore_consumer_capabilities + SET semantics = '[\"check\", \"once\", \"migrate\", \"reporting-schema\", \"redact\"]'::jsonb, + params = '{{\"redaction_manifest_url\": {{\"type\": \"text\", + \"default\": \"https://masks.example/{{version}}.yaml\"}}}}'::jsonb + WHERE consumer_device_id = '{device_id}'", + )) + .await + .expect("advertise redaction too"); + + let response = public + .get("/restore-worklist") + .add_header("x-forwarded-client-cert", &format!("Cert={cert}")) + .await; + response.assert_status_ok(); + let entries: Vec = response.json(); + + let ours: Vec<&serde_json::Value> = entries + .iter() + .filter(|e| e["intent"] == "schema-build") + .collect(); + assert_eq!(ours.len(), 1, "the pair is still dispatched"); + assert_eq!( + ours[0]["params"]["redaction_manifest_url"], + serde_json::Value::Null, + "the parameter is advertised, and sent unset" + ); + }, + ) + .await +} + +/// A builder registers artifacts for the group its declaration covers, and is +/// refused another's the same way it would be refused a group that does not +/// exist. +#[tokio::test(flavor = "multi_thread")] +async fn a_builder_publishes_only_for_its_own_group() { + commons_tests::server::run_with_device_auth( + "backup-restore", + async |mut conn, cert, device_id, public, _| { + seed(&mut conn, device_id).await; + + let ours = public + .post(&format!( + "/artifacts/groups/{GROUP}/2.60.0/reporting-schema/any" + )) + .add_header("x-forwarded-client-cert", &format!("Cert={cert}")) + .add_header("content-type", "application/sql") + .text("CREATE VIEW ...") + .await; + ours.assert_status_ok(); + + let theirs = public + .post(&format!( + "/artifacts/groups/{OTHER_GROUP}/2.60.0/reporting-schema/any" + )) + .add_header("x-forwarded-client-cert", &format!("Cert={cert}")) + .text("CREATE VIEW ...") + .await; + assert_eq!(theirs.status_code(), StatusCode::FORBIDDEN); + + let nowhere = public + .post( + "/artifacts/groups/99999999-9999-9999-9999-999999999999/2.60.0/reporting-schema/any", + ) + .add_header("x-forwarded-client-cert", &format!("Cert={cert}")) + .text("CREATE VIEW ...") + .await; + assert_eq!( + nowhere.status_code(), + theirs.status_code(), + "a group it is not authorised for and one that does not exist answer alike" + ); + }, + ) + .await +} + +/// A schema is registered against one exact version. Canopy resolves a range +/// artifact for every version it covers, so a range registration would hand a +/// server a schema built for a version it does not run. +#[tokio::test(flavor = "multi_thread")] +async fn a_schema_registered_against_a_range_is_refused() { + commons_tests::server::run_with_device_auth( + "backup-restore", + async |mut conn, cert, device_id, public, _| { + seed(&mut conn, device_id).await; + + let ranged = public + .post(&format!( + "/artifacts/groups/{GROUP}/2.60.x/reporting-schema/any" + )) + .add_header("x-forwarded-client-cert", &format!("Cert={cert}")) + .add_header("content-type", "application/sql") + .text("CREATE VIEW ...") + .await; + assert_eq!(ranged.status_code(), StatusCode::BAD_REQUEST); + }, + ) + .await +} + +/// A build is dispatched for a pair whose version Canopy already holds, so a +/// registration naming one it does not is refused. Drafting a release row for +/// it would put a builder's near-miss of a real version into the catalog every +/// machine reads. +#[tokio::test(flavor = "multi_thread")] +async fn a_schema_for_an_unknown_version_drafts_none() { + commons_tests::server::run_with_device_auth( + "backup-restore", + async |mut conn, cert, device_id, public, _| { + seed(&mut conn, device_id).await; + + let refused = public + .post(&format!( + "/artifacts/groups/{GROUP}/9999.0.0/reporting-schema/any" + )) + .add_header("x-forwarded-client-cert", &format!("Cert={cert}")) + .add_header("content-type", "application/sql") + .text("CREATE VIEW ...") + .await; + assert_eq!(refused.status_code(), StatusCode::BAD_REQUEST); + + let catalog = database::versions::Version::get_all_including_drafts(&mut conn) + .await + .expect("the version catalog"); + assert!( + !catalog.iter().any(|v| v.major == 9999), + "no release row is drafted for it" + ); + }, + ) + .await +} + +/// A declaration an operator has turned off does not authorise anything. It is +/// the enabled declaration that covers a group, so a builder whose declaration +/// is disabled is refused its own group's artifacts. +#[tokio::test(flavor = "multi_thread")] +async fn a_disabled_declaration_authorises_nothing() { + commons_tests::server::run_with_device_auth( + "backup-restore", + async |mut conn, cert, device_id, public, _| { + seed(&mut conn, device_id).await; + + conn.batch_execute(&format!( + "UPDATE restore_replicas SET enabled = false WHERE consumer_device_id = '{device_id}'" + )) + .await + .expect("disable the declaration"); + + let refused = public + .post(&format!( + "/artifacts/groups/{GROUP}/2.60.0/reporting-schema/any" + )) + .add_header("x-forwarded-client-cert", &format!("Cert={cert}")) + .add_header("content-type", "application/sql") + .text("CREATE VIEW ...") + .await; + + assert_eq!(refused.status_code(), StatusCode::FORBIDDEN); + }, + ) + .await +} + +/// Restoring for a group is not the same authority as building its schema. A +/// consumer whose declaration covers the group but whose intent advertises no +/// `reporting-schema` semantic is refused, so a verify or migrate consumer +/// cannot publish a schema for the group it already restores. +#[tokio::test(flavor = "multi_thread")] +async fn restoring_for_a_group_does_not_authorise_publishing_its_schema() { + commons_tests::server::run_with_device_auth( + "backup-restore", + async |mut conn, cert, device_id, public, _| { + seed(&mut conn, device_id).await; + + conn.batch_execute(&format!( + "UPDATE restore_consumer_capabilities + SET semantics = '[\"check\", \"once\", \"migrate\"]'::jsonb + WHERE consumer_device_id = '{device_id}'" + )) + .await + .expect("withdraw the semantic"); + + let refused = public + .post(&format!( + "/artifacts/groups/{GROUP}/2.60.0/reporting-schema/any" + )) + .add_header("x-forwarded-client-cert", &format!("Cert={cert}")) + .add_header("content-type", "application/sql") + .text("CREATE VIEW ...") + .await; + + assert_eq!(refused.status_code(), StatusCode::FORBIDDEN); + }, + ) + .await +} + +/// A consumer registers its own capability set, so the semantics an intent +/// carries are its own claim: a device declared for the group can put +/// `reporting-schema` back on its intent in one request. What the operator set +/// on the declaration is what decides, so the refusal stands. +/// +/// spec: RPT#the-build-contract +#[tokio::test(flavor = "multi_thread")] +async fn a_consumer_cannot_advertise_itself_into_publishing() { + commons_tests::server::run_with_device_auth( + "backup-restore", + async |mut conn, cert, device_id, public, _| { + seed(&mut conn, device_id).await; + + // An operator has this consumer restoring for the group, and has + // not made it the group's publisher. + conn.batch_execute(&format!( + "UPDATE restore_replicas SET publishes_schemas = false + WHERE consumer_device_id = '{device_id}'" + )) + .await + .expect("the operator has not granted publishing"); + + let readvertised = public + .post("/restore-capabilities") + .add_header("x-forwarded-client-cert", &format!("Cert={cert}")) + .json(&serde_json::json!({ + "intents": [{ + "intent": "schema-build", + "description": "builds schemas", + "semantics": ["check", "once", "migrate", "reporting-schema"], + "params": {}, + }], + })) + .await; + assert_eq!( + readvertised.status_code(), + StatusCode::NO_CONTENT, + "a consumer may advertise what it likes" + ); + + let refused = public + .post(&format!( + "/artifacts/groups/{GROUP}/2.60.0/reporting-schema/any" + )) + .add_header("x-forwarded-client-cert", &format!("Cert={cert}")) + .add_header("content-type", "application/sql") + .text("CREATE VIEW ...") + .await; + + assert_eq!( + refused.status_code(), + StatusCode::FORBIDDEN, + "advertising the semantic grants nothing" + ); + }, + ) + .await +} + +/// The flag is the operator's, and it is what the group's builds and the +/// operator page follow: a declaration without it is dispatched no build, so +/// Canopy never asks for one it would refuse to accept. +/// +/// spec: RPT#the-build-contract +#[tokio::test(flavor = "multi_thread")] +async fn a_declaration_that_does_not_publish_is_dispatched_no_build() { + commons_tests::server::run_with_device_auth( + "backup-restore", + async |mut conn, cert, device_id, public, _| { + seed(&mut conn, device_id).await; + + conn.batch_execute(&format!( + "UPDATE restore_replicas SET publishes_schemas = false + WHERE consumer_device_id = '{device_id}'" + )) + .await + .expect("withdraw publishing"); + + let response = public + .get("/restore-worklist") + .add_header("x-forwarded-client-cert", &format!("Cert={cert}")) + .await; + response.assert_status_ok(); + let entries: Vec = response.json(); + + assert!( + !entries.iter().any(|e| e["intent"] == "schema-build"), + "no build is dispatched for it: {entries:?}" + ); + }, + ) + .await +} + +/// A build report settles the pair: it stops the pair being dispatched again +/// and clears an operator's ask. A plain verify or migrate consumer declared +/// for the group can otherwise settle a pair no schema was ever built for, and +/// inject its own error string into the group's check. +#[tokio::test(flavor = "multi_thread")] +async fn restoring_for_a_group_does_not_authorise_settling_its_pairs() { + commons_tests::server::run_with_device_auth( + "backup-restore", + async |mut conn, cert, device_id, public, _| { + seed(&mut conn, device_id).await; + let replica = declaration_id(&mut conn).await; + + conn.batch_execute(&format!( + "UPDATE restore_consumer_capabilities + SET semantics = '[\"check\", \"once\", \"migrate\"]'::jsonb + WHERE consumer_device_id = '{device_id}'" + )) + .await + .expect("withdraw the semantic"); + + let refused = public + .post("/restore-verification") + .add_header("x-forwarded-client-cert", &format!("Cert={cert}")) + .json(&build_report( + replica, + serde_json::json!({ "target_version": "2.60.0", "built": true }), + )) + .await; + + assert_eq!(refused.status_code(), StatusCode::FORBIDDEN); + }, + ) + .await +} + +/// The artifacts route carries a body limit sized from the held-bytes cap, so a +/// schema past axum's 2 MiB default is taken in rather than answered with a +/// plain-text 413 for a limit sixteen times below the documented one. +#[tokio::test(flavor = "multi_thread")] +async fn a_schema_over_axum_s_default_is_taken_in() { + commons_tests::server::run_with_device_auth( + "backup-restore", + async |mut conn, cert, device_id, public, _| { + seed(&mut conn, device_id).await; + + let sql = "-- ".to_owned() + &"x".repeat(3 * 1024 * 1024); + let response = public + .post(&format!( + "/artifacts/groups/{GROUP}/2.60.0/reporting-schema/any" + )) + .add_header("x-forwarded-client-cert", &format!("Cert={cert}")) + .add_header("content-type", "application/sql") + .text(sql) + .await; + + response.assert_status_ok(); + }, + ) + .await +} + +/// A builder is authorised for the artifact its declaration names. Any other +/// type registered under that authority outranks the releaser's own for every +/// machine in the group, and those machines fetch and run what they are +/// offered. +#[tokio::test(flavor = "multi_thread")] +async fn a_builder_cannot_displace_the_group_s_installer() { + commons_tests::server::run_with_device_auth( + "backup-restore", + async |mut conn, cert, device_id, public, _| { + seed(&mut conn, device_id).await; + + let installer = public + .post(&format!( + "/artifacts/groups/{GROUP}/2.60.0/installer/windows" + )) + .add_header("x-forwarded-client-cert", &format!("Cert={cert}")) + .add_header("content-type", "application/octet-stream") + .text("MZ...") + .await; + assert_eq!(installer.status_code(), StatusCode::BAD_REQUEST); + + // The platform is fixed for the same reason the type is: offering + // dedupes per type and platform, so a schema per platform would + // have a group offered every one of them. + let other_platform = public + .post(&format!( + "/artifacts/groups/{GROUP}/2.60.0/reporting-schema/windows" + )) + .add_header("x-forwarded-client-cert", &format!("Cert={cert}")) + .add_header("content-type", "application/sql") + .text("CREATE VIEW ...") + .await; + assert_eq!(other_platform.status_code(), StatusCode::BAD_REQUEST); + + let schema = public + .post(&format!( + "/artifacts/groups/{GROUP}/2.60.0/reporting-schema/any" + )) + .add_header("x-forwarded-client-cert", &format!("Cert={cert}")) + .add_header("content-type", "application/sql") + .text("CREATE VIEW ...") + .await; + schema.assert_status_ok(); + }, + ) + .await +} + +/// Provenance a party can forge for itself answers nothing an operator asks of +/// it, so a run already recorded for another consumer is not one this +/// registration may name. A run Canopy has not seen is ordinary: the artifact +/// lands mid-restore, before the report of that restore does. +#[tokio::test(flavor = "multi_thread")] +async fn a_run_another_consumer_reported_cannot_be_claimed() { + commons_tests::server::run_with_device_auth( + "backup-restore", + async |mut conn, cert, device_id, public, _| { + seed(&mut conn, device_id).await; + + let run = "77777777-7777-7777-7777-777777777777"; + let stranger = "88888888-8888-8888-8888-888888888888"; + conn.batch_execute(&format!( + "INSERT INTO devices (id, role) VALUES ('{stranger}', 'backup-restore'); + INSERT INTO backup_runs + (id, device_id, group_id, machine_id, type, purpose, outcome, reported_at) + VALUES ('{run}', '{stranger}', '{GROUP}', '{MACHINE}', + 'tamanu-postgres', 'restore', 'success', now())" + )) + .await + .expect("another consumer's run"); + + let claimed = public + .post(&format!( + "/artifacts/groups/{GROUP}/2.60.0/reporting-schema/any?run={run}" + )) + .add_header("x-forwarded-client-cert", &format!("Cert={cert}")) + .add_header("content-type", "application/sql") + .text("CREATE VIEW ...") + .await; + assert_eq!(claimed.status_code(), StatusCode::BAD_REQUEST); + + let own = public + .post(&format!( + "/artifacts/groups/{GROUP}/2.60.0/reporting-schema/any?run=99999999-9999-9999-9999-999999999999" + )) + .add_header("x-forwarded-client-cert", &format!("Cert={cert}")) + .add_header("content-type", "application/sql") + .text("CREATE VIEW ...") + .await; + own.assert_status_ok(); + }, + ) + .await +} + +/// The declaration `seed` made, which a report has to name. +async fn declaration_id(conn: &mut database::diesel_async::AsyncPgConnection) -> uuid::Uuid { + use diesel::{QueryableByName, sql_query, sql_types}; + use diesel_async::RunQueryDsl; + + #[derive(QueryableByName)] + struct Row { + #[diesel(sql_type = sql_types::Uuid)] + id: uuid::Uuid, + } + + sql_query("SELECT id FROM restore_replicas LIMIT 1") + .get_result::(conn) + .await + .expect("the seeded declaration") + .id +} + +/// A builder's report of one run, with `build` as its reporting-schema block. +fn build_report(replica: uuid::Uuid, build: serde_json::Value) -> serde_json::Value { + serde_json::json!({ + "replica_id": replica, + "group": GROUP, + "machine_id": MACHINE, + "type": "tamanu-postgres", + "intent": "schema-build", + "snapshot_id": "snap-1", + "outcome": "success", + "replica_healthy": true, + "observed_at": "2026-09-07T00:00:00Z", + "reporting_schema": build, + }) +} + +/// The build a report carries settles the pair it names, and is held against +/// the group's central application, whose database the schema followed from. +#[tokio::test(flavor = "multi_thread")] +async fn a_build_report_settles_the_pair_it_names() { + commons_tests::server::run_with_device_auth( + "backup-restore", + async |mut conn, cert, device_id, public, _| { + seed(&mut conn, device_id).await; + let replica = declaration_id(&mut conn).await; + + let resp = public + .post("/restore-verification") + .add_header("x-forwarded-client-cert", &format!("Cert={cert}")) + .json(&build_report( + replica, + serde_json::json!({ "target_version": "2.60.0", "built": true }), + )) + .await; + resp.assert_status(StatusCode::NO_CONTENT); + + let build = database::reporting_schemas::ReportingSchemaBuild::latest_for_pair( + &mut conn, + GROUP.parse().unwrap(), + VERSION.parse().unwrap(), + ) + .await + .expect("read the build") + .expect("a build landed"); + + assert!(build.built); + assert_eq!( + build.application_id, + Some(CENTRAL.parse().unwrap()), + "held against the central, not the reporting device's own machine" + ); + }, + ) + .await +} + +/// A consumer may name the version by id rather than by semver, which is what +/// the worklist entry hands it. +#[tokio::test(flavor = "multi_thread")] +async fn a_build_report_may_name_its_version_by_id() { + commons_tests::server::run_with_device_auth( + "backup-restore", + async |mut conn, cert, device_id, public, _| { + seed(&mut conn, device_id).await; + let replica = declaration_id(&mut conn).await; + + let resp = public + .post("/restore-verification") + .add_header("x-forwarded-client-cert", &format!("Cert={cert}")) + .json(&build_report( + replica, + serde_json::json!({ "target_version_id": VERSION, "built": true }), + )) + .await; + resp.assert_status(StatusCode::NO_CONTENT); + + assert!( + database::reporting_schemas::ReportingSchemaBuild::is_settled( + &mut conn, + GROUP.parse().unwrap(), + VERSION.parse().unwrap(), + ) + .await + .expect("settled"), + ); + }, + ) + .await +} + +/// A build is for a pair, so a report that names no version cannot be +/// attributed to one and is refused rather than recorded against a guess. +#[tokio::test(flavor = "multi_thread")] +async fn a_build_report_naming_no_version_is_refused() { + commons_tests::server::run_with_device_auth( + "backup-restore", + async |mut conn, cert, device_id, public, _| { + seed(&mut conn, device_id).await; + let replica = declaration_id(&mut conn).await; + + let resp = public + .post("/restore-verification") + .add_header("x-forwarded-client-cert", &format!("Cert={cert}")) + .json(&build_report(replica, serde_json::json!({ "built": true }))) + .await; + + assert_eq!(resp.status_code(), StatusCode::BAD_REQUEST); + }, + ) + .await +} + +/// A build that produced nothing settles the pair too, carrying the builder's +/// own description of what went wrong. +#[tokio::test(flavor = "multi_thread")] +async fn a_failed_build_report_carries_its_description() { + commons_tests::server::run_with_device_auth( + "backup-restore", + async |mut conn, cert, device_id, public, _| { + seed(&mut conn, device_id).await; + let replica = declaration_id(&mut conn).await; + + let resp = public + .post("/restore-verification") + .add_header("x-forwarded-client-cert", &format!("Cert={cert}")) + .json(&build_report( + replica, + serde_json::json!({ + "target_version": "2.60.0", + "built": false, + "error": "views did not compile", + }), + )) + .await; + resp.assert_status(StatusCode::NO_CONTENT); + + let pairs = + database::reporting_schemas::pairs_for_group(&mut conn, GROUP.parse().unwrap()) + .await + .expect("pairs"); + let pair = pairs + .iter() + .find(|p| p.version == "2.60.0") + .expect("the pair"); + + assert_eq!(pair.state, database::reporting_schemas::PairState::Failed); + assert_eq!(pair.error.as_deref(), Some("views did not compile")); + }, + ) + .await +} + +/// A build rides the migrate pathway, so one run's report can carry both +/// blocks. The build is the one that settles the pair, and the migration +/// payload beside it is deliberately not recorded as a migration test. +#[tokio::test(flavor = "multi_thread")] +async fn a_report_carrying_both_records_only_the_build() { + commons_tests::server::run_with_device_auth( + "backup-restore", + async |mut conn, cert, device_id, public, _| { + use diesel::{QueryableByName, sql_query, sql_types}; + use diesel_async::RunQueryDsl; + + #[derive(QueryableByName)] + struct Count { + #[diesel(sql_type = sql_types::BigInt)] + count: i64, + } + + seed(&mut conn, device_id).await; + let replica = declaration_id(&mut conn).await; + + let mut body = build_report( + replica, + serde_json::json!({ "target_version": "2.60.0", "built": true }), + ); + body["migration"] = serde_json::json!({ + "target_version": "2.60.0", + "total_elapsed_seconds": 12, + "data_bytes_before": 1_000, + "data_bytes_after": 1_200, + "timings": [], + }); + + let resp = public + .post("/restore-verification") + .add_header("x-forwarded-client-cert", &format!("Cert={cert}")) + .json(&body) + .await; + resp.assert_status(StatusCode::NO_CONTENT); + + assert!( + database::reporting_schemas::ReportingSchemaBuild::is_settled( + &mut conn, + GROUP.parse().unwrap(), + VERSION.parse().unwrap(), + ) + .await + .expect("settled"), + "the build is what settles the pair" + ); + + let migrations = sql_query("SELECT COUNT(*) AS count FROM migration_tests") + .get_result::(&mut conn) + .await + .expect("count") + .count; + assert_eq!( + migrations, 0, + "the migration payload beside a build is not a migration test" + ); + }, + ) + .await +} + +/// A schema the builder registers is what the group's machines are later +/// offered, byte for byte, under the version it was built for. +/// +/// The one device stands in for both the builder and a machine of the group: +/// which credential may do which is settled by the refusals above and in +/// `artifact_scopes`, and what this asserts is that the bytes survive the trip +/// and that the listing's own `download_url` is the one that fetches them. +#[tokio::test(flavor = "multi_thread")] +async fn a_registered_schema_is_offered_back_byte_for_byte() { + commons_tests::server::run_with_device_auth( + "backup-restore", + async |mut conn, cert, device_id, public, _| { + seed(&mut conn, device_id).await; + conn.batch_execute(&format!( + "UPDATE machines SET device_id = '{device_id}' WHERE id = '{MACHINE}'" + )) + .await + .expect("enrol the machine"); + + let sql = "CREATE VIEW reporting.encounters AS SELECT 1;"; + + public + .post(&format!( + "/artifacts/groups/{GROUP}/2.60.0/reporting-schema/any" + )) + .add_header("x-forwarded-client-cert", &format!("Cert={cert}")) + .add_header("content-type", "application/sql") + .text(sql) + .await + .assert_status_ok(); + + let listing = public + .get("/versions/2.60.0/artifacts") + .add_header("x-forwarded-client-cert", &format!("Cert={cert}")) + .await; + listing.assert_status_ok(); + + let artifacts: Vec = listing.json(); + let schema = artifacts + .iter() + .find(|a| a["artifact_type"] == "reporting-schema") + .expect("the group's schema is offered"); + + assert_eq!(schema["platform"], "any"); + assert_eq!( + schema["version_id"], VERSION, + "published against the exact version, not a range" + ); + assert!( + schema["version_range_pattern"].is_null(), + "a schema follows the migrations one version applies: {schema}" + ); + assert_eq!( + schema["digest"].as_str().expect("a digest"), + database::artifacts::sri(&database::artifacts::digest_of(sql.as_bytes())), + "the digest describes the bytes canopy took in" + ); + + // Follow the URL the listing handed out rather than rebuilding it, + // so the offer a device actually receives is what gets fetched. + let offered_url = schema["download_url"].as_str().expect("a download url"); + let path = offered_url + .split_once("/versions/") + .map(|(_, rest)| format!("/versions/{rest}")) + .expect("the offer names a versions path"); + + let download = public + .get(&path) + .add_header("x-forwarded-client-cert", &format!("Cert={cert}")) + .await; + download.assert_status_ok(); + assert_eq!(download.text(), sql); + }, + ) + .await +} + +/// A facility is offered the same schema as its group's centrals: a schema +/// follows the group and the version rather than the application it was built +/// from, and the build only ever runs against a central's snapshot. +#[tokio::test(flavor = "multi_thread")] +async fn a_facility_is_offered_the_same_schema_as_its_centrals() { + commons_tests::server::run_with_device_auth( + "machine", + async |mut conn, cert, device_id, public, _| { + // A builder of its own, since the authenticated device here is the + // facility's machine rather than the consumer that built the schema. + let consumer = uuid::Uuid::new_v4(); + conn.batch_execute(&format!( + "INSERT INTO devices (id, role) VALUES ('{consumer}', 'backup-restore')" + )) + .await + .expect("the builder device"); + seed(&mut conn, consumer).await; + + let digest = hex::encode(database::artifacts::digest_of(b"the group's schema")); + conn.batch_execute(&format!( + "INSERT INTO artifacts + (version_id, platform, artifact_type, group_id, content_type, digest) + VALUES ('{VERSION}', 'any', 'reporting-schema', '{GROUP}', + 'application/sql', '\\x{digest}'::bytea); + + INSERT INTO machines (id, name, group_id, device_id) + VALUES (gen_random_uuid(), 'facility-box', '{GROUP}', '{device_id}')" + )) + .await + .expect("seed the schema and a facility box"); + + let listing = public + .get("/versions/2.60.0/artifacts") + .add_header("x-forwarded-client-cert", &format!("Cert={cert}")) + .await; + listing.assert_status_ok(); + + let artifacts: Vec = listing.json(); + let schema = artifacts + .iter() + .find(|a| a["artifact_type"] == "reporting-schema") + .expect("a facility's device is offered its group's schema"); + + assert_eq!(schema["version_id"], VERSION); + }, + ) + .await +} diff --git a/migrations/2026-09-06-223010-0000_reporting_schema_builds/down.sql b/migrations/2026-09-06-223010-0000_reporting_schema_builds/down.sql new file mode 100644 index 00000000..50a79d7c --- /dev/null +++ b/migrations/2026-09-06-223010-0000_reporting_schema_builds/down.sql @@ -0,0 +1,2 @@ +DROP TABLE reporting_schema_requests; +DROP TABLE reporting_schema_builds; diff --git a/migrations/2026-09-06-223010-0000_reporting_schema_builds/up.sql b/migrations/2026-09-06-223010-0000_reporting_schema_builds/up.sql new file mode 100644 index 00000000..10df934c --- /dev/null +++ b/migrations/2026-09-06-223010-0000_reporting_schema_builds/up.sql @@ -0,0 +1,30 @@ +-- A reporting-schema build's result, hanging off the restore-health report that +-- carries its common fields, the way a migration test's does. Its own table +-- rather than nullable columns on the report, because the pair a build is for +-- is the whole point of it and a plain restore report has nothing to put there. +-- +-- A row here is what settles a pair: a build that failed settles it as firmly +-- as one that produced a schema, since a build against a fixed version and +-- configuration fails the same way every time. +CREATE TABLE reporting_schema_builds ( + check_id BIGINT PRIMARY KEY REFERENCES backup_restore_checks (id) ON DELETE CASCADE, + group_id UUID NOT NULL REFERENCES server_groups (id) ON DELETE CASCADE, + version_id UUID NOT NULL REFERENCES versions (id) ON DELETE CASCADE, + application_id UUID REFERENCES applications (id) ON DELETE SET NULL, + built BOOLEAN NOT NULL, + error TEXT +); + +-- Whether a pair is settled is the question the worklist asks on every pass. +CREATE INDEX reporting_schema_builds_pair ON reporting_schema_builds (group_id, version_id); + +-- An operator asking for a pair's build, which is how a schema is refreshed +-- after the group's configuration changes and how a settled pair is reinstated. +-- Keyed on the pair rather than the machine, because the pair is what is built. +CREATE TABLE reporting_schema_requests ( + group_id UUID NOT NULL REFERENCES server_groups (id) ON DELETE CASCADE, + version_id UUID NOT NULL REFERENCES versions (id) ON DELETE CASCADE, + requested_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW(), + requested_by TEXT, + PRIMARY KEY (group_id, version_id) +); diff --git a/migrations/2026-09-06-233313-0000_reporting_schema_build_artifacts/down.sql b/migrations/2026-09-06-233313-0000_reporting_schema_build_artifacts/down.sql new file mode 100644 index 00000000..0cc7b00d --- /dev/null +++ b/migrations/2026-09-06-233313-0000_reporting_schema_build_artifacts/down.sql @@ -0,0 +1 @@ +ALTER TABLE reporting_schema_builds DROP COLUMN artifact_ids, DROP COLUMN built_at; diff --git a/migrations/2026-09-06-233313-0000_reporting_schema_build_artifacts/up.sql b/migrations/2026-09-06-233313-0000_reporting_schema_build_artifacts/up.sql new file mode 100644 index 00000000..ae8a4f93 --- /dev/null +++ b/migrations/2026-09-06-233313-0000_reporting_schema_build_artifacts/up.sql @@ -0,0 +1,9 @@ +-- The artifacts a build registered, of which the schema is one. An array +-- rather than a side table: the list is short, only ever read whole, and has +-- no fields of its own to carry. +ALTER TABLE reporting_schema_builds ADD COLUMN artifact_ids UUID[] NOT NULL DEFAULT '{}'; + +-- A pair is settled against the artifacts the version had when it was built, +-- so a build carries when it happened without a join back to its report. +ALTER TABLE reporting_schema_builds + ADD COLUMN built_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW(); diff --git a/migrations/2026-09-08-221635-0000_artifacts_version_updated/down.sql b/migrations/2026-09-08-221635-0000_artifacts_version_updated/down.sql new file mode 100644 index 00000000..9ba33e11 --- /dev/null +++ b/migrations/2026-09-08-221635-0000_artifacts_version_updated/down.sql @@ -0,0 +1 @@ +DROP INDEX artifacts_version_updated; diff --git a/migrations/2026-09-08-221635-0000_artifacts_version_updated/up.sql b/migrations/2026-09-08-221635-0000_artifacts_version_updated/up.sql new file mode 100644 index 00000000..4cf00ff2 --- /dev/null +++ b/migrations/2026-09-08-221635-0000_artifacts_version_updated/up.sql @@ -0,0 +1,8 @@ +-- Whether a pair is settled asks when a version's artifacts last changed, once +-- per pair on every worklist poll. Without an index leading on version_id that +-- is a sequential scan of artifacts each time: artifacts_identity leads with +-- artifact_type, and artifacts_group_id with group_id. The partial predicate +-- matches the query, which counts the unscoped artifacts alone. +CREATE INDEX artifacts_version_updated + ON artifacts (version_id, updated_at) + WHERE group_id IS NULL; diff --git a/migrations/2026-09-09-014829-0000_replica_publishes_schemas/down.sql b/migrations/2026-09-09-014829-0000_replica_publishes_schemas/down.sql new file mode 100644 index 00000000..612ee218 --- /dev/null +++ b/migrations/2026-09-09-014829-0000_replica_publishes_schemas/down.sql @@ -0,0 +1 @@ +ALTER TABLE restore_replicas DROP COLUMN publishes_schemas; diff --git a/migrations/2026-09-09-014829-0000_replica_publishes_schemas/up.sql b/migrations/2026-09-09-014829-0000_replica_publishes_schemas/up.sql new file mode 100644 index 00000000..25612263 --- /dev/null +++ b/migrations/2026-09-09-014829-0000_replica_publishes_schemas/up.sql @@ -0,0 +1,10 @@ +-- ── Who may publish a group's reporting schema ────────────────────────────── +-- +-- Publishing a group-scoped artifact is a privilege over every machine in the +-- group: they are offered what is registered and they run it. The intent +-- semantics a consumer advertises are the consumer's own claim, registered by +-- the device itself, so they shape dispatch but cannot be what grants this. +-- An operator sets this flag on the declaration through the admin API, and it +-- is the whole of the authorisation. +ALTER TABLE restore_replicas + ADD COLUMN publishes_schemas BOOLEAN NOT NULL DEFAULT FALSE; diff --git a/migrations/2026-09-09-105133-0000_one_schema_publisher_per_group/down.sql b/migrations/2026-09-09-105133-0000_one_schema_publisher_per_group/down.sql new file mode 100644 index 00000000..7f0e90b1 --- /dev/null +++ b/migrations/2026-09-09-105133-0000_one_schema_publisher_per_group/down.sql @@ -0,0 +1 @@ +DROP INDEX restore_replicas_one_schema_publisher; diff --git a/migrations/2026-09-09-105133-0000_one_schema_publisher_per_group/up.sql b/migrations/2026-09-09-105133-0000_one_schema_publisher_per_group/up.sql new file mode 100644 index 00000000..ce84575f --- /dev/null +++ b/migrations/2026-09-09-105133-0000_one_schema_publisher_per_group/up.sql @@ -0,0 +1,11 @@ +-- ── One publisher per group ───────────────────────────────────────────────── +-- +-- What a builder registers is offered to every machine in the group, and a +-- registration replaces whatever is already registered for the same version, +-- type, platform and group. Two enabled declarations publishing for one group +-- are therefore both dispatched the same pairs and each overwrite the other's +-- schema, with which one a machine ends up on decided by whichever reported +-- last. The mark is the operator's, so the operator holds it to one. +CREATE UNIQUE INDEX restore_replicas_one_schema_publisher + ON restore_replicas (group_id) + WHERE publishes_schemas AND enabled; diff --git a/migrations/2026-09-10-004500-0000_restore_checks_by_run/down.sql b/migrations/2026-09-10-004500-0000_restore_checks_by_run/down.sql new file mode 100644 index 00000000..126d7c07 --- /dev/null +++ b/migrations/2026-09-10-004500-0000_restore_checks_by_run/down.sql @@ -0,0 +1 @@ +DROP INDEX backup_restore_checks_run; diff --git a/migrations/2026-09-10-004500-0000_restore_checks_by_run/up.sql b/migrations/2026-09-10-004500-0000_restore_checks_by_run/up.sql new file mode 100644 index 00000000..9d85714f --- /dev/null +++ b/migrations/2026-09-10-004500-0000_restore_checks_by_run/up.sql @@ -0,0 +1,7 @@ +-- A registration asks whether a run has already reported for somebody else, +-- which counts the checks carrying that run. The table is an audit trail kept +-- indefinitely and nothing else indexes run_id, so the count reads every row +-- ever reported. +CREATE INDEX backup_restore_checks_run + ON backup_restore_checks (run_id) + WHERE run_id IS NOT NULL; diff --git a/private-web/e2e/reporting-schemas.spec.ts b/private-web/e2e/reporting-schemas.spec.ts new file mode 100644 index 00000000..f9bd4ef9 --- /dev/null +++ b/private-web/e2e/reporting-schemas.spec.ts @@ -0,0 +1,288 @@ +import type { Sql } from "./seed"; +import { + resetSeededTables, + seedApplicationReport, + seedDevice, + seedReportingSchemaBuild, + seedRestoreConsumerCapability, + seedRestoreReplica, + seedServer, + seedServerGroup, + seedUpgradePlan, + seedVersion, +} from "./seed"; +import { expect, test } from "./test-fixtures"; + +/// A consumer that advertises a schema-building intent, declared against the +/// group and marked by an operator as publishing its schema. That mark is what +/// brings the group's pairs into being: canopy owes a schema only where an +/// operator has put something there to build one. +/// +/// spec: RPT#pairs +async function declareBuilder(sql: Sql, groupId: string): Promise { + const consumer = await seedDevice(sql, { role: "backup-restore" }); + await seedRestoreConsumerCapability(sql, { + deviceId: consumer.id, + intents: [ + { + intent: "reporting-schema", + semantics: ["check", "once", "migrate", "reporting-schema"], + }, + ], + }); + await seedRestoreReplica(sql, { + consumerDeviceId: consumer.id, + groupId, + intent: "reporting-schema", + name: "kamaka-schemas", + publishesSchemas: true, + }); + return consumer.id; +} + +/// How a group's reporting-schema pairs are presented, and how an operator asks +/// for one to be built. +/// +/// spec: RPT +test.describe("reporting schemas", () => { + test.beforeEach(async ({ sql }) => { + await resetSeededTables(sql); + }); + + /// One pair per version the group's Tamanu applications report running. A + /// facility mid-rollout is on a different version from its central, so both + /// are pairs: a schema follows the version rather than the application. + /// + /// spec: RPT#pairs + test("a group shows a pair per version its applications run", async ({ + page, + sql, + }) => { + await seedVersion(sql, { major: 2, minor: 59, patch: 0, status: "published" }); + await seedVersion(sql, { major: 2, minor: 60, patch: 0, status: "published" }); + const group = await seedServerGroup(sql, { name: "kamaka" }); + await declareBuilder(sql, group.id); + + const central = await seedServer(sql, { + name: "central", + groupId: group.id, + type: "tamanu-central", + }); + const facility = await seedServer(sql, { + name: "facility", + groupId: group.id, + type: "tamanu-facility", + }); + await seedApplicationReport(sql, { + applicationId: central.id, + version: "2.60.0", + }); + await seedApplicationReport(sql, { + applicationId: facility.id, + version: "2.59.0", + }); + + await page.goto(`/groups/${group.id}`); + + const section = page.getByTestId("reporting-schemas"); + await expect(section).toBeVisible(); + await expect( + section.getByRole("heading", { level: 2, name: "Reporting schemas" }), + ).toBeVisible(); + await expect(section.getByTestId("reporting-schema-row")).toHaveCount(2); + await expect(section.getByText("2.59.0")).toBeVisible(); + await expect(section.getByText("2.60.0")).toBeVisible(); + + // Nothing has been built yet, so both are awaiting one. + await expect(section.getByText("Awaiting build")).toHaveCount(2); + + // A version string alone does not say which servers the row covers, and + // the group page carries no running version anywhere else. + const older = section + .getByTestId("reporting-schema-row") + .filter({ hasText: "2.59.0" }); + await expect(older.getByText("1 server")).toBeVisible(); + await older.getByText("1 server").hover(); + await expect(page.getByRole("tooltip")).toHaveText("facility"); + }); + + /// An operator asking for a build is what reinstates a pair, so the ask has + /// to be visible once made. + /// + /// spec: RPT#pairs + test("asking for a build records the ask", async ({ page, sql }) => { + await seedVersion(sql, { major: 2, minor: 60, patch: 0, status: "published" }); + const group = await seedServerGroup(sql, { name: "kamaka" }); + await declareBuilder(sql, group.id); + const central = await seedServer(sql, { + name: "central", + groupId: group.id, + type: "tamanu-central", + }); + await seedApplicationReport(sql, { + applicationId: central.id, + version: "2.60.0", + }); + + await page.goto(`/groups/${group.id}`); + + const section = page.getByTestId("reporting-schemas"); + await section.getByRole("button", { name: "Build sooner" }).click(); + + await expect(section.getByText("Build asked for")).toBeVisible(); + + const asks = await sql.query("SELECT requested_by FROM reporting_schema_requests"); + expect(asks).toHaveLength(1); + }); + + /// A group nothing builds schemas for is owed none, so it shows no pairs + /// even where its applications report published versions. Listing them + /// would offer an operator a build nothing will pick up, and a row stuck on + /// "Awaiting build" reads as a backlog rather than as an absent builder. + /// + /// spec: RPT#pairs + test("a group with no builder declared shows no pairs", async ({ + page, + sql, + }) => { + await seedVersion(sql, { major: 2, minor: 60, patch: 0, status: "published" }); + const group = await seedServerGroup(sql, { name: "drifting" }); + const central = await seedServer(sql, { + name: "central", + groupId: group.id, + type: "tamanu-central", + }); + await seedApplicationReport(sql, { + applicationId: central.id, + version: "2.60.0", + }); + + await page.goto(`/groups/${group.id}`); + + const section = page.getByTestId("reporting-schemas"); + await expect(section).toBeVisible(); + await expect(section.getByTestId("reporting-schema-row")).toHaveCount(0); + await expect(section.getByText(/no builder is declared/i)).toBeVisible(); + }); + + /// A group is owed a schema for where it is going as well as where it is: + /// the version its open plan moves it to is a pair before anything runs it, + /// so the schema is there when the upgrade lands rather than being built + /// after it. + /// + /// spec: RPT#pairs + test("an open upgrade plan contributes a pair", async ({ page, sql }) => { + await seedVersion(sql, { major: 2, minor: 59, patch: 0, status: "published" }); + const target = await seedVersion(sql, { + major: 2, + minor: 60, + patch: 0, + status: "published", + }); + const group = await seedServerGroup(sql, { name: "kamaka" }); + await declareBuilder(sql, group.id); + + const central = await seedServer(sql, { + name: "central", + groupId: group.id, + type: "tamanu-central", + }); + await seedApplicationReport(sql, { + applicationId: central.id, + version: "2.59.0", + }); + await seedUpgradePlan(sql, { + groupId: group.id, + targetVersionId: target.id, + plannedFor: "2026-12-01", + }); + + await page.goto(`/groups/${group.id}`); + + const section = page.getByTestId("reporting-schemas"); + await expect(section.getByTestId("reporting-schema-row")).toHaveCount(2); + await expect(section.getByText("2.59.0")).toBeVisible(); + await expect(section.getByText("2.60.0")).toBeVisible(); + }); + + /// A built pair and a failed one read differently on the screen, and the + /// failed one carries the builder's own description, which is the only + /// place an operator can read why it failed. + /// + /// spec: RPT#presentation + test("a built pair and a failed one read differently", async ({ + page, + sql, + }) => { + const built = await seedVersion(sql, { + major: 2, + minor: 59, + patch: 0, + status: "published", + }); + const failed = await seedVersion(sql, { + major: 2, + minor: 60, + patch: 0, + status: "published", + }); + const group = await seedServerGroup(sql, { name: "kamaka" }); + const consumer = await declareBuilder(sql, group.id); + + const central = await seedServer(sql, { + name: "central", + groupId: group.id, + type: "tamanu-central", + }); + const facility = await seedServer(sql, { + name: "facility", + groupId: group.id, + type: "tamanu-facility", + }); + await seedApplicationReport(sql, { + applicationId: central.id, + version: "2.60.0", + }); + await seedApplicationReport(sql, { + applicationId: facility.id, + version: "2.59.0", + }); + + await seedReportingSchemaBuild(sql, { + consumerDeviceId: consumer, + groupId: group.id, + machineId: central.machineId, + applicationId: central.id, + versionId: built.id, + built: true, + }); + await seedReportingSchemaBuild(sql, { + consumerDeviceId: consumer, + groupId: group.id, + machineId: central.machineId, + applicationId: central.id, + versionId: failed.id, + built: false, + error: "views did not compile", + }); + + await page.goto(`/groups/${group.id}`); + + const section = page.getByTestId("reporting-schemas"); + await expect(section.getByText("Built", { exact: true })).toBeVisible(); + await expect(section.getByText("Failed", { exact: true })).toBeVisible(); + await expect( + section.getByText("Awaiting build", { exact: true }), + ).toHaveCount(0); + + // The description is only reachable by hovering the chip, which is the + // whole of an operator's access to why the build failed. + await section.getByText("Failed", { exact: true }).hover(); + await expect(page.getByText("views did not compile")).toBeVisible(); + + // A settled pair offers a rebuild rather than a first build. + await expect( + section.getByRole("button", { name: "Build again" }), + ).toHaveCount(2); + }); +}); diff --git a/private-web/e2e/restore-replicas.spec.ts b/private-web/e2e/restore-replicas.spec.ts index d79b4f55..58ff0700 100644 --- a/private-web/e2e/restore-replicas.spec.ts +++ b/private-web/e2e/restore-replicas.spec.ts @@ -178,6 +178,87 @@ test.describe("restore replicas", () => { expect(rows[0]?.redacts).toBe(true); }); + /** A consumer advertising an intent that builds reporting schemas. */ + async function schemaBuildingConsumer(sql: Sql): Promise { + const consumer = await seedDevice(sql, { role: "backup-restore" }); + await seedRestoreConsumerCapability(sql, { + deviceId: consumer.id, + intents: [ + { + intent: "schema-build", + semantics: ["check", "once", "migrate", "reporting-schema"], + }, + ], + }); + return consumer.id; + } + + /// Publishing a group's schema is the operator's grant, so it is set on the + /// declaration rather than followed from what the consumer advertises. + /// + /// spec: RPT#the-build-contract + test("an operator marks which declaration publishes the group's schema", async ({ + page, + sql, + }) => { + const consumer = await schemaBuildingConsumer(sql); + const groupId = await groupWithBackups(sql, "publish-declare"); + await seedServer(sql, { groupId, name: "publish-srv" }); + + await page.goto(`/fleet/groups/${groupId}/backups`); + await page.getByRole("button", { name: /declare replica/i }).click(); + + const dialog = page.getByRole("dialog"); + const publishes = dialog.getByRole("switch", { + name: /publish this group's reporting schema/i, + }); + await expect(publishes).not.toBeChecked(); + await publishes.check(); + await dialog.getByRole("button", { name: /^declare$/i }).click(); + + await expect(dialog).toHaveCount(0); + const rows = await sql.query<{ publishes_schemas: boolean }>( + `SELECT publishes_schemas FROM restore_replicas WHERE consumer_device_id = $1`, + [consumer], + ); + expect(rows[0]?.publishes_schemas).toBe(true); + + await expect( + page.getByRole("row", { name: /publish-declare/ }).first(), + ).toBeVisible(); + await expect(page.getByText("publishes schema").first()).toBeVisible(); + }); + + /// A build is dispatched per group from data the masking manifest has not + /// altered, so a declaration narrowed to one machine cannot be the group's + /// publisher. + /// + /// spec: RPT#the-build-contract + test("a machine-scoped declaration cannot publish the schema", async ({ + page, + sql, + }) => { + await schemaBuildingConsumer(sql); + const groupId = await groupWithBackups(sql, "publish-scope"); + await seedServer(sql, { groupId, name: "publish-one" }); + + await page.goto(`/fleet/groups/${groupId}/backups`); + await page.getByRole("button", { name: /declare replica/i }).click(); + + const dialog = page.getByRole("dialog"); + const publishes = dialog.getByRole("switch", { + name: /publish this group's reporting schema/i, + }); + await publishes.check(); + await expect(publishes).toBeChecked(); + + await dialog.getByLabel("Machine").click(); + await page.getByRole("option", { name: "publish-one" }).click(); + + await expect(publishes).toBeDisabled(); + await expect(publishes).not.toBeChecked(); + }); + test("a partial redaction shows against the report that carried it", async ({ page, sql, diff --git a/private-web/e2e/seed.ts b/private-web/e2e/seed.ts index ba10ee0d..0204b6fb 100644 --- a/private-web/e2e/seed.ts +++ b/private-web/e2e/seed.ts @@ -167,7 +167,7 @@ async function applicationTypeOf(sql: Sql, applicationId: string): Promise { await sql.query( - "TRUNCATE statuses, application_reported_detail, machine_reported_detail, issues, device_keys, applications, machines, server_groups, server_group_domains, devices, versions, tailscale_users, check_policies, scoped_check_policies, source_policies, server_group_backup_config, server_group_backup_schedule, machine_backup_capabilities, backup_requests, backup_runs, backup_run_progress, backup_repo_stats, backup_maintenance_runs, backup_credential_issuances, restore_replicas, restore_consumer_capabilities, backup_restore_checks, migration_tests, migration_timings, upgrade_plans, maintenance_windows, version_known_issues, recovery_vault_writes, application_names, application_certificates, compromised_keys RESTART IDENTITY CASCADE", + "TRUNCATE statuses, application_reported_detail, machine_reported_detail, issues, device_keys, applications, machines, server_groups, server_group_domains, devices, versions, tailscale_users, check_policies, scoped_check_policies, source_policies, server_group_backup_config, server_group_backup_schedule, machine_backup_capabilities, backup_requests, backup_runs, backup_run_progress, backup_repo_stats, backup_maintenance_runs, backup_credential_issuances, restore_replicas, restore_consumer_capabilities, backup_restore_checks, migration_tests, migration_timings, reporting_schema_builds, reporting_schema_requests, upgrade_plans, maintenance_windows, version_known_issues, recovery_vault_writes, application_names, application_certificates, compromised_keys RESTART IDENTITY CASCADE", ); // The truncate takes the migration-seeded nil "Canopy" application with // it; self-alerts attach to that row, so put it back. @@ -1475,6 +1475,8 @@ export async function seedRestoreReplica( enabled?: boolean; /** Whether the replica is served de-identified. */ redacts?: boolean; + /** Whether the operator has made this the group's schema publisher. */ + publishesSchemas?: boolean; }, ): Promise { const id = randomUUID(); @@ -1483,8 +1485,8 @@ export async function seedRestoreReplica( if (overdue == null) { await sql.query( `INSERT INTO restore_replicas - (id, consumer_device_id, group_id, machine_id, type, intent, name, params, enabled, redacts) - VALUES ($1, $2, $3, $4, $5, $6, $7, $8::jsonb, $9, $10)`, + (id, consumer_device_id, group_id, machine_id, type, intent, name, params, enabled, redacts, publishes_schemas) + VALUES ($1, $2, $3, $4, $5, $6, $7, $8::jsonb, $9, $10, $11)`, [ id, opts.consumerDeviceId, @@ -1496,13 +1498,14 @@ export async function seedRestoreReplica( params, opts.enabled ?? true, opts.redacts ?? false, + opts.publishesSchemas ?? false, ], ); } else { await sql.query( `INSERT INTO restore_replicas - (id, consumer_device_id, group_id, machine_id, type, intent, name, overdue_after, params, enabled, redacts) - VALUES ($1, $2, $3, $4, $5, $6, $7, make_interval(secs => $8), $9::jsonb, $10, $11)`, + (id, consumer_device_id, group_id, machine_id, type, intent, name, overdue_after, params, enabled, redacts, publishes_schemas) + VALUES ($1, $2, $3, $4, $5, $6, $7, make_interval(secs => $8), $9::jsonb, $10, $11, $12)`, [ id, opts.consumerDeviceId, @@ -1515,6 +1518,7 @@ export async function seedRestoreReplica( params, opts.enabled ?? true, opts.redacts ?? false, + opts.publishesSchemas ?? false, ], ); } @@ -1638,6 +1642,54 @@ export async function seedMigrationTest( } } +/** Seed a reporting-schema build: the restore-health report that carries the + * common fields, plus the build outcome hung off it. `built: false` with an + * `error` is what makes the pair read as failed; a build settles the pair + * either way. */ +export async function seedReportingSchemaBuild( + sql: Sql, + opts: { + consumerDeviceId: string; + groupId: string; + /** The machine whose snapshot the schema was built from. */ + machineId: string; + /** The group's central, which the build is held against. */ + applicationId?: string | null; + versionId: string; + snapshotId?: string; + built?: boolean; + error?: string | null; + /** Artifact ids the build registered, of which the schema is one. */ + artifactIds?: string[]; + }, +): Promise { + const built = opts.built ?? true; + const rows = await sql.query<{ id: string }>( + `INSERT INTO backup_restore_checks + (consumer_device_id, group_id, machine_id, type, intent, snapshot_id, outcome, + replica_healthy, observed_at) + VALUES ($1, $2, $3, 'tamanu-postgres', 'reporting-schema', $4, 'success', true, NOW()) + RETURNING id`, + [opts.consumerDeviceId, opts.groupId, opts.machineId, opts.snapshotId ?? "snap-1"], + ); + const checkId = rows[0]!.id; + + await sql.query( + `INSERT INTO reporting_schema_builds + (check_id, group_id, version_id, application_id, built, error, artifact_ids) + VALUES ($1, $2, $3, $4, $5, $6, $7::uuid[])`, + [ + checkId, + opts.groupId, + opts.versionId, + opts.applicationId ?? null, + built, + built ? null : (opts.error ?? "the build failed"), + opts.artifactIds ?? [], + ], + ); +} + /** Record where a group is going. `plannedFor` is `YYYY-MM-DD`; omit for a plan * with no date. */ export interface SeededMaintenanceWindow { diff --git a/private-web/openapi.json b/private-web/openapi.json index 0e49ae68..d5a5b4a0 100644 --- a/private-web/openapi.json +++ b/private-web/openapi.json @@ -6188,6 +6188,126 @@ ] } }, + "/api/reporting_schemas/build": { + "post": { + "tags": [ + "reporting_schemas" + ], + "summary": "Ask for a pair's schema to be built.", + "description": "This is how a schema is refreshed after the group's configuration changes,\nand how a settled pair is put back on the worklist: a build against a fixed\nversion and configuration fails the same way every time, so a failed pair\nwaits for this rather than retrying on its own.", + "operationId": "reporting_schemas_build", + "requestBody": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/BuildPairArgs" + } + } + }, + "required": true + }, + "responses": { + "200": { + "description": "" + }, + "400": { + "description": "", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ProblemDetailsSchema" + } + } + } + }, + "401": { + "description": "", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ProblemDetailsSchema" + } + } + } + }, + "403": { + "description": "", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ProblemDetailsSchema" + } + } + } + } + }, + "security": [ + { + "tailscale-admin": [] + } + ] + } + }, + "/api/reporting_schemas/for_group": { + "post": { + "tags": [ + "reporting_schemas" + ], + "summary": "Where each of a group's pairs of group and Tamanu version stands.", + "description": "One entry per published version the group's Tamanu applications report\nrunning, plus the version its open plan moves it to, so whether a group's\napplications can be offered the schema for the version they run or are\nmoving to is answered in one place.", + "operationId": "reporting_schemas_for_group", + "requestBody": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/PairsForGroupArgs" + } + } + }, + "required": true + }, + "responses": { + "200": { + "description": "Pairs, one per version the group runs or is moving to.", + "content": { + "application/json": { + "schema": { + "type": "array", + "items": { + "$ref": "#/components/schemas/Pair" + } + } + } + } + }, + "401": { + "description": "", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ProblemDetailsSchema" + } + } + } + }, + "403": { + "description": "", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ProblemDetailsSchema" + } + } + } + } + }, + "security": [ + { + "tailscale-admin": [] + } + ] + } + }, "/api/restore_replicas/checks": { "post": { "tags": [ @@ -9413,6 +9533,26 @@ } } }, + "BuildPairArgs": { + "type": "object", + "description": "Which pair to build.", + "required": [ + "group_id", + "version_id" + ], + "properties": { + "group_id": { + "type": "string", + "format": "uuid", + "description": "The group whose schema to build." + }, + "version_id": { + "type": "string", + "format": "uuid", + "description": "The Tamanu version to build it for." + } + } + }, "Caps": { "type": "object", "description": "What Canopy does for an application of a given type.\n\nReachability, health checks and backups are deliberately absent: checks are\ngraded by the source that reports them, and backup types are advertised per\nmachine by the agent, so both already work for any type.", @@ -11863,7 +12003,7 @@ "items": { "type": "string" }, - "description": "Behaviours this intent opts into. Recognised values are `check` (a\nhealth report is expected for each replica), `once` (a given snapshot\nis only ever dispatched to a replica once, rather than repeatedly\nuntil overdue), and `url` (a replica's health report includes a link\nto it). Unrecognised values are stored but have no effect." + "description": "Behaviours this intent opts into. Recognised values are `check` (a\nhealth report is expected for each replica), `once` (a given snapshot\nis only ever dispatched to a replica once, rather than repeatedly until\noverdue), `url` (a replica's health report includes a link to it),\n`migrate` (Canopy names a target version and the replica applies that\nversion's migrations), `redact` (the replica de-identifies the restored\ndata before serving it), and `reporting-schema` (the replica builds a\nTamanu reporting schema and registers it for the group). Unrecognised\nvalues are stored but have no effect, so a consumer may advertise ahead\nof Canopy support." } } }, @@ -14084,6 +14224,79 @@ } } }, + "Pair": { + "type": "object", + "description": "One pair of group and Tamanu version, and where it stands.", + "required": [ + "group_id", + "version_id", + "version", + "state", + "requested", + "applications" + ], + "properties": { + "applications": { + "type": "array", + "items": { + "type": "string" + }, + "description": "The group's Tamanu applications reporting this version, by name. Empty\nwhere the pair comes from the open plan rather than from something\nrunning it." + }, + "error": { + "type": [ + "string", + "null" + ], + "description": "What went wrong, where a build failed." + }, + "group_id": { + "type": "string", + "format": "uuid", + "description": "The group this pair is for." + }, + "requested": { + "type": "boolean", + "description": "Whether an operator has asked for this pair to be built again." + }, + "state": { + "$ref": "#/components/schemas/PairState", + "description": "Whether the pair has a schema, failed to build one, or is awaiting one." + }, + "version": { + "type": "string", + "description": "That version as semver, for display." + }, + "version_id": { + "type": "string", + "format": "uuid", + "description": "The Tamanu version this pair is for." + } + } + }, + "PairState": { + "type": "string", + "description": "Where a pair stands, for the operator view.", + "enum": [ + "awaiting", + "built", + "failed" + ] + }, + "PairsForGroupArgs": { + "type": "object", + "description": "Request body for reading a group's pairs.", + "required": [ + "group_id" + ], + "properties": { + "group_id": { + "type": "string", + "format": "uuid", + "description": "The group to report on." + } + } + }, "ParamType": { "type": "string", "description": "The data type of a restore-replica configuration parameter, which\ndetermines how its value is validated. `duration` and `bytes` values must\nbe non-negative integers (a count of seconds and of bytes, respectively);\n`integer` accepts any whole number, positive or negative; `boolean` is a\nJSON boolean; `text` is a JSON string.", @@ -15309,6 +15522,8 @@ "params", "redacts", "can_redact", + "publishes_schemas", + "can_publish_schemas", "redaction_gaps", "enabled", "gap", @@ -15316,6 +15531,10 @@ "updated_at" ], "properties": { + "can_publish_schemas": { + "type": "boolean", + "description": "True when the intent carries the `reporting-schema` semantic, so the\ndeclaration can be made the group's publisher." + }, "can_redact": { "type": "boolean", "description": "True when the intent carries the `redact` semantic, so the declaration\ncan be switched to redacting." @@ -15388,6 +15607,10 @@ "type": "object", "description": "Operator-supplied parameter values (name → value). Values of\n`duration` and `bytes` parameters are formatted as human-friendly\nstrings (e.g. `2h 30m`, `20Gi`) when the intent's schema is known;\n`create` and `update` accept these strings back." }, + "publishes_schemas": { + "type": "boolean", + "description": "Whether this declaration's consumer may publish the group's reporting\nschema. Only an operator sets it." + }, "redaction_gaps": { "type": "array", "items": { @@ -15457,6 +15680,10 @@ "type": "object", "description": "Parameter values for the intent (name → value), validated against the\nconsumer's advertised parameter schema. `duration` and `bytes`\nparameters accept human-unit strings (e.g. `2h 30m`, `20Gi`) as well\nas raw integer seconds/bytes. Defaults to empty." }, + "publishes_schemas": { + "type": "boolean", + "description": "Whether this consumer may publish the group's reporting schema.\nAccepted only for a group-wide, non-redacting declaration whose intent\ncarries the `reporting-schema` semantic. Defaults to false, so a\nconsumer publishes only where an operator has said it may." + }, "redacts": { "type": "boolean", "description": "Whether the replica is served de-identified. Accepted only for an\nintent carrying the `redact` semantic; Canopy resolves the masking\nmanifest itself from the server's product, so there is nothing else\nto set. Defaults to false." @@ -15540,6 +15767,10 @@ "type": "object", "description": "New parameter values (name → value), validated against the intent's\nadvertised parameter schema. `duration` and `bytes` parameters accept\nhuman-unit strings (e.g. `2h 30m`, `20Gi`) as well as raw integer\nseconds/bytes. Defaults to empty." }, + "publishes_schemas": { + "type": "boolean", + "description": "Whether this consumer may publish the group's reporting schema.\nAccepted only for a group-wide, non-redacting declaration whose intent\ncarries the `reporting-schema` semantic. Defaults to false." + }, "redacts": { "type": "boolean", "description": "Whether the replica is served de-identified. Accepted only for an\nintent carrying the `redact` semantic. Defaults to false." @@ -18336,6 +18567,10 @@ "name": "migration_tests", "description": "Where each server stands against the version it would take next." }, + { + "name": "reporting_schemas", + "description": "Which of a group's versions have a reporting schema built for them." + }, { "name": "restore_replicas", "description": "Managed restore replicas: capabilities, worklist, and health." diff --git a/private-web/src/api-types.ts b/private-web/src/api-types.ts index 2dd496e7..db7022bf 100644 --- a/private-web/src/api-types.ts +++ b/private-web/src/api-types.ts @@ -3091,6 +3091,52 @@ export interface paths { patch?: never; trace?: never; }; + "/api/reporting_schemas/build": { + parameters: { + query?: never; + header?: never; + path?: never; + cookie?: never; + }; + get?: never; + put?: never; + /** + * Ask for a pair's schema to be built. + * @description This is how a schema is refreshed after the group's configuration changes, + * and how a settled pair is put back on the worklist: a build against a fixed + * version and configuration fails the same way every time, so a failed pair + * waits for this rather than retrying on its own. + */ + post: operations["reporting_schemas_build"]; + delete?: never; + options?: never; + head?: never; + patch?: never; + trace?: never; + }; + "/api/reporting_schemas/for_group": { + parameters: { + query?: never; + header?: never; + path?: never; + cookie?: never; + }; + get?: never; + put?: never; + /** + * Where each of a group's pairs of group and Tamanu version stands. + * @description One entry per published version the group's Tamanu applications report + * running, plus the version its open plan moves it to, so whether a group's + * applications can be offered the schema for the version they run or are + * moving to is answered in one place. + */ + post: operations["reporting_schemas_for_group"]; + delete?: never; + options?: never; + head?: never; + patch?: never; + trace?: never; + }; "/api/restore_replicas/checks": { parameters: { query?: never; @@ -4926,6 +4972,19 @@ export interface components { /** @description Label value. */ value: string; }; + /** @description Which pair to build. */ + BuildPairArgs: { + /** + * Format: uuid + * @description The group whose schema to build. + */ + group_id: string; + /** + * Format: uuid + * @description The Tamanu version to build it for. + */ + version_id: string; + }; /** * @description What Canopy does for an application of a given type. * @@ -6533,9 +6592,14 @@ export interface components { /** * @description Behaviours this intent opts into. Recognised values are `check` (a * health report is expected for each replica), `once` (a given snapshot - * is only ever dispatched to a replica once, rather than repeatedly - * until overdue), and `url` (a replica's health report includes a link - * to it). Unrecognised values are stored but have no effect. + * is only ever dispatched to a replica once, rather than repeatedly until + * overdue), `url` (a replica's health report includes a link to it), + * `migrate` (Canopy names a target version and the replica applies that + * version's migrations), `redact` (the replica de-identifies the restored + * data before serving it), and `reporting-schema` (the replica builds a + * Tamanu reporting schema and registers it for the group). Unrecognised + * values are stored but have no effect, so a consumer may advertise ahead + * of Canopy support. */ semantics?: string[]; }; @@ -7925,6 +7989,46 @@ export interface components { */ offset: number; }; + /** @description One pair of group and Tamanu version, and where it stands. */ + Pair: { + /** + * @description The group's Tamanu applications reporting this version, by name. Empty + * where the pair comes from the open plan rather than from something + * running it. + */ + applications: string[]; + /** @description What went wrong, where a build failed. */ + error?: string | null; + /** + * Format: uuid + * @description The group this pair is for. + */ + group_id: string; + /** @description Whether an operator has asked for this pair to be built again. */ + requested: boolean; + /** @description Whether the pair has a schema, failed to build one, or is awaiting one. */ + state: components["schemas"]["PairState"]; + /** @description That version as semver, for display. */ + version: string; + /** + * Format: uuid + * @description The Tamanu version this pair is for. + */ + version_id: string; + }; + /** + * @description Where a pair stands, for the operator view. + * @enum {string} + */ + PairState: "awaiting" | "built" | "failed"; + /** @description Request body for reading a group's pairs. */ + PairsForGroupArgs: { + /** + * Format: uuid + * @description The group to report on. + */ + group_id: string; + }; /** * @description The data type of a restore-replica configuration parameter, which * determines how its value is validated. `duration` and `bytes` values must @@ -8694,6 +8798,11 @@ export interface components { * read access to the covered backups while it is enabled. */ RestoreReplicaView: { + /** + * @description True when the intent carries the `reporting-schema` semantic, so the + * declaration can be made the group's publisher. + */ + can_publish_schemas: boolean; /** * @description True when the intent carries the `redact` semantic, so the declaration * can be switched to redacting. @@ -8758,6 +8867,11 @@ export interface components { * `create` and `update` accept these strings back. */ params: Record; + /** + * @description Whether this declaration's consumer may publish the group's reporting + * schema. Only an operator sets it. + */ + publishes_schemas: boolean; /** * @description Servers this declaration covers that cannot currently be redacted: * either their product publishes no masking manifest, or the version @@ -8817,6 +8931,13 @@ export interface components { * as raw integer seconds/bytes. Defaults to empty. */ params?: Record; + /** + * @description Whether this consumer may publish the group's reporting schema. + * Accepted only for a group-wide, non-redacting declaration whose intent + * carries the `reporting-schema` semantic. Defaults to false, so a + * consumer publishes only where an operator has said it may. + */ + publishes_schemas?: boolean; /** * @description Whether the replica is served de-identified. Accepted only for an * intent carrying the `redact` semantic; Canopy resolves the masking @@ -8891,6 +9012,12 @@ export interface components { * seconds/bytes. Defaults to empty. */ params?: Record; + /** + * @description Whether this consumer may publish the group's reporting schema. + * Accepted only for a group-wide, non-redacting declaration whose intent + * carries the `reporting-schema` semantic. Defaults to false. + */ + publishes_schemas?: boolean; /** * @description Whether the replica is served de-identified. Accepted only for an * intent carrying the `redact` semantic. Defaults to false. @@ -14817,6 +14944,91 @@ export interface operations { }; }; }; + reporting_schemas_build: { + parameters: { + query?: never; + header?: never; + path?: never; + cookie?: never; + }; + requestBody: { + content: { + "application/json": components["schemas"]["BuildPairArgs"]; + }; + }; + responses: { + 200: { + headers: { + [name: string]: unknown; + }; + content?: never; + }; + 400: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["ProblemDetailsSchema"]; + }; + }; + 401: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["ProblemDetailsSchema"]; + }; + }; + 403: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["ProblemDetailsSchema"]; + }; + }; + }; + }; + reporting_schemas_for_group: { + parameters: { + query?: never; + header?: never; + path?: never; + cookie?: never; + }; + requestBody: { + content: { + "application/json": components["schemas"]["PairsForGroupArgs"]; + }; + }; + responses: { + /** @description Pairs, one per version the group runs or is moving to. */ + 200: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["Pair"][]; + }; + }; + 401: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["ProblemDetailsSchema"]; + }; + }; + 403: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["ProblemDetailsSchema"]; + }; + }; + }; + }; restore_replicas_checks: { parameters: { query?: never; diff --git a/private-web/src/components/ReportingSchemasSection.test.tsx b/private-web/src/components/ReportingSchemasSection.test.tsx new file mode 100644 index 00000000..1e607505 --- /dev/null +++ b/private-web/src/components/ReportingSchemasSection.test.tsx @@ -0,0 +1,193 @@ +import { fireEvent, render, screen, waitFor } from "@testing-library/react"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import ReportingSchemasSection from "./ReportingSchemasSection"; + +// The admin probe belongs to the page this section is mounted in, so the +// section is rendered here with the answer it would have been given. +const admin = vi.hoisted(() => ({ is: true as boolean | undefined })); +vi.mock("../hooks/useIsAdmin", () => ({ useIsAdmin: () => admin.is })); + +type Pair = { + group_id: string; + version_id: string; + version: string; + state: "awaiting" | "built" | "failed"; + error?: string | null; + requested: boolean; + applications: string[]; +}; + +const GROUP = "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa"; + +function pair(over: Partial = {}): Pair { + return { + group_id: GROUP, + version_id: "11111111-1111-1111-1111-111111111111", + version: "2.60.0", + state: "awaiting", + error: null, + requested: false, + applications: ["kamaka-central"], + ...over, + }; +} + +/// Answer `for_group` with `pairs`, and `build` with either a 200 or a +/// ProblemDetails the component is expected to surface. +function stubApi(pairs: Pair[], build: { status: number; body?: unknown } = { status: 200 }) { + const calls: { url: string; body: unknown }[] = []; + const fetch = vi.fn(async (input: RequestInfo | URL, init?: RequestInit) => { + const url = typeof input === "string" ? input : input.toString(); + calls.push({ url, body: init?.body ? JSON.parse(String(init.body)) : undefined }); + + if (url.includes("reporting_schemas/build")) { + return new Response(JSON.stringify(build.body ?? {}), { + status: build.status, + headers: { "content-type": "application/json" }, + }); + } + return new Response(JSON.stringify(pairs), { + status: 200, + headers: { "content-type": "application/json" }, + }); + }); + + vi.stubGlobal("fetch", fetch); + return calls; +} + +beforeEach(() => { + admin.is = true; +}); + +afterEach(() => { + vi.unstubAllGlobals(); +}); + +describe("a pair's state reads off the chip", () => { + it("shows a built pair as built", async () => { + stubApi([pair({ state: "built" })]); + render(); + + expect(await screen.findByText("Built")).toBeTruthy(); + }); + + it("shows an unbuilt pair as awaiting, which is not a failure", async () => { + stubApi([pair()]); + render(); + + expect(await screen.findByText("Awaiting build")).toBeTruthy(); + expect(screen.queryByText("Failed")).toBeNull(); + }); + + it("carries the builder's own error on the failed chip", async () => { + stubApi([pair({ state: "failed", error: "views did not compile" })]); + render(); + + fireEvent.mouseOver(await screen.findByText("Failed")); + expect(await screen.findByText("views did not compile")).toBeTruthy(); + }); + + it("falls back where the build reported no description", async () => { + stubApi([pair({ state: "failed", error: null })]); + render(); + + fireEvent.mouseOver(await screen.findByText("Failed")); + expect(await screen.findByText("the build failed")).toBeTruthy(); + }); +}); + +describe("which servers a pair covers", () => { + it("counts them on the row and names them behind it", async () => { + stubApi([ + pair({ + applications: ["kamaka-central", "kamaka-clinic-north"], + }), + ]); + render(); + + fireEvent.mouseOver(await screen.findByText("2 servers")); + expect( + await screen.findByText("kamaka-central, kamaka-clinic-north"), + ).toBeTruthy(); + }); + + it("says where a pair comes from the plan rather than from a server", async () => { + stubApi([pair({ applications: [] })]); + render(); + + expect(await screen.findByText("upgrade plan")).toBeTruthy(); + expect(screen.queryByText("0 servers")).toBeNull(); + }); +}); + +describe("asking for a build", () => { + it("offers a first build on an unbuilt pair and a rebuild on a settled one", async () => { + stubApi([ + pair({ version_id: "1", version: "2.59.0", state: "awaiting" }), + pair({ version_id: "2", version: "2.60.0", state: "built" }), + pair({ version_id: "3", version: "2.61.0", state: "failed" }), + ]); + render(); + + expect(await screen.findByText("Build sooner")).toBeTruthy(); + expect(screen.getAllByText("Build again")).toHaveLength(2); + }); + + it("offers no build to an operator who cannot ask for one", async () => { + admin.is = false; + stubApi([pair({ state: "failed" }), pair({ version_id: "2", state: "built" })]); + render(); + + expect(await screen.findByText("Failed")).toBeTruthy(); + expect(screen.queryByText("Build sooner")).toBeNull(); + expect(screen.queryByText("Build again")).toBeNull(); + }); + + it("names the pair rather than the group's latest version", async () => { + const calls = stubApi([pair({ version_id: "abc", version: "2.59.0" })]); + render(); + + fireEvent.click(await screen.findByText("Build sooner")); + + await waitFor(() => { + const ask = calls.find((c) => c.url.includes("reporting_schemas/build")); + expect(ask?.body).toEqual({ group_id: GROUP, version_id: "abc" }); + }); + }); + + it("replaces the control once an ask is recorded, so it is not asked twice", async () => { + stubApi([pair({ requested: true })]); + render(); + + expect(await screen.findByText("Build asked for")).toBeTruthy(); + expect(screen.queryByText("Build sooner")).toBeNull(); + }); + + it("surfaces a refused ask rather than looking like it worked", async () => { + stubApi([pair()], { + status: 403, + body: { title: "insufficient permissions: admin role required" }, + }); + render(); + + fireEvent.click(await screen.findByText("Build sooner")); + + expect(await screen.findByText(/insufficient permissions/)).toBeTruthy(); + }); +}); + +describe("a group with nothing to build", () => { + // Two different reasons reach the same empty answer: no builder is declared + // for the group, or nothing in it reports a published version. Naming both + // is what stops an operator reading an absent builder as a backlog. + it("names both reasons rather than showing an empty table", async () => { + stubApi([]); + render(); + + const empty = await screen.findByText(/nothing to build for this group/i); + expect(empty.textContent).toMatch(/no builder is declared/i); + expect(empty.textContent).toMatch(/reports a published version/i); + expect(screen.queryByText("Version")).toBeNull(); + }); +}); diff --git a/private-web/src/components/ReportingSchemasSection.tsx b/private-web/src/components/ReportingSchemasSection.tsx new file mode 100644 index 00000000..d095f5e3 --- /dev/null +++ b/private-web/src/components/ReportingSchemasSection.tsx @@ -0,0 +1,190 @@ +import { + Alert, + Box, + Button, + Chip, + LinearProgress, + Paper, + Table, + TableBody, + TableCell, + TableHead, + TableRow, + Tooltip, + Typography, +} from "@mui/material"; +import { useApi, useApiAction } from "../api"; +import { useIsAdmin } from "../hooks/useIsAdmin"; + +type PairState = "awaiting" | "built" | "failed"; + +/// Which of the group's versions have a reporting schema, which failed, and +/// which are still to be built, so whether the group's applications can be +/// offered the schema for the version they run or are moving to is answered in +/// one place. +// spec: RPT#alerting +export default function ReportingSchemasSection({ + groupId, +}: { + groupId: string; +}) { + const pairs = useApi( + "reporting_schemas", + "for_group", + { group_id: groupId }, + [groupId], + ); + const build = useApiAction("reporting_schemas", "build"); + const isAdmin = useIsAdmin() === true; + + if (pairs.status === "loading" || pairs.status === "idle") { + return ( + + + + + ); + } + if (pairs.status === "error") { + return ( + + + {pairs.error.message} + + ); + } + + if (pairs.data.length === 0) { + return ( + + + + Nothing to build for this group: either no builder is declared for it + under Backups, or no Tamanu application in it reports a published + version. + + + ); + } + + const ask = async (versionId: string) => { + try { + await build.call({ group_id: groupId, version_id: versionId }); + pairs.reload(); + } catch { + /* surfaced via build.error */ + } + }; + + return ( + + + {build.error && ( + + {build.error.message} + + )} + + + + Version + Schema + On + + + + + {pairs.data.map((pair) => ( + + + {pair.version} + + + + + + + + + {pair.requested ? ( + + Build asked for + + ) : ( + isAdmin && ( + + ) + )} + + + ))} + +
+ + ); +} + +/// Which of the group's applications a pair covers. +/// +/// A pair is per version, so one row stands for every application on it. The +/// count is what an operator sizes the row by; the names are behind it because +/// a group of any size would otherwise make the table taller than it is wide. +function Running({ applications }: { applications: string[] }) { + if (applications.length === 0) { + return ( + + upgrade plan + + ); + } + + return ( + + + {applications.length === 1 + ? "1 server" + : `${applications.length} servers`} + + + ); +} + +function StateChip({ + state, + error, +}: { + state: PairState; + error?: string | null; +}) { + if (state === "built") { + return ; + } + if (state === "awaiting") { + return ; + } + return ( + + + + ); +} + +function SectionHeading() { + return ( + + + Reporting schemas + + + One per version this group runs or is moving to, built from a replica + of the group's own data. + + + ); +} diff --git a/private-web/src/components/RestoreReplicasSection.tsx b/private-web/src/components/RestoreReplicasSection.tsx index 0818875d..64f37113 100644 --- a/private-web/src/components/RestoreReplicasSection.tsx +++ b/private-web/src/components/RestoreReplicasSection.tsx @@ -131,6 +131,7 @@ export default function RestoreReplicasSection({ overdue_after: r.overdue_after, params: r.params as Record, redacts: r.redacts, + publishes_schemas: r.publishes_schemas, enabled, }); reload(); @@ -217,6 +218,15 @@ export default function RestoreReplicasSection({ )} + {r.publishes_schemas && ( + + + + )} {r.overdue_after ?? "no bound"} @@ -527,6 +537,46 @@ function RedactionField({ ); } +/** The publishing switch, shown only for an intent that builds reporting + * schemas. A build is per group from data the masking manifest has not + * altered, so a machine-scoped or redacting declaration cannot carry it. */ +function PublishesSchemasField({ + value, + onChange, + disabled, + why, +}: { + value: boolean; + onChange: (value: boolean) => void; + disabled: boolean; + why: string; +}) { + return ( + onChange(e.target.checked)} + /> + } + label={ + + + Publish this group's reporting schema + + + {disabled + ? why + : "Lets this consumer register the schema every application in the group is offered."} + + + } + /> + ); +} + /** Convert the typed form fields into the wire params object, omitting any the * operator left unset (the consumer resolves those to their default or null). * Returns an error message string if a numeric field doesn't parse. */ @@ -616,6 +666,8 @@ function useIntentSchema( const advertised = (selectedDescriptor?.params as Record | undefined) ?? {}; const canRedact = selectedDescriptor?.semantics?.includes("redact") ?? false; + const canPublishSchemas = + selectedDescriptor?.semantics?.includes("reporting-schema") ?? false; // Canopy owns the masking parameters for a `redact` intent in both states, // so they get no field: the redaction switch is the whole of the operator's // say in it. @@ -626,7 +678,13 @@ function useIntentSchema( ), ) : advertised; - return { intentOptions, selectedDescriptor, paramSchema, canRedact }; + return { + intentOptions, + selectedDescriptor, + paramSchema, + canRedact, + canPublishSchemas, + }; } /** Consumer, server (or whole-group), type, and intent selects, shared by the @@ -781,11 +839,17 @@ function CreateReplicaDialog({ const [overdue, setOverdue] = useState(""); const [paramValues, setParamValues] = useState>({}); const [redacts, setRedacts] = useState(false); + const [publishesSchemas, setPublishesSchemas] = useState(false); const [pending, setPending] = useState(false); const [error, setError] = useState(null); - const { intentOptions, selectedDescriptor, paramSchema, canRedact } = - useIntentSchema(consumers, consumerId, intent); + const { + intentOptions, + selectedDescriptor, + paramSchema, + canRedact, + canPublishSchemas, + } = useIntentSchema(consumers, consumerId, intent); // Auto-select the sole consumer, if there's only one to choose from. useEffect(() => { @@ -814,6 +878,12 @@ function CreateReplicaDialog({ if (!canRedact) setRedacts(false); }, [canRedact]); + // A build is dispatched per group from unmasked data, so narrowing the + // declaration to a machine or turning redaction on drops the flag with it. + useEffect(() => { + if (!canPublishSchemas || redacts || serverId) setPublishesSchemas(false); + }, [canPublishSchemas, redacts, serverId]); + // Suggest a name from the group, (if picked) server, and intent, until the // operator types their own. The intent is part of it because names are // unique per consumer: without it, declaring a second intent for the same @@ -863,6 +933,7 @@ function CreateReplicaDialog({ overdue_after, params, redacts, + publishes_schemas: publishesSchemas, }); onCreated(); } catch (err) { @@ -920,6 +991,19 @@ function CreateReplicaDialog({ )} + {canPublishSchemas && ( + + )} + >(() => { const initialDescriptor = consumers .find((c) => c.device_id === replica.consumer_device_id) @@ -987,8 +1074,13 @@ function EditReplicaDialog({ const [pending, setPending] = useState(false); const [error, setError] = useState(null); - const { intentOptions, selectedDescriptor, paramSchema, canRedact } = - useIntentSchema(consumers, consumerId, intent); + const { + intentOptions, + selectedDescriptor, + paramSchema, + canRedact, + canPublishSchemas, + } = useIntentSchema(consumers, consumerId, intent); // Retargeting to an intent that can't redact drops the flag with it, so the // declaration doesn't carry an intent the new consumer can't honour. @@ -996,6 +1088,12 @@ function EditReplicaDialog({ if (!canRedact) setRedacts(false); }, [canRedact]); + // A build is dispatched per group from unmasked data, so narrowing the + // declaration to a machine or turning redaction on drops the flag with it. + useEffect(() => { + if (!canPublishSchemas || redacts || serverId) setPublishesSchemas(false); + }, [canPublishSchemas, redacts, serverId]); + // Re-derive parameter values whenever the consumer or intent changes: keep // values for parameter names the new schema still has, drop the rest. useEffect(() => { @@ -1038,6 +1136,7 @@ function EditReplicaDialog({ overdue_after, params, redacts, + publishes_schemas: publishesSchemas, enabled, }); onUpdated(); @@ -1103,6 +1202,19 @@ function EditReplicaDialog({ )} + {canPublishSchemas && ( + + )} + +