diff --git a/CLAUDE.md b/CLAUDE.md index 98ac96b4..2d0a71d2 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -733,6 +733,19 @@ the source's VCTM/MDDL file and ships no copy of it, and on **Fly** its document the apigw over it (`http: request body too large`, gdc 2026-10-20; `tests/test_fly_payload.py` guards it). So on Fly `make datastore-sync` is what adds the twins' documents, also after a storage clear. +**Without twins (SUNET/vc#840, per environment).** An environment that sets `issuer.allowPreauthOffer: true` (gdc, once its +apigw image carries #840) gets `allow_preauth_offer: true` on every login-gated datastore scope, and the chart DROPS the +twins for it (`vc_render.drop_offer_twins`, applied to the base values in `render.py`: the `_offer` credential types +and their bootstrapping documents), so the issuer's `credential_configurations_supported`, the datastore scopes and the +verifier lists carry one entry per type. `make datastore-offer SCOPE=` then mints against the base scope: the +command follows the RENDERED apigw config (`scripts/datastore.py offer_scope`): a flagged base scope is used as it is, an +unflagged one uses its twin, a named `_offer` is never rewritten, and a twin-style DOCUMENT_ID is mapped to the base +document id. The flag is OFF by default: a vc image with #840 refuses to start with it set unless `apigw.api_server.api_auth` +has `jwks.enable` (and the admin UI off) or `oidc.enable`, and an older image ignores it. Migration order on an +environment: flag + the new image, move minting, verify an offer for the base scope redeems, THEN drop the twins (an +offer naming a configuration id the issuer no longer publishes is unredeemable). Twin documents already in a live datastore +stay there until removed (`search --scope _offer`). + **The issuer and the verifier show the SUNET logo:** they are showing vc's built-in assets, which means `common.branding` never reached them. The chart points `logo_path`/`favicon_path` at `/branding-assets/*.png` and expects an diff --git a/README.md b/README.md index d0d03a87..f120da4e 100644 --- a/README.md +++ b/README.md @@ -401,7 +401,7 @@ picks between them. | `pid_mdoc` | alice-001 | bob-002 | | `mdl_zk4` | alice-001 | bob-002 | | `photoid` | alice-001, bob-002 | carol-003 | -| `_offer` for every row above | as their source | as their source (pre-authorized twins for `make datastore-offer`; copies, kept equal by a test) | +| `_offer` for every row above | as their source | as their source (pre-authorized twins for `make datastore-offer`; copies, kept equal by a test). Only in environments that do NOT set `issuer.allowPreauthOffer` (gdc and nymauth do not have them: SUNET/vc#840) | | `ebw_oid` | erik-010, maria-011 | sophie-013 | | `eucc` | erik-010, maria-011, jan-012 | sophie-013 | | `eu_poa` | jan-012 | sophie-013 | diff --git a/chart/templates/04-issuer.yaml b/chart/templates/04-issuer.yaml index eff07379..f69c0d3c 100644 --- a/chart/templates/04-issuer.yaml +++ b/chart/templates/04-issuer.yaml @@ -192,6 +192,9 @@ apigw: {{- range $id, $data := . }} {{ $id | quote }}: auth_provider: {{ $data.issuance.authProvider | quote }} + {{- if and $.Values.issuer.allowPreauthOffer (ne $data.issuance.authProvider "preauth") }} + allow_preauth_offer: true + {{- end }} {{- if and $.Values.issuer.datastoreAuthenticSource (ne $data.issuance.authProvider "preauth") }} authentic_source: {{ $.Values.issuer.datastoreAuthenticSource | quote }} {{- end }} diff --git a/docs/NYMAUTH-TESTING.md b/docs/NYMAUTH-TESTING.md index bfdd23dc..56366f2a 100644 --- a/docs/NYMAUTH-TESTING.md +++ b/docs/NYMAUTH-TESTING.md @@ -7,7 +7,8 @@ what that needs; `diff environments/gdc.yaml environments/nymauth.yaml` shows ex **Status.** The flow below was proven live on `gdc` (a Pixel, the Kotlin sample app): issuance with seed delivery and notification; same-device presentation of both presets; cross-device presentation by scanning the verifier page's QR in a -browser. **Nobody has deployed `ENV=nymauth` yet**; it reproduces the same pins on its own hostnames. +browser. **Nobody has deployed `ENV=nymauth` yet**; it reproduces gdc's pins on its own hostnames. (The vc images were rebuilt +from vc `c8c5b1ea` - classified refusals and #840 - after the proven run; the Pixel run on exactly that set is the next step.) > **Every pinned image except the PDP and the frontend is a local build of UNMERGED pull requests.** Re-pin them to > released images when the PRs land. What went into each build (tags, digests, base commits, PR heads) is the @@ -31,8 +32,7 @@ sample app and a browser. ### Deploy ```sh flyctl auth docker # registry login (expires in minutes: run it right before) -make fly-up ENV=nymauth -make datastore-sync ENV=nymauth # puts the offer twins' documents into the datastore (once, and after a storage clear) +make fly-up ENV=nymauth # a fresh datastore imports the fixtures on first start: nothing else to run ``` `make fly-up ENV=nymauth` would create, in the org's `arn` region: **11 apps** - `sirosid-nymauth-` + `mongodb`, `mini-oidc`, `pdp`, `vc-registry`, `vc-issuer`, `vc-apigw`, `vc-verifier`, `wallet-backend`, `wallet-proxy`, `wallet-frontend`, @@ -51,11 +51,12 @@ Do not run these unless you mean to create those resources. ### Mint an offer for alice-001 ```sh -make datastore-offer SCOPE=nym_auth DOCUMENT_ID=document_id_nym_auth_offer_alice-001 ENV=nymauth +make datastore-offer SCOPE=nym_auth DOCUMENT_ID=document_id_nym_auth_alice-001 ENV=nymauth ``` It prints an `openid-credential-offer://...` URL (add `QR=yes` for a terminal QR). **The code is valid for 5 minutes and -single use**: mint it when the phone is in your hand. `nym_auth_offer` is the pre-authorized twin of `nym_auth`; once the -twins migration lands (PRs for SUNET/vc#840, see sirosid-dev PR list) the same command mints against `nym_auth` itself. +single use**: mint it when the phone is in your hand. The offer names the `nym_auth` configuration itself: this environment +has no `_offer` twins, because its vc images carry SUNET/vc#840 (`allow_preauth_offer`) and the chart sets +`issuer.allowPreauthOffer` (a standing rule until #840 is merged to vc main: pin vc images built from a SHA that contains it). --- @@ -120,6 +121,7 @@ wallet's engine connection is thrashing (two client connections replacing each o | a verifier request is refused as untrusted | the verifier's request-signing certificate lacks the verifier host as a DNS SAN | `values.verifier.signingCertDnsSan: true` (set) re-issues it on deploy | | the debug APK shows a 16 KB page-size alignment dialog on first start | a debug-build notice | dismiss it | | offer redeem fails with invalid/expired code | the code is single use and valid 5 minutes | mint a new one | +| minting an offer fails with `invalid_scope` | the apigw image lacks SUNET/vc#840 (it ignores `allow_preauth_offer`) | pin vc images built from a SHA that contains #840 | --- @@ -178,9 +180,9 @@ URL, x5c = the DS leaf) is true from the mDOC IACA registry with the leaf alone. ### The pins | component | pin | built from | |---|---|---| -| vc-verifier | `registry.fly.io/sirosid-gdc-vc-verifier:local-1792603698` | vc `integration/gdc-nymauth-v5` @ `a56c4be9e` + circuit-artifact overlay | -| vc-apigw / issuer / registry | `...:local-1792473614` | vc `integration/gdc-nymauth-v4` @ `5e546b2d` | +| vc-verifier / apigw / issuer / registry | `registry.fly.io/sirosid-gdc-vc-:local-1792609658` | vc `integration/gdc-nymauth-v5` @ `c8c5b1ea` (includes #840; verifier + circuit-artifact overlay) | | wallet-backend (engine) | `registry.fly.io/sirosid-gdc-wallet-backend:local-1792564143` | go-wallet-backend local branch @ `9cb6e57` (#469, #471, #163, #472; #473 inside #471) | | pdp | `ghcr.io/sirosfoundation/go-trust:0.26.0` | released (go-trust#218) | -Next vc build (not in the proven set): `c8c5b1ea` with classified zkp_req refusals and SUNET/vc#840 (`allow_preauth_offer`). +The previously PROVEN vc build was `a56c4be9e` (verifier `local-1792603698`) with apigw/issuer/registry `local-1792473614`; the current pins +add the classified zkp_req refusals (HTTP 400 with a stable code) and #840, and are not yet proven by a Pixel run. diff --git a/environments/gdc.yaml b/environments/gdc.yaml index 0879f2d7..3e30f972 100644 --- a/environments/gdc.yaml +++ b/environments/gdc.yaml @@ -39,9 +39,9 @@ images: # engine, PDP and issuer images below (verifier vc a56c4be9e = local-1792603698, engine 9cb6e57 = local-1792564143, # PDP go-trust 0.26.0 with the verifier_ids policies, apigw/issuer/registry local-1792473614), zk-cred-longfellow v0.1.2 # and zk-cred-vega v0.0.7, matcher https://github.com/sirosfoundation/siros-dc-matcher v0.11.0 on the wallet side, and the - # circuit catalog https://github.com/sirosfoundation/go-zk-circuits release v0.1.0-release.16 (zk-circuits.fly.dev). The next - # vc build (c8c5b1ea: classified zkp_req refusals + https://github.com/SUNET/vc/pull/840 allow_preauth_offer) is NOT in this - # set yet. environments/nymauth.yaml reproduces this set on its own hosts (docs/NYMAUTH-TESTING.md). + # circuit catalog https://github.com/sirosfoundation/go-zk-circuits release v0.1.0-release.16 (zk-circuits.fly.dev). The vc build + # below (c8c5b1ea: classified zkp_req refusals + https://github.com/SUNET/vc/pull/840 allow_preauth_offer, twins removed) is the + # NEXT set and is not proven by a Pixel run yet. environments/nymauth.yaml reproduces this set on its own hosts (docs/NYMAUTH-TESTING.md). # # NYMAUTH-CRITICAL SETTINGS in this file (each one broke the flow when wrong; see docs/NYMAUTH-TESTING.md, gotchas): # verifier.nym_auth.verifier_id = the PREFIXED client_id x509_san_dns:; verifier.preferred_vp_formats.zkp_req @@ -50,42 +50,35 @@ images: # delivery); signingCertDnsSan (the verifier's host in the request-signing cert); the engine image (request encryption, # client_metadata, WMP) with the config unchanged (dcql_consent_check default). # - # vc-verifier registry.fly.io/sirosid-gdc-vc-verifier:local-1792603698 - # digest sha256:96c41f16dd9324da6890c76b9faa7810d4c8259cf142ffd16360e413c8ca8945 - # source sirosfoundation/vc `integration/gdc-nymauth-v5` @ a56c4be9ec42ef394d536f5ea08d80b50f2a95a0 (merge base - # SUNET/vc main 4bba38e614e1; integration of the stack below, NOT for upstream merge), --target - # runtime-verifier, zknative, + the nym_auth circuit artifact overlay (dockerfiles/verifier-nymauth.Dockerfile) + # vc-verifier / vc-apigw / vc-issuer / vc-registry registry.fly.io/sirosid-gdc-vc-:local-1792609658 + # digests verifier sha256:9385f5bf1b676b87f2fe6891769fafbc2cd6bd84c65b10bc614104512233891a (+ the circuit artifact overlay) + # apigw sha256:bb1a46f329211124e32e33421aa9526a96170bb3fa5e10dc082dab4535fef6c7 + # issuer sha256:9064f735c12d4d97962eed55c54db9c6ce123197eb38f9de3a197709ca57ea48 + # registry sha256:afa31586c9a8c17711c963ba21d17b7ff1bb80d3ddc90b27eeb18823dbdeb4f9 + # source sirosfoundation/vc `integration/gdc-nymauth-v5` @ c8c5b1eafd0ff08c14ca3cf8ebaceaa4473533f0 (NOT for upstream merge); + # the verifier is --target runtime-verifier with zknative; all four are built from this one SHA # PRs https://github.com/SUNET/vc/pull/827 @ 80d6ee9ef (nym_auth circuit allowlist; open) - # https://github.com/SUNET/vc/pull/829 @ 55a8c9d7f (validate a zkp_req response against the request; open) - # https://github.com/SUNET/vc/pull/830 @ 770885f31 (dispatch a zkp_req presentation to the nym_auth verifier, - # incl. the allow-all-PDP fix; open) - # https://github.com/SUNET/vc/pull/831 @ deb0c19b6 (bind a redirect-delivered presentation to its handover; open) - # https://github.com/SUNET/vc/pull/832 @ 6649047e4 (operator docs; open) - # plus the rulebook v0.4 port, the verifier UI fixes (req-schema.js `fn_id`, posted `req` vs wire `_req`) and - # the meta.source_format fix, all on that integration branch + # https://github.com/SUNET/vc/pull/829 @ 19efd24c7 (validate a zkp_req response; open) + # https://github.com/SUNET/vc/pull/830 @ 90f227fcd (dispatch to the nym_auth verifier, allow-all-PDP fix, classified + # refusals: HTTP 400 with a stable code, e.g. zkp_req_issuer_key_set_size; open) + # https://github.com/SUNET/vc/pull/831 @ 0364b170b (bind a redirect-delivered presentation to its handover; open) + # https://github.com/SUNET/vc/pull/832 @ 3f8efba54 (operator docs; open) + # https://github.com/SUNET/vc/pull/839 @ 94a6e6732 (v0.4 conversion of main's pkg/nymauth; open) + # https://github.com/SUNET/vc/pull/840 @ 952426dc5 (`allow_preauth_offer`: an offer for a login-gated scope, no twin; open) + # plus upstream SUNET/vc main 6130eed28, which brings https://github.com/SUNET/vc/pull/793 (seed-backed + # commitments), https://github.com/SUNET/vc/pull/819, https://github.com/SUNET/vc/pull/836, + # https://github.com/SUNET/vc/pull/837 and https://github.com/SUNET/vc/pull/838 (all merged upstream); the apigw seed + # delivery rides on https://github.com/SUNET/vc/pull/821 as before (in the v4 line this branch superseded) # ZK libs https://github.com/sirosfoundation/zk-cred-longfellow v0.1.2 @ 58de20f07487bbb6ebe32148feec3bef4d92677e and # https://github.com/sirosfoundation/zk-cred-vega v0.0.7 @ 8e7be172e9010e8b47f3192f046d5f990f602790 - # matcher https://github.com/sirosfoundation/siros-dc-matcher v0.11.0 (the wallet side's request matcher; not in this - # image); circuit catalog https://github.com/sirosfoundation/go-zk-circuits release v0.1.0-release.16 is served - # from zk-circuits.fly.dev (values.verifier.zkCircuits) - # previous pins: local-1792603698 (vc 253a17e92), local-1792532233 (094549cc; cannot answer a zkp_req request, never - # deployed), local-1792523015 (ae33687f), local-1792485810 (2a30a609, never deployed), local-1792456320 - # (6c3ac47) - # - # vc-apigw / vc-issuer / vc-registry registry.fly.io/sirosid-gdc-vc-:local-1792473614 - # digests apigw sha256:6d532929f7a5238d16767fe071d928880b68634d8b1c0065510779206c92eb27 - # issuer sha256:c0efa58f3f72bccc73f750be3bf89cfc5f331386e78908520340c4023c758b58 - # registry sha256:458cb70c8b9918fd9841b06fcfbf6954b39030c05500ff0a717c377a6a453268 - # source sirosfoundation/vc `integration/gdc-nymauth-v4` @ 5e546b2d61d4756f04b39ea84d26cf6b18b830ba (NOT for upstream - # merge), on SUNET/vc main as of 2026-10-20, so it also carries upstream #809 (mdoc DeviceAuthentication per ISO - # 18013-5) and #790 (SD-JWT holder binding) - # PRs https://github.com/SUNET/vc/pull/793 @ 3bfa96b6e (seed-backed commitment claims; MERGED upstream since) - # https://github.com/SUNET/vc/pull/821 @ fd078a057 (deliver the secret seed in credential_metadata; open) - # https://github.com/SUNET/vc/pull/813 @ 2e69dd2b4 (identity_attributes / document_claims; closed) - # https://github.com/SUNET/vc/pull/796 @ 62c93fa01 and https://github.com/SUNET/vc/pull/820 @ 6db9dd19b (verifier - # V1 / circuit_zk request; both closed, superseded by the verifier stack above) - # plus startup refusals for a delivering scope without notification_endpoint / credential_encryption.keys and a - # 'Delivered a secret seed' INFO line + # matcher https://github.com/sirosfoundation/siros-dc-matcher v0.11.0 (wallet side; not in the image); circuit catalog + # https://github.com/sirosfoundation/go-zk-circuits release v0.1.0-release.16 served from zk-circuits.fly.dev + # STANDARD PRACTICE until https://github.com/SUNET/vc/pull/840 is merged to vc main: every vc image pinned for an environment + # that sets issuer.allowPreauthOffer MUST be built from a vc SHA that contains it (an image without it ignores the flag + # silently, so a pre-authorized offer for a base scope fails with invalid_scope). tests/test_pins_carry_840.py checks + # that this block names #840 whenever the flag is on. Re-pin to released images once #840 and the stack are released. + # previous pins: verifier local-1792603698 (vc a56c4be9e, PROVEN flow); apigw/issuer/registry local-1792473614 (vc v4 + # 5e546b2d: #793, #821, #813, #796, #820); older verifier builds local-1792542047, -1792532233, -1792523015 # # wallet-backend (the ENGINE) registry.fly.io/sirosid-gdc-wallet-backend:local-1792564143 # digest sha256:285a7ca4ee1eaa27cacbe45435965ba08b15e2e98b680636a29ee5073da97749 @@ -162,10 +155,10 @@ images: # The verifier config (verifier.nym_auth, zkp_req formats, presets, trust.pdp_url) is under values.verifier below. # A real wallet presentation is expected to fail proof verification until the verification time T is agreed between # wallet and verifier (spec decision pending); that is not a config problem. - vc-verifier: registry.fly.io/sirosid-gdc-vc-verifier:local-1792603698 - vc-apigw: registry.fly.io/sirosid-gdc-vc-apigw:local-1792473614 - vc-issuer: registry.fly.io/sirosid-gdc-vc-issuer:local-1792473614 - vc-registry: registry.fly.io/sirosid-gdc-vc-registry:local-1792473614 + vc-verifier: registry.fly.io/sirosid-gdc-vc-verifier:local-1792609658 + vc-apigw: registry.fly.io/sirosid-gdc-vc-apigw:local-1792609658 + vc-issuer: registry.fly.io/sirosid-gdc-vc-issuer:local-1792609658 + vc-registry: registry.fly.io/sirosid-gdc-vc-registry:local-1792609658 # Pinned here rather than in values-fly.yaml because that file is shared by # every Fly environment and this is gdc's pin, not a vetted default for all @@ -361,6 +354,12 @@ rical_root_cert: fixtures/trusted-roots/geneva2026-rical-root.pem # without a code change anywhere else. values: issuer: + # SUNET/vc#840: mint pre-authorized offers for the BASE scopes, so there are no `_offer` twins (the chart drops them + # for an environment that sets this, sirosid_core/vc_render.py drop_offer_twins). Needs vc images built from a SHA that + # contains #840 (see PIN PROVENANCE: standard practice until #840 is merged to vc main; an older image ignores the key and + # a base-scope mint fails with invalid_scope), and #840's startup guard needs api_auth.jwks (or oidc) AND the admin UI + # off or oidc enabled. gdc: api_auth.jwks.enable true and admin_ui_enable false (its default), so the guard passes. + allowPreauthOffer: true apigw: # nym_auth's seed-backed ssc and its delivery (SUNET/vc#793/#821; the integration build pinned above): # the issuer derives each person's seed from a per-environment secret (generated once, PERMANENT) and the diff --git a/environments/nymauth.yaml b/environments/nymauth.yaml index bc54ec04..c45df4f2 100644 --- a/environments/nymauth.yaml +++ b/environments/nymauth.yaml @@ -18,8 +18,9 @@ # pinned verifier image was proven with them), wallet attestation, the sample app's signing keys. # # EVERY pinned image except the PDP and the frontend is a LOCAL BUILD OF UNMERGED PULL REQUESTS; re-pin to released images -# when they land (list in gdc.yaml's PIN PROVENANCE block). Twins: allowPreauthOffer is NOT set here (the pinned apigw -# lacks SUNET/vc#840), so `make datastore-offer SCOPE=nym_auth` mints against the nym_auth_offer twin. +# when they land (list in gdc.yaml's PIN PROVENANCE block). allowPreauthOffer is set (as in gdc) and the pinned vc images +# are built from a SHA that contains SUNET/vc#840, so there are NO `_offer` twins and `make datastore-offer +# SCOPE=nym_auth` mints against nym_auth itself. region: arn @@ -38,10 +39,10 @@ credential_registries: images: # Same pins as gdc (PIN PROVENANCE in environments/gdc.yaml): tag, digest, base commit and PR heads of every build. - vc-verifier: registry.fly.io/sirosid-gdc-vc-verifier:local-1792603698 # vc a56c4be9e + artifact overlay (zkp_req) - vc-apigw: registry.fly.io/sirosid-gdc-vc-apigw:local-1792473614 # vc v4 5e546b2d (seed delivery) - vc-issuer: registry.fly.io/sirosid-gdc-vc-issuer:local-1792473614 - vc-registry: registry.fly.io/sirosid-gdc-vc-registry:local-1792473614 + vc-verifier: registry.fly.io/sirosid-gdc-vc-verifier:local-1792609658 # vc c8c5b1ea + artifact overlay (zkp_req, #840) + vc-apigw: registry.fly.io/sirosid-gdc-vc-apigw:local-1792609658 # vc c8c5b1ea (seed delivery, #840) + vc-issuer: registry.fly.io/sirosid-gdc-vc-issuer:local-1792609658 + vc-registry: registry.fly.io/sirosid-gdc-vc-registry:local-1792609658 wallet-backend: registry.fly.io/sirosid-gdc-wallet-backend:local-1792564143 # engine 9cb6e57 (#469/#471/#163/#472) pdp: ghcr.io/sirosfoundation/go-trust:0.26.0 # released; verifier_ids (go-trust#218) wallet-frontend: ghcr.io/sirosfoundation/wallet-frontend:0.5.0-1-sirosid.11 @@ -82,6 +83,12 @@ wallet_attestation: true # without a code change anywhere else. values: issuer: + # SUNET/vc#840: mint pre-authorized offers for the BASE scopes, so there are no `_offer` twins (the chart drops them + # for an environment that sets this, sirosid_core/vc_render.py drop_offer_twins). Needs vc images built from a SHA that + # contains #840 (see PIN PROVENANCE: standard practice until #840 is merged to vc main; an older image ignores the key and + # a base-scope mint fails with invalid_scope), and #840's startup guard needs api_auth.jwks (or oidc) AND the admin UI + # off or oidc enabled. gdc: api_auth.jwks.enable true and admin_ui_enable false (its default), so the guard passes. + allowPreauthOffer: true apigw: # nym_auth's seed-backed ssc and its delivery (SUNET/vc#793/#821; the integration build pinned above): # the issuer derives each person's seed from a per-environment secret (generated once, PERMANENT) and the diff --git a/fixtures/vc-config-golden/accepted-diffs.yaml b/fixtures/vc-config-golden/accepted-diffs.yaml index 8689e5d3..5939ae0e 100644 --- a/fixtures/vc-config-golden/accepted-diffs.yaml +++ b/fixtures/vc-config-golden/accepted-diffs.yaml @@ -290,3 +290,18 @@ fly-gdc: - path: verifier.nym_auth kind: added reason: "gdc verifies nym_auth pseudonym proofs (SUNET/vc#832; environments/gdc.yaml verifier.extraConfig). Needs a zknative verifier image with the circuit artifact baked in (dockerfiles/verifier-nymauth.Dockerfile)." + - path: apigw.data_sources.datastore.scopes.diploma.allow_preauth_offer + kind: added + reason: "gdc sets issuer.allowPreauthOffer (SUNET/vc#840): an offer can be minted for the base scope, so no _offer twin is needed" + - path: apigw.data_sources.datastore.scopes.mdl.allow_preauth_offer + kind: added + reason: "gdc sets issuer.allowPreauthOffer (SUNET/vc#840): an offer can be minted for the base scope, so no _offer twin is needed" + - path: apigw.data_sources.datastore.scopes.mdl_zk4.allow_preauth_offer + kind: added + reason: "gdc sets issuer.allowPreauthOffer (SUNET/vc#840): an offer can be minted for the base scope, so no _offer twin is needed" + - path: apigw.data_sources.datastore.scopes.pid_1_8.allow_preauth_offer + kind: added + reason: "gdc sets issuer.allowPreauthOffer (SUNET/vc#840): an offer can be minted for the base scope, so no _offer twin is needed" + - path: apigw.data_sources.datastore.scopes.pid_mdoc.allow_preauth_offer + kind: added + reason: "gdc sets issuer.allowPreauthOffer (SUNET/vc#840): an offer can be minted for the base scope, so no _offer twin is needed" diff --git a/scripts/datastore.py b/scripts/datastore.py index 207c9617..afd43d29 100644 --- a/scripts/datastore.py +++ b/scripts/datastore.py @@ -57,6 +57,7 @@ def __init__(self, env: str = None, url: str = None): where = f"make fly-up ENV={env}" if env else "make up VC=yes" sys.exit(f"{config_path} or {self.key} missing - render this target first ({where})") config = yaml.safe_load(config_path.read_text()) + self.datastore_scopes = (((config["apigw"].get("data_sources") or {}).get("datastore") or {}).get("scopes")) or {} api_server = config["apigw"]["api_server"] self.auth = api_server["api_auth"]["jwks"] self.subject = spocp_subject(api_server["api_auth"].get("rules") or []) @@ -103,6 +104,25 @@ def spocp_subject(rules: list) -> str: sys.exit("no SPOCP rule with a concrete subject for /api/v1/* in the rendered api_auth block") +def offer_scope(scopes: dict, scope: str, document_id: str = None) -> tuple: + """The scope (and document id) an offer is minted against. + + vc mints an offer for a scope declared auth_provider: preauth (SUNET/vc requirePreAuthScope) or, from SUNET/vc#840, + a login-gated scope marked `allow_preauth_offer`. `scopes` is the RENDERED apigw datastore scope map, so this + follows what the environment's config says: a base scope with the flag is used as it is; otherwise the chart's + `_offer` twin is (`SCOPE=pid_1_8` means `pid_1_8_offer`); naming a twin always selects it. The twins' + documents carry `_offer_` in their ids (document_id_nym_auth_offer_alice-001); against the base scope the same + request names document_id_nym_auth_alice-001. + """ + if scope.endswith("_offer"): + return scope, document_id + if (scopes.get(scope) or {}).get("allow_preauth_offer"): + if document_id: + document_id = document_id.replace(f"_{scope}_offer_", f"_{scope}_", 1) + return scope, document_id + return scope + "_offer", document_id + + def offer(target: "Target", scope: str, document_id: str = None, authentic_source: str = None, qr: bool = False) -> None: """A pre-authorized credential offer for one datastore document. @@ -113,16 +133,13 @@ def offer(target: "Target", scope: str, document_id: str = None, authentic_sourc PID-authenticated types too - the document is chosen here rather than resolved from a presented PID. """ - # vc mints an offer only for a scope declared auth_provider: preauth (SUNET/vc requirePreAuthScope), and a - # scope has one provider, so the chart declares `_offer` twins; `SCOPE=pid_1_8` means `pid_1_8_offer`. - if not scope.endswith("_offer"): - scope += "_offer" + scope, document_id = offer_scope(target.datastore_scopes, scope, document_id) docs = target.request("GET", "/api/v1/datastore/search", query={"scope": scope, "limit": 200})["data"] if not docs: - sys.exit(f"no documents in scope {scope} - only types with an `_offer` twin in values-base.yaml can be " - f"offered; `search --scope {scope}` shows what is there (an existing environment needs " - f"`make datastore-sync` to get the twin's documents)") + sys.exit(f"no documents in scope {scope} - only datastore-sourced types can be offered (a type with an " + f"`_offer` twin, or a base scope marked allow_preauth_offer); `search --scope {scope}` shows what " + f"is there (an existing environment needs `make datastore-sync` to get the twin's documents)") if document_id: match = [d for d in docs if d["meta"]["document_id"] == document_id] if not match: diff --git a/sirosid_core/render.py b/sirosid_core/render.py index 34a8355b..5bb7ee12 100644 --- a/sirosid_core/render.py +++ b/sirosid_core/render.py @@ -712,6 +712,8 @@ def render(target: str, chart_dir: Path, env: str = None, android_apk_key_hashes vc_render.inline_file_refs(base, resources.root) vc_render.resolve_branding_refs(base, resources.root) vc_render.expand_presentation_request_templates(base, resources.root) + if ((env_values or {}).get("issuer") or {}).get("allowPreauthOffer"): + vc_render.drop_offer_twins(base) base_path = out_dir / "values.base-inlined.yaml" base_path.write_text(yaml.dump(base, sort_keys=False)) diff --git a/sirosid_core/vc_render.py b/sirosid_core/vc_render.py index 77976b87..f75b86f6 100644 --- a/sirosid_core/vc_render.py +++ b/sirosid_core/vc_render.py @@ -386,6 +386,20 @@ def is_twin_document(name: str) -> bool: return name.endswith(TWIN_DOCUMENT_SUFFIX) +def drop_offer_twins(values: dict) -> dict: + """An environment that sets issuer.allowPreauthOffer (SUNET/vc#840) needs no `_offer` twins: its base scopes can + be offered directly. Remove them from the base values - the credential types whose issuance.authProvider is + preauth and their bootstrapping documents - so the issuer's credential_configurations_supported, the datastore + scopes and the verifier's lists each carry one entry per type. Every other environment keeps them.""" + types = values.get("features", {}).get("credentialTypes") or {} + for name in [n for n, t in types.items() if (t.get("issuance") or {}).get("authProvider") == "preauth"]: + del types[name] + docs = ((values.get("issuer") or {}).get("datastoreImport") or {}).get("documents") or {} + for name in [n for n in docs if n.endswith("_offer")]: + del docs[name] + return values + + def drop_twin_imports(config: dict) -> dict: """Fly: the twins' documents are not shipped with the apigw, so they must not be imported either. diff --git a/tests/golden/fly_up.json b/tests/golden/fly_up.json index 39e3ccb5..db471235 100644 --- a/tests/golden/fly_up.json +++ b/tests/golden/fly_up.json @@ -168,7 +168,7 @@ "pdp.yaml": "00d8aee1ad455620", "pres-reqs/pres-reqs.yaml": "b0f8dff4515f79ce", "values.api-auth.yaml": "(volatile)", - "values.base-inlined.yaml": "f8a9f16b225f3572", + "values.base-inlined.yaml": "f1357d2024e2679d", "values.generated.yaml": "8d0085e00604c29a", "vc-apigw.fly.toml": "62a74456aeda6700", "vc-apigw.yaml": "11833e4e0cfe44ee", @@ -347,7 +347,7 @@ "pdp.yaml": "ec43dcc3736a4a3c", "pres-reqs/pres-reqs.yaml": "b0f8dff4515f79ce", "values.api-auth.yaml": "(volatile)", - "values.base-inlined.yaml": "f8a9f16b225f3572", + "values.base-inlined.yaml": "f1357d2024e2679d", "values.generated.yaml": "a24b0a8f6522c6fe", "vc-apigw.fly.toml": "62a74456aeda6700", "vc-apigw.yaml": "11833e4e0cfe44ee", @@ -526,7 +526,7 @@ "pdp.yaml": "941ff7f896171bde", "pres-reqs/pres-reqs.yaml": "b0f8dff4515f79ce", "values.api-auth.yaml": "(volatile)", - "values.base-inlined.yaml": "f8a9f16b225f3572", + "values.base-inlined.yaml": "f1357d2024e2679d", "values.environment.yaml": "a68fa91f82976efc", "values.generated.yaml": "96eb4c7dcadd8472", "values.toggles.yaml": "25be7b6793eda8a7", @@ -684,7 +684,7 @@ "pdp.yaml": "ec43dcc3736a4a3c", "pres-reqs/pres-reqs.yaml": "b0f8dff4515f79ce", "values.api-auth.yaml": "(volatile)", - "values.base-inlined.yaml": "f8a9f16b225f3572", + "values.base-inlined.yaml": "f1357d2024e2679d", "values.generated.yaml": "a24b0a8f6522c6fe", "vc-apigw.fly.toml": "62a74456aeda6700", "vc-apigw.yaml": "11833e4e0cfe44ee", @@ -1025,7 +1025,7 @@ "pdp.yaml": "077060a4843bdbc6", "pres-reqs/pres-reqs.yaml": "b0f8dff4515f79ce", "values.api-auth.yaml": "(volatile)", - "values.base-inlined.yaml": "f8a9f16b225f3572", + "values.base-inlined.yaml": "f1357d2024e2679d", "values.generated.yaml": "588f06bf89465652", "vc-apigw.fly.toml": "62a74456aeda6700", "vc-apigw.yaml": "7acf0a61335c1c9c", @@ -1204,7 +1204,7 @@ "pdp.yaml": "b5248c0c2bf703b3", "pres-reqs/pres-reqs.yaml": "b0f8dff4515f79ce", "values.api-auth.yaml": "(volatile)", - "values.base-inlined.yaml": "f8a9f16b225f3572", + "values.base-inlined.yaml": "f1357d2024e2679d", "values.generated.yaml": "656d3d5ad9b7759c", "vc-apigw.fly.toml": "62a74456aeda6700", "vc-apigw.yaml": "11833e4e0cfe44ee", diff --git a/tests/test_nymauth_environment.py b/tests/test_nymauth_environment.py index d8ddcfa8..08eea541 100644 --- a/tests/test_nymauth_environment.py +++ b/tests/test_nymauth_environment.py @@ -91,7 +91,9 @@ def test_the_issuer_delivers_the_seed_encrypted_with_a_notification_endpoint(sel self.assertEqual(apigw["seeds"]["primary"]["model"], "derived") self.assertIn("credential_encryption", apigw["issuer_metadata"]) self.assertEqual(apigw["issuer_metadata"]["notification_endpoint"], "https://sirosid-nymauth-vc-apigw.fly.dev/notification") - self.assertIn("nym_auth_offer", self.cfg["vc-apigw"]["common"]["credential_metadata"], "twins stay: no #840 in the pinned apigw") + meta = self.cfg["vc-apigw"]["common"]["credential_metadata"] + self.assertNotIn("nym_auth_offer", meta, "no twins: the pinned vc images carry SUNET/vc#840") + self.assertTrue(self.cfg["vc-apigw"]["apigw"]["data_sources"]["datastore"]["scopes"]["nym_auth"]["allow_preauth_offer"]) def test_no_other_environments_hostname_leaks_into_the_configs(self): for name, cfg in self.cfg.items(): diff --git a/tests/test_offer_without_twins.py b/tests/test_offer_without_twins.py new file mode 100644 index 00000000..7b06e8d9 --- /dev/null +++ b/tests/test_offer_without_twins.py @@ -0,0 +1,83 @@ +#!/usr/bin/env python3 +"""Offers without twins (SUNET/vc#840 `allow_preauth_offer`): the chart renders the flag only where an environment asks +for it, never on a preauth scope, and `make datastore-offer` follows the rendered config - the base scope when it is +flagged, the `_offer` twin otherwise, and a named twin always.""" +import shutil +import sys +import tempfile +import unittest +from pathlib import Path + +import yaml + +ROOT = Path(__file__).resolve().parent.parent +sys.path.insert(0, str(ROOT)) +sys.path.insert(0, str(ROOT / "scripts")) + +import datastore # noqa: E402 +import env_config # noqa: E402 +from sirosid_core.naming import Naming # noqa: E402 +from sirosid_core.render import render # noqa: E402 +from sirosid_core.resources import Resources # noqa: E402 + +NEEDS = unittest.skipUnless(shutil.which("helm"), "needs helm") + + +def render_scopes(env_values): + tmp = Path(tempfile.mkdtemp()) + try: + render("fly", ROOT / "chart", env="t", resources=Resources(ROOT), out_dir=tmp / "o", secrets_dir=tmp / "s", + env_values=env_values, naming=Naming(env="t"), say=lambda *a, **k: None, mongo_password="x") + cfg = yaml.safe_load((tmp / "o" / "fly-t" / "vc-apigw.yaml").read_text()) + return cfg["apigw"]["data_sources"]["datastore"]["scopes"] + finally: + shutil.rmtree(tmp, ignore_errors=True) + + +@NEEDS +class RenderedFlag(unittest.TestCase): + def test_off_by_default_because_an_image_without_the_guard_ignores_it_and_one_with_it_may_refuse_to_start(self): + scopes = render_scopes({}) + self.assertEqual([k for k, v in scopes.items() if "allow_preauth_offer" in v], []) + + def test_on_for_every_login_gated_scope_and_never_for_a_preauth_one(self): + scopes = render_scopes({"issuer": {"allowPreauthOffer": True}}) + flagged = {k for k, v in scopes.items() if v.get("allow_preauth_offer")} + gated = {k for k, v in scopes.items() if v["auth_provider"] != "preauth"} + self.assertEqual(flagged, gated) + self.assertIn("nym_auth", flagged) + self.assertTrue(all("allow_preauth_offer" not in v for v in scopes.values() if v["auth_provider"] == "preauth")) + + def test_gdc_turns_it_on(self): + values = env_config.load_environment_config("gdc").get("values") or {} + self.assertTrue(values["issuer"]["allowPreauthOffer"]) + + +class OfferScope(unittest.TestCase): + SCOPES = {"nym_auth": {"auth_provider": "openid4vp", "allow_preauth_offer": True}, + "nym_auth_offer": {"auth_provider": "preauth"}, + "mdl": {"auth_provider": "oidc"}, "mdl_offer": {"auth_provider": "preauth"}} + + def test_a_flagged_base_scope_is_used_as_it_is(self): + self.assertEqual(datastore.offer_scope(self.SCOPES, "nym_auth"), ("nym_auth", None)) + + def test_an_unflagged_base_scope_falls_back_to_its_twin(self): + self.assertEqual(datastore.offer_scope(self.SCOPES, "mdl"), ("mdl_offer", None)) + + def test_a_named_twin_is_never_rewritten(self): + self.assertEqual(datastore.offer_scope(self.SCOPES, "nym_auth_offer", "document_id_nym_auth_offer_alice-001"), + ("nym_auth_offer", "document_id_nym_auth_offer_alice-001")) + + def test_the_twin_style_document_id_names_the_base_document_against_a_flagged_scope(self): + self.assertEqual(datastore.offer_scope(self.SCOPES, "nym_auth", "document_id_nym_auth_offer_alice-001"), + ("nym_auth", "document_id_nym_auth_alice-001")) + self.assertEqual(datastore.offer_scope(self.SCOPES, "nym_auth", "document_id_nym_auth_alice-001"), + ("nym_auth", "document_id_nym_auth_alice-001")) + + def test_an_environment_without_the_flag_keeps_minting_against_the_twin(self): + self.assertEqual(datastore.offer_scope({}, "nym_auth", "document_id_nym_auth_offer_alice-001"), + ("nym_auth_offer", "document_id_nym_auth_offer_alice-001")) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_pins_carry_840.py b/tests/test_pins_carry_840.py new file mode 100644 index 00000000..9f5c6200 --- /dev/null +++ b/tests/test_pins_carry_840.py @@ -0,0 +1,51 @@ +#!/usr/bin/env python3 +"""Standing rule (until SUNET/vc#840 is merged to vc main): an environment that sets issuer.allowPreauthOffer must pin vc +images built from a vc SHA that contains #840. An image without it IGNORES the flag silently, so a pre-authorized offer for +a base scope would fail with invalid_scope and nothing in the config says why. The pins' provenance (environments/gdc.yaml's +PIN PROVENANCE block) must therefore name #840, and every environment that sets the flag must pin the same vc images as the +environment that records them.""" +import sys +import unittest +from pathlib import Path + +ROOT = Path(__file__).resolve().parent.parent +sys.path.insert(0, str(ROOT / "scripts")) + +import env_config # noqa: E402 + +VC = ("vc-apigw", "vc-issuer", "vc-registry", "vc-verifier") + + +class PinsCarry840(unittest.TestCase): + def flagged(self): + out = [] + for path in sorted((ROOT / "environments").glob("*.yaml")): + cfg = env_config.load_environment_config(path.stem) + if ((cfg.get("values") or {}).get("issuer") or {}).get("allowPreauthOffer"): + out.append((path.stem, cfg)) + return out + + def test_gdc_is_one_of_them(self): + self.assertIn("gdc", [n for n, _ in self.flagged()]) + + def test_the_provenance_block_names_840_and_the_sha_it_was_built_from(self): + text = (ROOT / "environments" / "gdc.yaml").read_text() + block = text[text.index("PIN PROVENANCE"):text.index("# Other images are released versions")] + self.assertIn("https://github.com/SUNET/vc/pull/840", block) + self.assertRegex(block, r"integration/gdc-nymauth-v5` @ [0-9a-f]{40}") + + def test_flagged_environments_pin_the_same_vc_images_as_gdc(self): + gdc = env_config.load_environment_config("gdc")["images"] + for name, cfg in self.flagged(): + for component in VC: + self.assertEqual(cfg["images"][component], gdc[component], f"{name}: {component}") + self.assertRegex(cfg["images"][component], r":local-\d+$|:\d", f"{name}: {component} is not a pinned build") + + def test_one_epoch_tag_for_all_four_vc_images(self): + """They are built together from one SHA and share one config shape: mixing builds crash-loops the older one.""" + gdc = env_config.load_environment_config("gdc")["images"] + self.assertEqual(len({gdc[c].rsplit(":", 1)[1] for c in VC}), 1) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_verifier_credential_metadata.py b/tests/test_verifier_credential_metadata.py index 409b04bf..bea9e67a 100644 --- a/tests/test_verifier_credential_metadata.py +++ b/tests/test_verifier_credential_metadata.py @@ -24,13 +24,12 @@ NEEDS = unittest.skipUnless(shutil.which("helm"), "needs helm") -def render_gdc(): +def render_blocks(values): tmp = Path(tempfile.mkdtemp()) try: - values = env_config.load_environment_config("gdc").get("values") or {} - render("fly", ROOT / "chart", env="gdc", resources=Resources(ROOT), out_dir=tmp / "o", secrets_dir=tmp / "s", - env_values=values, naming=Naming(env="gdc"), say=lambda *a, **k: None, mongo_password="x") - out = tmp / "o" / "fly-gdc" + render("fly", ROOT / "chart", env="t", resources=Resources(ROOT), out_dir=tmp / "o", secrets_dir=tmp / "s", + env_values=values, naming=Naming(env="t"), say=lambda *a, **k: None, mongo_password="x") + out = tmp / "o" / "fly-t" return {n: yaml.safe_load((out / f"{n}.yaml").read_text()) for n in ("vc-verifier", "vc-apigw", "vc-issuer")} finally: shutil.rmtree(tmp, ignore_errors=True) @@ -38,9 +37,11 @@ def render_gdc(): @NEEDS class VerifierHidesOfferTwins(unittest.TestCase): + """An environment WITHOUT issuer.allowPreauthOffer still has the twins (its images cannot offer a login-gated scope).""" + @classmethod def setUpClass(cls): - cls.cfg = render_gdc() + cls.cfg = render_blocks({}) cls.verifier = cls.cfg["vc-verifier"]["common"]["credential_metadata"] cls.apigw = cls.cfg["vc-apigw"]["common"]["credential_metadata"] @@ -65,7 +66,34 @@ def test_the_picker_sees_only_base_scopes(self): """The verifier's picker iterates common.credential_metadata wholesale; this is the count it renders.""" twins = [k for k in self.apigw if k.endswith("_offer")] self.assertEqual(len(self.verifier), len(self.apigw) - len(twins)) - self.assertEqual(len(self.verifier), 14, "gdc: 26 entries minus the 12 twins") + self.assertEqual(len(self.verifier), 14, "26 entries minus the 12 twins") + + +@NEEDS +class NoTwinsWhereOffersAreAllowed(unittest.TestCase): + """issuer.allowPreauthOffer (SUNET/vc#840): the base scopes are offered directly, so the chart carries one entry per + type everywhere - the issuer's credential_configurations_supported source included.""" + + @classmethod + def setUpClass(cls): + values = env_config.load_environment_config("gdc").get("values") or {} + cls.cfg = render_blocks(values) + + def test_no_twin_anywhere(self): + for service, cfg in self.cfg.items(): + self.assertEqual([k for k in cfg["common"]["credential_metadata"] if k.endswith("_offer")], [], service) + + def test_one_entry_per_type_and_nym_auth_is_still_issuable(self): + meta = self.cfg["vc-apigw"]["common"]["credential_metadata"] + self.assertEqual(len(meta), 14) + self.assertEqual(set(meta), set(self.cfg["vc-verifier"]["common"]["credential_metadata"])) + scopes = self.cfg["vc-apigw"]["apigw"]["data_sources"]["datastore"]["scopes"] + self.assertTrue(scopes["nym_auth"]["allow_preauth_offer"]) + self.assertEqual([k for k in scopes if k.endswith("_offer")], []) + + def test_the_twins_documents_are_not_imported(self): + files = self.cfg["vc-apigw"]["apigw"]["data_sources"]["datastore"].get("import", {}).get("file_paths", []) + self.assertEqual([f for f in files if "_offer" in f], []) if __name__ == "__main__": diff --git a/values-base.yaml b/values-base.yaml index a68bab17..8091dfd5 100644 --- a/values-base.yaml +++ b/values-base.yaml @@ -471,6 +471,14 @@ issuer: # under (fixtures/vc-bootstrapping/identity_mappings.json: mini-oidc). Older vc images read config with a plain # yaml.Unmarshal and ignore the key, so this is safe as the default for every environment. datastoreAuthenticSource: mini-oidc + # SUNET/vc#840 (`allow_preauth_offer`): let the admin mint endpoint make a pre-authorized offer for a LOGIN-GATED + # datastore scope, so a type needs no `_offer` twin. Rendered on every datastore scope whose auth_provider is + # not preauth. OFF by default: a vc image with #840 REFUSES TO START with the flag set unless + # apigw.api_server.api_auth configures jwks or oidc (and, with the admin UI on, oidc), and an older image ignores + # it silently (so minting for the base scope fails with invalid_scope there). Environments whose apigw image carries + # #840 and satisfies that guard turn it on (environments/gdc.yaml). `make datastore-offer` then mints against the + # base scope; see scripts/datastore.py. + allowPreauthOffer: false # One file per datastore-sourced scope, keyed by holder. Each scope has at # least one document covering only that type's mandatory claims and exactly # one whose `meta.document_id` ends in `-full`, covering mandatory AND every