ML-DSA signatures in CCF

[maxtropets]

An attempt to #7971 (ledger signing part), with already quite a few changes (the more effort I put in, the more I lean towards simplifying things).

This's an ☔ for subtasks/PRs, to keep track of PQ signing completion in general.

---

Why?

• As we want PQ update to be a live code update, and as we want identities rotation flexibility, there's a separate ticket to solve that: Flexible service identity rotation #8264.
• This ticket focuses on supporting multiple signing identities for new or DR-ed services.

Structural changes planned in this one

• New MLDSA PKI interface and impl (MLDSA PKI #8378)
• Identity schema changes (Introduce identity type map #8263, Introduce separate signing keys table #8304)
  • current service identity serves as TLS identity, it's no longer back-endorsed
  • signing identities are stored separately
    • public parts: in the new table
    • private parts: in node's memory
• COSE signatures schema change (COSE signatures becomes a map (was singleton value) #8334)
  • CLASSICAL=0 == ServiceKeySingleton for backwards compatibility
  • PQ=1 - new value
• Deprecate previous_service_identity and replace with previous_signing_keys which will hold multiple {KeyType->Key} ("Previous service signing keys" replace cert #8477)
• Snapshot validation on join requires pulling signing keys
• Introduce previous_signing_keys and mark previous_service_identity as deprecated, that table will be gone in 8.x
• Back endorsements
  • Schema changes (per identity type) (Identity back-endorsement becomes per-identity-type #8401)
  • Big change to endorsement chain handling, same KV change as for signatures
  • Separately, when recovering from a snapshot, a different endorsement handling mechanism is in use, needs handling different endorsement types
• Endpoints/interfaces to serve signing keys for external/internal use cases, currently TBD
• Config-enabled PQ on service-recover/create (with e2e coverage on AL4)

Workflow change

• On fresh network start or recovery, the identity choice is determined by the config
• Signing identities are distributed alongside the existing service TLS identity on Join request, always in sync (until rotation is solved)
• Removing identity type is currently prohibited, as in CLASSICAL+PQ to a single is disallowed
  • allows to have a realistic scope of changes, and keep endorsement chain contract
  • later on, we can come up with identity termination, but there's no plan for such a need rn, as HYBRID signing is what we're aiming for the foreseeable future

[achamayou]

A is a non-starter because it would mean that we would stop producing EC COSE signatures and break existing SCITT relying parties. Compatibility with Dual is not required, but COSE EC + COSE PQ is necessary for the foreseeable future.

I am wary about table re-org, I would have thought that we could just add a new key to the existing one.

[maxtropets]

I am wary about table re-org, I would have thought that we could just add a new key to the existing one

Current SIGNATURES / COSE_SIGNATURES are single value tables, they have no keys, so there's compatibility work required anyway

[achamayou]

That's what it looks like type-wise, but really (i.e. when serialised) they have a single key, a size_t 0.

[achamayou]

Ideally btw what we have is configurable PQC signatures, which support ML-DSA only at first, but can be set to other values later on as well. See the curve_id setting for node keys (except here it's for the PQC service identity).