Feature/auth refactor

[stephen-dixon]

TLS transport encryption and OIDC bearer-token authentication

Read docs/authentication.md first — it is the reference for everything below, and
docs/authentication-deployment.md is the runbook. Between them they cover the behaviour
better than this description can.

At a glance

• Adds optional TLS and OIDC bearer-token authentication. Off by default: with
  ENABLE_AUTH=OFF (the default) nothing changes for any existing deployment.
• A client sets UDA_AUTH_TOKEN with a token it obtained elsewhere. The server verifies it
  once, at the handshake, against the issuer's published keys, then applies a claim
  policy deciding whether that bearer is allowed in. Verified claims reach plugins.
• A session lasts for the lifetime of the connection or the lifetime of the token,
  whichever is shorter. On expiry the server returns 706 and closes the connection.
• Refusals return stable numeric error codes (700–708, 710–712) and are recorded one
  JSON line each in refused_requests.log.
• Protocol bumped 10 → 11. Mixed estates are safe: a v11 client talking to a v10 server
  does not send the token at all. A v10 client hitting an auth-enabled v11 server is
  refused with 700 and told to upgrade.
• No new vendored dependencies. nlohmann/json and jwt-cpp are resolved with
  find_package, falling back to a pinned download. picojson was removed entirely.

Scale: ~9,800 lines across source, tests and docs. Roughly a third is tests.

---

1. "Does this affect me if I don't use it?"

• ENABLE_AUTH=OFF is the default; auth code is not compiled in.
• Five build configurations are verified: auth on/off × client-only/full, plus fat-client.
• Behaviour change for everyone: server logs now default to append instead of
  truncate. The server forks per connection, so the old default meant every new connection
  wiped the previous one's logs. They now grow and need rotation. UDA_LOG_MODE=w
  restores the old behaviour.
• Fixed: a per-request memory leak of up to 16 KB (the client block, token included, is
  re-received per request and was re-allocated without release).
• Fixed: IDAM_PLUGIN_INTERFACE was left uninitialised at five sites, so a plugin
  calling authPayloadValue() could read an indeterminate pointer.

2. "How does a request get authenticated?"

Path: udaClient.cpp → xdrlib.cpp → handshake_auth.cpp → oauth_authentication.cpp

• Client attaches the token to the client block when UDA_AUTH_TOKEN is set.
• xdr_authentication_block caps the payload at 16 KB before allocating.
• check_oidc_client_auth gates: protocol version, block present, no embedded NULs, then
  verification.
• authenticate() does discovery → JWKS (cached, TTL 300s) → signature → issuer →
  audience → claim policy. Algorithm is pinned by config, never taken from the token's
  own alg header.
• Key rotation is transparent: an unknown kid triggers one refresh and retry.

3. "What stops a bad token getting in?"

• Signature verified against the issuer's JWKS; exp/nbf always checked.
• A claim policy is mandatory. With no audience and no required claims the server
  refuses everything with 701 rather than accepting any valid token the issuer ever minted.
  UDA_SERVER_OIDC_POLICY=none opts out explicitly and warns on every connection.
• HTTPS enforced for discovery/JWKS unless explicitly overridden.
• TLS floor raised to 1.2 (previously only SSLv2 was excluded, leaving 1.0/1.1
  negotiable). CRL checking now covers the whole chain, not just the leaf.
• Tested attacks: algorithm confusion (HS256 signed with the RSA public key, correct
  kid), alg: none, tampered signature, malformed tokens.

4. "What happens when something is refused?"

• Client gets a stable code: 700 missing, 701 config, 704 invalid, 705 claim policy,
  706 expired, 707 bad issuer, 708 bad audience, 710–712 TLS.
• Server writes one JSON line to refused_requests.log with stage, reason code, peer,
  per-connection pid, and for TLS the offending certificate's subject and expiry.
• The audit log is append-only and independent of UDA_LOG — turning down debug
  logging does not switch off the audit trail.
• A TLS refusal cannot reach the client (it happens before UDA's protocol exists), so the
  server-side record is the only account of it.

5. "How do plugins use the claims?"

• authPayloadValue(key, pi) — flat claim.
• authPayloadPath(path, pi) — nesting and array indexing, e.g. realm_access.roles[0].
• authPayloadContains(path, value, pi) — membership.
• Paths use the same syntax as UDA_SERVER_OIDC_REQUIRED_CLAIMS; one shared engine
  (claim_access.cpp) backs both, so a policy path and a plugin path mean the same thing.
• HELP::authorise() demonstrates calling an external authorisation service.
• HELP::servermetadata() reports compile flags and effective runtime config.

6. "Can I see it work?"

• demo/oidc/ — docker compose up -d --build && ./demo.sh. Keycloak + a UDA server +
  an authz service, with a guided tour that prints each command before running it.
• Two users make the point: alice is in /uda-users and gets in; adam is a real,
  enabled user in /uda-observers who authenticates perfectly and is then refused with 705.
  Authentication is not authorisation.
• test/e2e/keycloak/run_matrix.sh — 44 automated checks against a real IdP and a real
  server. Can point at an existing deployment with UDA_EXTERNAL_SERVER=1.

---

Decisions taken (please confirm you agree)

• Auth is verified once per connection, not per request. Deliberate — no signature
  check on the data path.
• The token sent with later requests is ignored without being decoded. Bytes are
  consumed (the wire format is fixed by protocol version) and dropped.
• Session ends when the token expires, using the handshake token's exp plus the same
  clock skew used to verify it.
• Changing token mid-session is unsupported — reconnect to use a new one. UDA has no
  channel to answer a request while also saying "your new token was ignored".
• Token over a non-TLS connection warns, it does not refuse. Refusing would break local
  development against a plain-HTTP IdP. UDA_ALLOW_TOKEN_WITHOUT_TLS=1 silences it.
  Flipping this to a refusal is a one-line change in each of client and server.
• Server logs default to append (see §1) — needs log rotation.
• Dependencies resolved, not vendored; UDA_FETCH_DEPENDENCIES=OFF for air-gapped and
  packaging builds.

Deliberately omitted

• Revocation checking. Expiry is observed; revocation is not. Short tokens are the
  mitigation.
• Token acquisition/refresh. Clients obtain tokens externally.
• A warning channel for an ignored mid-session token.
• Token fingerprint in responses — a natural fit for future provenance work, and the
  point at which the server could report which token was actually in scope.
• uda-auth-tools — standalone OIDC/TLS diagnostic CLI, to ship separately as a PyPI
  package rather than in this PR.

Needs explicit review

1. HELP::servermetadata() exposure. Only reachable by authenticated callers when auth
   is on. On an auth-off server it discloses exact version, build date and compile flags to
   anyone who can connect. Low severity, but it is the tool you reach for precisely when
   auth is the thing that's broken. Decision deferred — accept or restrict.
2. Log append default. Fixes a real data-loss bug but needs rotation in place.
3. Claim policy mandatory by default. Anyone upgrading with UDA_SERVER_AUTHENTICATION
   set but no policy will see 701 until they set one. Intended, but it is a breaking change
   for such a configuration.
4. Protocol 11 rollout order. Servers first with auth off, then clients, then enable.
   Enabling auth while v10 clients remain locks them out with no workaround but upgrading.
5. .gitguardian.yaml declares the test realm's published dummy credentials file by
   file (not a directory glob, so a real secret added elsewhere is still caught).

Verification

• Builds: 5 configurations (auth on/off × client-only/full, fat-client).
• Unit: 10 auth suites, 135 cases, 393 assertions — plus the pre-existing suites.
• End-to-end: 44 checks against live Keycloak, run against three deployment kinds —
  spawned, launchd socket-activated, and containerised.
• Protocol compatibility: all 7 client/server combinations verified against real servers.
• Session expiry verified live with a 40-second token held across three requests.
• CI: unit tests now run on the ssl=ON job (they previously never ran), the SSL system
  tests were gated on a runner not in the matrix and now execute, and the OIDC e2e suite
  runs with log collection on failure.

[gitguardian]

⚠️ GitGuardian has uncovered 1 secret following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secret in your pull request

GitGuardian id	GitGuardian status	Secret	Commit	Filename
37429956	Triggered	Generic CLI Secret	b472fc5	test/e2e/keycloak/run_matrix.sh	View secret

🛠 Guidelines to remediate hardcoded secrets

1. Understand the implications of revoking this secret by investigating where it is used in your code.
2. Replace and store your secret safely. Learn here the best practices.
3. Revoke and rotate this secret.
4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider

• following these best practices for managing and storing secrets including API keys and other credentials
• install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.

---

^{🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.}

[stephen-dixon]

@jholloc do you have access to the gitguardian dash to resolve the spurious issue raised?