[pournasserian]

Closes #2611

What

The second slice of F-WHK (#2431): delivery. When a change set is published or reverted, every enabled subscription that wants the event gets one signed request. It is sent from the job queue with retries, and each attempt is logged (API-05, SEC-05, ADR-0010).

• Core:
  • [ExtensionPoint] IChangeSetEvents.PublishedAsync(published, reverted) is told about a publish inside its unit of work. A revert's publish also names the set it reverted.
  • WebhookPayloads builds the Standard Webhooks body: type, id (the delivery's id, which is also its webhook-id), timestamp and data. The data holds the set's id, rationale, sources and admin link, its items (kind, id, operation, version), and for a revert, the set it reverted.
  • WebhookSignatures signs it: v1, and the base64 HMAC-SHA256 of id.timestamp.body, keyed with the secret's bytes. It goes in webhook-id, webhook-timestamp and webhook-signature.
  • IWebhookDirectory gains DeliveriesAsync (the latest attempts) and SendTestAsync (a webhook.test delivery that appends webhook.test_requested). Both are for people only.
• Change sets: ChangeSetDirectory.PublishAsync calls the registered listeners just before it commits. This is a three-line edit in the F-CHG lane's file.
• Infrastructure:
  • WebhookPublishListener queues one webhook.deliver job per enabled subscription that wants the event. The job carries the body built at publish time with its own delivery id, so every attempt sends the same request under the same id.
  • WebhookDeliveryJob checks the subscription is still enabled and its host is still allow-listed, signs, and POSTs. The client behind the outbound allow-list handler follows no redirects and has a timeout.
    • A 2xx answer is delivered.
    • Any other answer, a timeout or a network error is thrown, so the queue retries it with its growing delay.
    • A disabled subscription is skipped, and a host taken off the allow-list ends the job as failed with no retry.
    • So does a secret that can't be decrypted, for example after the Data Protection keys were replaced. Every one of these cases is logged as an attempt.
    • Only the response's status is read, never its body.
  • WebhookDeliveries logs each attempt's status or error, duration and time, with migrations for both providers. It never holds the secret or the signature.
  • Webhooks__AdminUrl, __Timeout (10 s) and __MaxAttempts (6), validated at startup.
• Admin: /webhooks shows each subscription's three latest deliveries, and has a "Send test" button.
• CONTRIBUTING.md: the delivery, the payload and how a receiver verifies a request.

Why

S1 in #2431: when the catalog sync set publishes, the endpoint receives one signed request with the set and its items within seconds, and the S3 revert sends a revert event. A failing endpoint is retried with backoff, with the failures visible, and delivery never blocks publish.

Choices worth a look:

• A listener extension point, not a call to webhooks from publish. Publish knows nothing of webhooks. Notifications (M2) can listen the same way, and the edit to ChangeSetDirectory stays at one loop.
• Queued in the publish's unit of work. A delivery exists exactly when the publish committed, and publishing returns before any endpoint is called.
• The body is built at publish time and stored in the job, so a retry sends what the event said even if the set changed since.
• A revert's publish is one change_set.reverted event naming both sets, not a published event as well, so a receiver sees one event per user-facing action.
• No IHttpClientFactory. One client created as the model gateway's are, with the allow-list handler in front, so no package is added.
• The link is a path in the admin unless Webhooks__AdminUrl is set, since a job has no request to learn the admin's address from.
• Not in this PR: replaying a dead delivery by hand, other events, and per-user notifications (M2).

Tests

Unit: the signature is HMAC-SHA256 of id.timestamp.body with the secret's bytes, and a malformed secret is refused. The payload has the set, sources, items and link, absolute or as a path, and a revert names the set it reverted.

Integration, each on SQLite and PostgreSQL, with a real host, its job queue, and an endpoint on this machine:

• A publish sends one request.
  • The receiver's own check accepts its signature with the subscription's secret and rejects it with another.
  • The body names the set and its item.
  • The log holds one delivered attempt under the same webhook-id.
  • No table holds the secret's text.
• Publishing a revert sends change_set.reverted naming the revert set and the set it reverted.
• An endpoint that answers 500 and then 503 is retried. It gets the same id and body three times, and the log holds attempts 1 to 3 with 500, 503 and 200.
• A redirect is not followed. With one attempt allowed, the endpoint sees one request, the 302 is logged and the job is dead.
• A host taken off the allow-list in a later start sends nothing. The job fails on its first attempt with the reason logged.
• A disabled subscription and one that doesn't want the event get no job at all.
• A test delivery is sent, signed and audited, and a disabled webhook refuses one.
• A webhook disabled while a retry waits gets nothing more: the second attempt is logged as skipped, and the job succeeds.
• While attempts fail and then succeed, the worker logs the failure, and no log line contains the secret.
• An endpoint slower than Webhooks__Timeout is a failed attempt, "did not answer in time".
• An endpoint nobody listens on fails to connect on each of three attempts, waits at least 300 ms and then 600 ms between them, and the job ends dead.

Admin: "Send test" queues a delivery, the endpoint receives it, and the list then shows it as delivered with its status.

The full suite ran locally against a postgres:17 container as well as SQLite, and markdownlint passes. No new package and no new project reference.

Checklist

• Linked to an issue a maintainer has confirmed
• Targets v2
• Tests or docs updated where behaviour changed
• No tool-generated co-author trailers or footers