[pournasserian]

Closes #2592

What

Fixes from the post-merge review of #2536, the job queue from ADR-0010.

• Lease heartbeat (Medium):
  • JobQueueWorker.KeepLeaseAsync renews at a third of the lease until the handler is done. The lease is counted from just before the claim.
  • A renewal that throws is tried again while the lease holds. The wait before the next try is a beat, or what is left of the lease when that is less, and each renewal is cancelled when the lease's end comes.
  • It gives up as soon as a renewal is refused because another worker holds the lease, or the lease's end passes with no renewal.
  • Once the handler is done, a renewal that fails no longer matters, so it can't stop the outcome being recorded.
• Lost lease:
  • The handler now runs on a token the heartbeat cancels when it gives up, so it stops instead of running alongside the next claimant.
  • The worker records nothing for that attempt, because the job is the next claimant's.
• Shutdown (Low):
  • A host that stops mid-job puts the job back in the queue, due at once, with its attempt given back, through the new IJobLeases.ReleaseAsync(jobId, owner). The update is guarded by the owner, like every other lease change.
  • If the release fails, the worker logs it, and the lease runs out as before.
• Retry delay (Low): doubling stops at MaxRetryDelay, so no attempt count overflows TimeSpan and skips RetryAsync.
• CONTRIBUTING.md: the heartbeat, the cancelled token a handler should honour, and the release on shutdown.

Why

A 20-minute handler with no progress that hit one transient database error at minute 2 lost its lease for good. Another worker then claimed the job and ran it at the same time. A graceful restart used up an attempt, so a job on its last attempt ended dead. The retry delay threw from attempt 36 with the 30-second default.

Choices worth a look:

• The deadline is the last successful renewal plus the lease, or the claim plus the lease before any renewal, measured on the host's TimeProvider. No wait and no renewal outlasts it, so a slow or hanging database stops the handler when the lease ends, not a beat or a command timeout later. After that another worker may claim the job.
• Nothing is recorded for a lost attempt. The row belongs to whoever holds the lease, and the guarded updates would change nothing anyway. Leaving it alone keeps the attempt count and the error the claimant sees.
• The heartbeat is a public static on the worker, like RetryDelay, so its rules are tested without a database or the wall clock.
• The slow-handler test now moves a manual clock past the lease twice over. Each step waits until the heartbeat has renewed at the clock's time, so it no longer races the machine's load. The clock starts on a whole second, because PostgreSQL keeps microseconds.
• Not in this PR: which handlers may run as the system (the checks lane), and cancelling a job on request.

Tests

Integration, each on SQLite and PostgreSQL:

• A handler that works past its lease without progress keeps the job and runs once, while the clock moves eight seconds past a three-second lease.
• A lease another worker took stops the handler at the next beat, and the worker records nothing. The row keeps the other owner, one attempt and no error, and the worker goes on to the next job.
• A host that stops while a handler runs leaves the job queued, due at once, with no attempt counted and no lease. A job allowed one attempt then runs and succeeds after a restart.

Integration, SQLite:

• A trigger makes a running worker's renewals fail for a second while the clock stands still. Once they work again, the clock moves past the lease twice over. The job is still the worker's, and it succeeds on its first attempt with one run.

Unit:

• A renewal that throws is tried again, and the lease is kept.
• Renewals that keep failing hold the lease while the clock stands still, and give it up once the clock passes the deadline, without another try.
• On the real clock, a renewal that hangs gives the lease up at its end.
• A refused renewal gives up at once.
• A renewal that fails as the handler finishes counts as done.
• The retry delay for attempts 36, 1,000 and int.MaxValue is the ceiling, and a zero delay stays zero.

The job tests passed three runs in a row on both providers. The full suite ran locally against a postgres:17 container as well as SQLite, and markdownlint passes. No new package and no new project reference.

Checklist

• Linked to an issue a maintainer has confirmed
• Targets v2
• Tests or docs updated where behaviour changed
• No tool-generated co-author trailers or footers