[FEAT] Valkey adapters for the state ports, the counters, and the Durable boot arm - #454
Open
justin13888 wants to merge 11 commits into
Open
Conversation
…t/valkey-adapters-403
`redis` 1.2.2 (the version the workspace declares; the caret pin resolved to 1.6.0 and is locked back with `--precise`) with `tokio-rustls-comp` so a `rediss://` URL terminates TLS in rustls, and `script` for the Lua scripts every multi-key mutation becomes. `testcontainers` and `testcontainers-modules` (`valkey`) as dev-dependencies for the env-gated live suite. No `bb8`: one multiplexed `ConnectionManager` is the whole of what a server talking to one Valkey needs. The Volatile state row in design/dependencies.md records the scope, the primitives and the two rejections (a pool, Redis Cluster). Refs #403
One adapter per port in `store::valkey` — sessions, upload sessions, the three ceremonies and the device-cohort map — and `counter::valkey` for the counter port, all over one multiplexed, self-reconnecting `ConnectionManager`. Every multi-key mutation or decide-and-write is one Lua script (`EVALSHA`, `SCRIPT LOAD` on `NOSCRIPT`): the finalize claim, the challenge consume, the enrollment redeem and the counter hit cannot be read-then-written because there is no read a caller performs separately. Every derived index — the per-user and per-uploader sets, the per-album and pending-address sets, the global progress sorted-set — resolves each member through its record inside the script and drops a stale one, so an expired record leaves no listing entry behind without a second lifetime on the index. Expiry is decided by the injected `Clock`, written into each record as `expires_at`; `PEXPIRE` on the same key is only the collector. That is what lets the shared conformance suite drive this adapter with a manual clock exactly as it drives the in-memory double, one nanosecond either side of a boundary, with no sleeps. The counter port's cases move from `counter/tests.rs` into `counter::conformance`, generic over the store and with a racing case on a multi-threaded runtime; the in-memory adapter runs them case by case and in one pass. `tests/valkey.rs` runs both suites, a contested finalize claim and a contested counter hit against a `valkey/valkey` container (`CAPSULE_TEST_VALKEY=1`) or a running server (`CAPSULE_TEST_VALKEY_URL`), and passes as skipped otherwise; `.config/nextest.toml` places it in the one-thread `containers` group. Refs #403
`assemble` on `Backends::Durable` now connects to `VALKEY_URL`, proves it answers `PING`, and builds every Valkey store before anything else is assembled; a server that cannot be reached is `BootError::Valkey`, whose detail never carries the URL. The Postgres half is not written, so `durable` then refuses as `AdapterUnavailable` naming `DATABASE_URL` and in-memory doubles — #402 fills that one function's body. `.env.example`, the local-development page and the S-C29 slice row say what `serve` does with each variable now. Refs #403
Deploying capsule with
|
| Latest commit: |
e05c633
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://204df76f.capsule-22k.pages.dev |
| Branch Preview URL: | https://feat-valkey-adapters-403.capsule-22k.pages.dev |
- A failure the driver cannot place before the command was sent — a response timeout, a connection dropped mid-flight — is `Rejected`, not `Unavailable`: the script may already have burned the challenge or won the claim, and the port defines `Unavailable` as "certainly did not happen". Only a refused connection and a server that declined to execute (`LOADING`, `TRYAGAIN`, `MASTERDOWN`, `CLUSTERDOWN`) stay `Unavailable`. - A reply of the wrong shape is `Corrupt` with the driver's error *kind* only: redis-rs quotes the offending value in a type error, and for the ceremony stores that value is the record carrying the bearer secret. - A derived index set's TTL is only ever raised (`extend` in Lua), so one member's remaining life cannot shorten another's; and re-opening an upload under a different uploader, owner, hash or album unindexes the previous record, as `OPEN_SESSION` already did for a previous user. - Listings report expired members (`debug`) apart from mismatched ones (`warn`), so the routine heal is not the drift signal. - The eviction query over-fetches one page so the members sharing the horizon's microsecond do not cost a candidate; `peek` on an unopened window answers `Admitted` for every limit, as the double does; the counter adapter reuses the store's `Lua` type. Refs #403
`CAPSULE_TEST_CONTAINER_USERNS` (for instance `keep-id`) is passed to testcontainers as the container's userns mode, the knob a rootless podman may need to run the image at all. Refs #403
The binary smoke asserted the refusal the Durable boot arm replaced — a set `VALKEY_URL` naming #403 as unimplemented. The arm now connects to it, so the case points at a port nothing listens on and asserts what the new refusal promises: a non-zero exit, `VALKEY_URL` named, and the address never echoed. The reachable case lives in `tests/valkey.rs`. Refs #403
…apter `UploadSessionStatus::is_evictable` — `Pending` or `Uploading` — is now the one predicate `least_recently_progressed` applies. The in-memory double used `is_active`, which kept a `WaitingForProcessing` session a pressure-eviction candidate after a finalizer had claimed it; the port's `claim_finalize` doc promised the opposite, and the upload protocol's finalization claim is that promise. The conformance suite gains `a_claimed_session_leaves_the_eviction_view` so both adapters are held to it, and the `set_status`/`least_recently_progressed` docs name the real predicate. Refs #403
`live()` answers whether a record has passed its logical `expires_at` and leaves the key for `PEXPIRE` to collect: a replica whose clock ran ahead must not delete, for every other replica, state that is still live by the store's own lifetime, and the derived indexes already heal on a miss. The live suite asserts a logically expired record is absent to a reader and still present with a collector TTL. Also: `PEXPIRE` lifetimes round up to whole milliseconds so the collector never runs ahead of the logical lifetime; `NOSCRIPT` is classified `Unavailable` (nothing executed, retryable) and the classification is unit-tested, including that a corrupt reply's text never reaches the error; the pending-address key asserts its two variable segments carry no `:`; the module doc records the microsecond floor and that the progress view heals only when the pressure sweep runs; the adapter uses the shared `UploadSessionStatus::is_evictable`. Refs #403
…ect reads A false "not live" verdict from a fast clock costs only that replica a miss on the direct-read scripts. The index-listing scripts remove the member from a shared index, so it hides a still-live record from every replica's listings until a state change or real expiry: the residual accepted in decision 12 for not requiring synchronised clocks, now stated where the gate is documented. Refs #403
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Lane W-VALKEY of the unattended backlog run (issue #403), stacked on #435 (
feat/server-binary-config-operator-commands-401).Summary
capsule-server/src/store/valkey.rs:redis-rsadapters for all six state ports (AuthStateStore,UploadSessionStore,ChallengeStore,EnrollmentStore,ChannelStore,CohortStore) over one multiplexedConnectionManager, plus theValkeyStoresbundle. Every multi-key mutation or decide-and-write is one Lua script (EVALSHA,SCRIPT LOADonNOSCRIPT); every derived index heals on read; expiry is decided by the injectedClockand written into each record, withPEXPIREas the collector.capsule-server/src/counter/valkey.rs:ValkeyCounters, one hash and one script per key — open, charge or refuse in one server-side step.capsule-server/src/counter/conformance.rs: the counter port's suite, extracted fromcounter/tests.rsand generic over the store, with a racing case on a multi-threaded runtime; the in-memory adapter now runs it case by case and in one pass.capsule-server/tests/valkey.rs: the store and counter suites plus a contested finalize claim and a contested counter hit, against avalkey/valkeycontainer (CAPSULE_TEST_VALKEY=1) or a running server (CAPSULE_TEST_VALKEY_URL); skipped with one line otherwise..config/nextest.tomlputs it in the one-threadcontainersgroup.capsule-server/src/boot.rs: theDurablearm connects to andPINGsVALKEY_URLfirst (BootError::Valkeyon failure, never quoting the URL), then refuses namingDATABASE_URLand server: Postgres adapters and a conformance suite for every durable port #402 until the Postgres half lands..env.example,local-development.md,dependencies.md(Volatile state row) andSLICES.md(S-C29) say so.Validation
Run inside the worktree at head
eb1e8d2b(review round 1 applied), foreground,CARGO_TARGET_DIR=/var/tmp/capsule-lane-403/targetunless noted (the scratch volume was full when the lane started; the earliertargetsymlink is removed, F4).cargo nextest run -p capsule-serverDOCKER_HOST=unix:///run/user/2000/podman/podman.sock CAPSULE_TEST_CONTAINER_USERNS=keep-id CAPSULE_TEST_VALKEY=1 cargo nextest run -p capsule-server --test valkey --no-fail-fasta_claimed_session_leaves_the_eviction_view), the whole counter suite, the 16-way finalize race, the 40-way counter race, the durable arm reaching Valkey then namingDATABASE_URL/#402, anda_logically_expired_record_is_dead_but_left_for_the_collector; each test started its ownvalkey/valkey:9.0.4container through the podman socketCAPSULE_TEST_CONTAINER_USERNS(earlier head)WaitContainer(WaitLog(EndOfStream([])))— this host's rootless podman cannot read the image entrypoint withoutkeep-idcargo nextest run -p capsule-server --test binarymise run doc-check-rustmise run check-docs-truthmise run check-mdmise run check-rustgen-bindings—mise-tasks/gen-bindingshardcodestarget/debug/…andtarget/bindings, so it cannot find the library underCARGO_TARGET_DIR. The 13 sub-tasks before it (format-check, lint-check, doc-check, i18n-check, i18n-guard, openapi-check-kynos, architecture-check, license-check, translate-readme-check, build-rust, build-check-wasm, build-ffi, lint-check-ffi) passed.CARGO_TARGET_DIR; sub-tasks re-run belowmise run gen-bindings(default target dir)mise run verify-examples(default target dir)mise run test-rustcargo nextest run --workspace1824 passed;-p capsule-core --features ffi729 passed;-p capsule-sdk --features ffi160 passedmise run check-commits(ata959cb36)mise run check-rustexit 0 ata959cb36with the worktreetarget/in place;mise run test-rustexit 0 at0af7870c(1820 passed)Risks and rollout
Backends::Memoryare untouched; the Valkey adapters are reached only throughBackends::Durable, which still refuses at the end of the boot path until server: Postgres adapters and a conformance suite for every durable port #402. Nothing persists a schema; every key is undercapsule:and carries a TTL except the cohort hash.CohortStoreon Valkey is interim (decision 3); the key has no TTL so the data migrates with oneHGETALLper account.abd6a5a3,dc447549,eb1e8d2b.Related Issues
Closes #403. Refs #402 (the Postgres half of the
Durablearm,DATABASE_URL).Decisions taken
Unresolved review notes
capsule-server's rustdoc is not gated bydoc-check-rust, andcapsule-server/src/lib.rs:72and:98carry two pre-existing ambiguous intra-doc links ([openapi]); for the gates lane.mise-tasks/gen-bindingshardcodestarget/debug/…andtarget/bindings, somise run check-rustcannot pass under aCARGO_TARGET_DIR; out of manifest, recorded for the gates lane.capsule:challenge:{token},capsule:enroll:code:{code}), as in the retired server, so they appear inSLOWLOG/MONITOR. The record values carry the same secrets, so hashing the key names alone would not remove them fromMONITOR; a follow-up if the operational exposure matters.