v0.34.0: the release that fixes the gates - #140
Merged
Merged
Conversation
…-kept list
The mutation gate's file list lived in ci.yml and nothing checked it for
completeness, so coverage drifted as files gained trust decisions. linestatus.rs
gained the two-document preference in v0.33.0, nobody noticed it was outside the
gate, and a mutant permitting YANK SUPPRESSION survived until someone chose to
run the tool on an ungated file. Choosing to look is not a control.
Measured properly this time: 63 source files across the three crates, 34 gated,
29 not. My own requirement said "16 of 39", having counted only varve-core —
the number was smaller than the problem.
Every file is now either in the gate or declared in `mutation-scope.toml` with
one of three reasons, and a file that is neither FAILS. The point is not the
current split, which is mostly debt. It is that a NEW file cannot land ungated
without someone writing down why.
The three reasons are meant to be arguable:
bin-target kill criteria are `--workspace --lib`, so no lib test can kill a
mutant in a binary crate's modules. Gating those would report
survivors nothing could ever kill — including docs.rs, whose 24
tests live in the bin crate.
re-export declarations only; nothing to mutate.
not-yet trust-relevant and NOT gated. Honest debt, COUNTED every run so it
cannot become permanent by being quiet, and capped at its current
22 with a cap that only ratchets down.
Five assertions, and each fails for a different reason: an undeclared file, a
file both gated and excused, a stale declaration, a stale GATE entry (a rename
there removes a file from testing while the line still reads as coverage), and
the backlog exceeding its cap.
Both directions proved rather than assumed. A scratch file added to the crate
failed the check by name; renaming a gated file to something nonexistent failed
the stale-entry check. Restoring afterwards over-reverted ci.yml and dropped the
legitimate edit with the probe — caught by re-running, which is the argument for
having the check at all.
The check found its own file on the first run: mutationscope.rs was neither
gated nor declared. It is gated now, and cargo-mutants finds ZERO mutants in it,
because a const and some tests are nothing to mutate. Zero survivors out of zero
mutants proves nothing, and the file says so rather than letting a number imply
otherwise. It is gated anyway so that logic added there later is covered without
anyone remembering.
Clause 3 — linestatus.rs, lineindex.rs and registry.rs INTO the gate — is not
done here. They are declared `not-yet` with what they cost, and closing them is
the next commit rather than a claim made in this one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019TNtfRjLNhEz82G2ggeeNu
…r the gate Layer 2026.09.2 was deposited on 2026-09-09 and was ALREADY four tools behind when it landed: rivet v0.35.0 against v0.36.0, synth v0.61.0 against v0.65.0 — four minor versions — meld v0.53.0 against v0.55.1, kiln v0.4.4 against v0.5.0. Nothing was watching. The one scanner in the organisation lives in varve, reads its pins out of the LEGACY deposit workflow's env-var encoding, and has been failing since 2026-09-06 because the realm moved out from under it. It only ran on a schedule, where nobody looks. `varve-producer scan` replaces it. Not a style preference: the first version of this was a Python scanner and a bash id-deriver dropped into a workflow — untested, ungated, unversioned, unsigned, and destined to be copied per realm. A program that decides what gets SIGNED belongs where the tests and the mutation gate are, not in a YAML heredoc nothing can run locally. It ships in the layer with everything else. Pins come from the realm manifest and nowhere else. A second place the realm is defined is a place the two disagree, which is precisely how the existing scanner broke. AN UPSTREAM THAT CANNOT BE ASKED IS AN ERROR, never "nothing moved", and the type makes that unrepresentable rather than merely discouraged. A scanner that reported no movement because it could not ask would freeze the realm while every check stayed green — releases would simply stop arriving and nobody would be told. Silence is the expensive failure here; loud is the cheap one. A repository MISSING from the answers counts the same way, because a lookup loop that skips one leaves the map short and an under-report reads exactly like calm. A blank `tagName` is refused rather than compared. Empty string against a pin reports every payload as moved, and with nobody reading the output that gets signed. Autonomy is scoped to a channel that promises nothing. `rolling` says outright that it makes no qualification claim, so a layer arriving unattended is consistent with what its consumers were already told; `qualified` promises the opposite. Read from the manifest rather than taken from a flag — a flag is a claim, the realm's file is the fact — and an UNKNOWN channel is refused rather than allowed by omission, so a new channel must opt in to being signed without a person. Ten tests against fixtures, because the failure being replaced went unnoticed for three days precisely because nothing ran it anywhere but a cron. Zero mutation survivors. The scope check from the previous commit caught scan.rs as undeclared before I remembered to gate it, which is the second time today it has found a file its author had not thought about. STILL BASH, and not claimed otherwise: `next-layer-id.sh` derives the layer id and counter from the published record. Porting it is the other half of "all in Rust" and is not in this commit. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019TNtfRjLNhEz82G2ggeeNu
Asked to make sure the producer works on GitHub Enterprise, and the answer was that it did until an hour ago, when I broke it. `forge_env` exists precisely for this: `GH_HOST` is what `gh` itself uses to target an instance, so varve passes the same variable rather than inventing a second one. `source.rs` passes it on both of its `gh` invocations. The scan handler I added passed `&[]`. On an enterprise instance every lookup would have failed — or, far worse, SUCCEEDED against a same-named repository on the public forge. A wrong answer that arrives is more dangerous than no answer, and this one feeds an unattended depositor: the scanner would report movement that public github saw, the manifest would be bumped to a version the enterprise upstream never published, and the deposit would then fail somewhere further along with a confusing message about a missing release. Audited the rest rather than fixing only my line. `source.rs` was already correct in both places. The `oras` calls pass no forge environment and should not: a registry host comes from the repository string itself, not from `GH_HOST`. Cosign gets its identity and issuer from the Forge already. Every remaining mention of github.com in the producer is a test fixture or a comment. The loop moved out of `main` and into `scan::latest_releases` so the property is TESTABLE rather than asserted. Three tests now hold it: every lookup on an enterprise forge carries GH_HOST; public github gets no host override, because setting one there is how a working setup starts failing for a reason nobody can see; and a missing `gh` is named as itself rather than reported as upstream silence. One thing the move surfaced while I was there: the loop asks once per REPOSITORY, not once per payload. varve ships `varve` and `varve-producer` from one repo, and at four scans an hour a duplicate query is the difference between comfortable and rate-limited. Tested. Verified against the real binary: with GH_HOST set to an instance that does not exist, all ten lookups fail against THAT host and the command exits 1. An incomplete scan is never "nothing moved". Zero mutation survivors. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019TNtfRjLNhEz82G2ggeeNu
Ports tools/next-layer-id.sh, the last piece of the autonomous pipeline that was
still a shell script parsed out of a workflow. Nobody types a layer id any more,
so the derivation has to be right with no one reading it — and a wrong one is
unrecoverable, because varve has neither revocation nor deletion and a spent id
is spent.
The logic is unchanged from the bash, including the part most worth preserving:
the counter comes from the published BASELINE, never from the patch number.
Advisories issued by `sign-status` between deposits also advance the line
counter, so a deposit that reused one of those numbers would break the per-line
anti-rollback ordering that is varve's whole point. Layer 2026.09.2 carries
counter 3 — they are already unequal in the real realm.
What changes is that it is now tested. Twenty-three tests, against fixtures and
known instants rather than a cron job: the highest is numeric and not
lexicographic (string ordering answers "2026.09.2" when .10 exists, and spends
an id that is already published); only a bare decimal suffix is a layer, so
`2026.09.2-rc1` does not silently become layer 2; a computed id that already
exists is refused; a layer with no baseline refuses rather than inventing a
counter; and anything unreadable in the envelope refuses rather than guessing.
MUTATION TESTING CAUGHT ME SMOKE-TESTING A CALENDAR. I hand-rolled
civil-from-days and covered it with a test that asserted the SHAPE of the output
— four digits, two digits, a plausible range. Twenty-four different mutations of
that arithmetic satisfy those assertions, and all twenty-four survived. The
function now takes a unix time instead of reading the clock, so it can be given
known instants, and the thirteen expectations were cross-checked against
`date -u -r` rather than computed in my head — including the three leap rules
(2000 by the 400 rule, 2004 by the 4, 2100 excluded by the 100) and the month
boundary, which matters because a scanner running every fifteen minutes WILL run
in the minute a line rolls over.
The twenty-fifth survivor was a redundant `p.is_empty()` guard: `parse::<u32>("")`
already fails, so the check could not change an outcome, which is exactly why
mutating it survived. Removed rather than tested around — a guard that cannot
alter a result is not caution.
75 mutants, 75 caught, 0 survivors.
Verified against the real registry: `next-layer --repo ghcr.io/pulseengine/layers`
answers `2026.09.3 4`, which is right — 2026.09.2 is published and carries
counter 3.
The scope check caught nextlayer.rs as undeclared before I remembered to gate
it. Third time today.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019TNtfRjLNhEz82G2ggeeNu
…nts binaries varve's docs gate is real and works — every subcommand must have a topic, mechanically, so a new one cannot ship undocumented. It covered ONE of the two shipped binaries. varve-producer had nine subcommands and zero topics, while being the program other repositories' CI actually runs, and the one whose asset template language has silently dropped a tool from a published layer. Ten topics now, compiled into the producer: one per subcommand plus the ingest ladder. `docs`, `docs <topic>`, `--grep`, `--format json`, and `docs check --coverage --strict` for a gate. No files, no network — a producer often runs where there is neither. Documenting it inside `varve docs` would NOT have discharged this, and the requirement says so: a CI job holding the producer may not hold varve, and sending someone to another binary for the manual is the friction this removes. CLAUSE 3 IS THE ONE WORTH THE EFFORT. Naming varve-producer as the second gated binary would repeat the mistake one binary later, so the check enumerates them: a crate with `src/main.rs` is a shipped binary, and each must have a `src/docs.rs` carrying a `coverage_gaps` built from its OWN cli. A third binary fails the build until it has one. The CLI moved from `main.rs` into `cli.rs`, and not for tidiness. The gate's kill criteria are `--workspace --lib`, so a CLI defined in a binary cannot be enumerated by a lib test — the invariant could not be asserted where it runs. That move broke an immutability test which read the flag definition out of `main.rs`; its intent was unchanged, so it now reads `cli.rs`. MUTATION TESTING FOUND THE GATE COULD NOT FAIL. `coverage_gaps` was replaceable by `vec![]` with nothing noticing, because the only test asserted the real CLI has no gaps — which an empty list satisfies for everything. A gate proven only by passing is the shape this whole release is about. It is now given a synthetic command with an undocumented subcommand and must name it. Two more from the same run. `render_list` was replaceable by "xyzzy": nothing asserted the list contains the topics it lists. And the `!= "help"` filter was DEAD — clap does not report its generated help subcommand from `get_subcommands()` here, so the comparison could be inverted with no effect. Removed rather than tested around, along with the `-1` that corrected a count for a subcommand that was never in it. That `-1` is why the coverage line used to read 8; it reads 9 now, which is the true number. The scope check from earlier in this release caught cli.rs and docs.rs as undeclared before I remembered to gate them. Fourth time today. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019TNtfRjLNhEz82G2ggeeNu
`rivet validate` reports a disconnected artifact as a WARNING, among 212 of them, so the one that matters is invisible. The gate's own central invariant was advice. It is an error now. I CORRECTED THE REQUIREMENT BEFORE IMPLEMENTING IT. Clause 5 said the 38 existing orphans must be closed or excepted before the error could be turned on. There is no such backlog. rivet's orphan warning counts GRAPH LINKS ONLY, so a requirement discharged by a source marker reads as disconnected while carrying real evidence — which is what the clean-room reviewer saw, and what I wrote down without checking. Measured across all 125: 97 verified, 85 with a marker, 75 with an incoming edge, 63 with both, ZERO with neither. Satisfiable today, so turned on today. EVIDENCE IS EITHER KIND, and that is not a compromise. A source marker discharges a property of the CODE. An incoming `verifies` edge discharges a property of the PIPELINE — that fuzzing runs, that the matrix covers four platforms, that the mutation gate is required — which no unit test can assert. Demanding a marker for those would push someone to write a fake test to satisfy a gate, which is worse than the gap. Requiring only edges would mean authoring 85 verification artifacts for requirements that already have real evidence. Two refusals, both exercised rather than reasoned about: a requirement set to `verified` with nothing behind it exits 1 and names it; a marker naming REQ-DOES-NOT-EXIST-999 exits 1 and says it verifies nothing. Removing each returns 0. The second is the one that rots quietly — a rename leaves a marker behind that discharges nothing while `rivet coverage` counts it. It refuses to pass when it finds NO verified requirements at all, because a gate reading the wrong path would otherwise report success by finding nothing. And it refuses to run without PyYAML rather than skipping, for the same reason. The gate's own requirement satisfies the gate: VER-TRACEGATE-001 carries the measurement and the falsification, and REQ-TRACEGATE-001 is verified through it rather than by assertion. Also splits the mutation shard BEFORE it cancels. `varve-core (layers)` ran 54m46s against a 60-minute cap — five minutes of headroom, on the shard that WAS cancelled at 60:16 two releases ago, with 22 files declared `not-yet` waiting to be gated into it. Split into `export` and `lines`, roughly 2.9k and 3.1k lines. Waiting would have cost more than acting: a cancelled shard reports nothing, so zero-survivor goes unverified while the check looks like it ran. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019TNtfRjLNhEz82G2ggeeNu
Reading artifacts from Rust needs a YAML parser, and the obvious one is unmaintained — serde_yaml 0.9.34+deprecated, last published 2024-03-25. Adding it to a supply-chain tool to satisfy a test-only need is the wrong trade, so the gate is a script and varve's dependency graph is unchanged. Recorded as a stopgap rather than left as a silent choice. rivet has a rowan-based LOSSLESS YAML CST parser with two fuzz targets — the thing that lets `rivet modify` rewrite an artifact without destroying its comments — and pulseengine/rivet#930 asks for it as its own crate. When that lands this belongs in Rust beside the other gates, where `cargo test` runs it without anyone remembering to. I had claimed in that issue that rivet has no YAML implementation, having read its Cargo.toml and not its src/. It has about 240KB of one. Corrected there. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019TNtfRjLNhEz82G2ggeeNu
Two defects, both found by pointing the scanner at the realm's real manifest rather than at a fixture — which is the argument for running a tool on the thing it was written for before believing it. IT COMPARED A VERSION AGAINST A RELEASE TAG. `pulseengine/jess` tags `v0.7.2` and ships `with-device` at `0.2.2`; the scanner compared `0.2.1` against the latest TAG and reported movement. It happened to be right, for the wrong reason, and would have reported that payload as moved on every scan forever — `0.2.1` is never equal to a `v`-prefixed tag. The consequence is worse than noise. An unattended depositor acting on that would write the TAG into `version`, putting `v0.7.2` in a signed manifest for a binary that answers `0.2.2` — the layer stating something untrue about its own contents, which is the one thing it exists not to do, and precisely what REQ-PAYLOADID-001 added the `release` field to prevent. The comparison now uses the release tag, and a hub payload is marked NOT AUTO-BUMPABLE: the new tag is known, the new payload version is not, and only upstream's release notes say what it is. It is reported for a person and never acted on. IT WALKED `tools` AND NOT `vsix`. Two payloads were never scanned at all, and the realm had already drifted behind them: `rivet-sdlc` sat at v0.35.0 while the rivet TOOL moved to v0.37.0, and every scan answered "nothing moved". A scanner blind to a payload kind reports calm about a realm that is drifting — the same silence this module refuses everywhere else, arriving through a section of the manifest I did not think about. Both kinds are now one list, because the scanner has no reason to care which table a payload was written in. Five tests: a hub payload at its pinned tag has not moved; a moved one reports both numbers; it is never auto-bumpable while an ordinary payload is; a vsix behind its upstream is reported; and an unreachable vsix upstream is an incomplete scan rather than a silent skip. 17 mutants, 16 caught, 1 unviable. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019TNtfRjLNhEz82G2ggeeNu
Four requirements, all verified. v0.33.0 shipped four capabilities that were complete, tested and carried by no artifact, and a fifth found inside the fix for the fourth. Behind them was one pattern — each gate was real, worked, and had a boundary nothing checked. This release checks the boundaries. The mutation gate's scope is decided by a check: 63 source files, 34 gated, 29 declared with a reason, and a file that is neither fails. The docs gate reaches every shipped binary rather than a named one. A `verified` requirement with nothing behind it fails instead of warning. And `varve-producer scan` replaces a shell script that had been dead for three days in a repository the realm had already left. The part worth reading is what mutation testing found in this release's OWN new code: a docs gate that could not fail, a calendar covered by a test asserting only the shape of its output, and a scanner that asked github.com whatever the forge said. Each was written by someone who believed it worked. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019TNtfRjLNhEz82G2ggeeNu
avrabe
marked this pull request as ready for review
September 10, 2026 07:28
avrabe
changed the title
v0.33.0 gave `[[tool]]` a `release` key so a hub — a repository that tags v0.7.2 and ships `with-device` at 0.2.2 — could state the tag to fetch apart from the version the payload answers to. The field parsed under deny_unknown_fields. It carried a doc comment naming the exact failure it prevents. The requirement was marked `verified`. Nothing read it. `plan_tool` took the fetch tag, the `%R` expansion, the per-release verification grouping and the recorded `source.release` all from `version`. So the pulseengine realm was edited to carry `with-device` again, and the 2026.09.3 deposit asked pulseengine/jess for a release tagged `0.2.2`, which does not exist. The payload stayed missing — the outcome this requirement exists to prevent, reached through the field added to prevent it. The evidence behind `verified` was real and proved the wrong thing: source markers on tests that exercised PARSING the field, never CONSUMING it. A capability is not shipped when a struct holds it; it is shipped when an artifact carries it. Sixth time this shape has appeared in this campaign. `PayloadPlan` now carries `release` beside `version`. `%R` and `%V` read different strings rather than deriving one from the other. `by_release` and `releases()` group by the tag, because the tag is what is fetched and what one cosign verification covers. `varve-producer plan` prints the tag whenever it differs from the version — that divergence was invisible while nothing read the field. `assets` takes `--release`, defaulting to `--version`. The guard is general, not specific to this field: `no_optional_manifest_field_is_inert` sets every optional `ManifestTool` field to a distinctive value and asserts each is observable in the resulting plan, so a field added without being consumed fails the build. Negative- controlled: with `release` made inert again it fails saying `release` is inert, and the hub test reproduces the production error exactly — left "0.2.2", right "v0.7.2". Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019TNtfRjLNhEz82G2ggeeNu
Correcting this release's own account of the defect. The commit before this one said "nothing read it", which is false and buries the more useful fact: varve ships TWO assemblers for one job, and `release` was taught to one. `varve layer-spec` — a live, documented subcommand that encodes a manifest into the environment the older shell assembler reads — implements `release` completely: the fetch tag, and the fifth positional field carrying the payload's own version. It even carries a comment about `with-device` from `pulseengine/jess` needing exactly this. `varve-producer`, the assembler realms actually run, never read it. So the capability went to the path being retired and not to the path in production, and nothing compared the two. The realm's manifest was correct. Both assemblers were individually tested. The deposit still failed. `both_assemblers_resolve_the_same_fetch_tag` holds them to the same answer for one manifest, checking the fetch tag and the payload version. Negative- controlled: with the planner inert it fails naming both sides — "layer-spec says v0.7.2, the planner says 0.2.2". While both ship, disagreement between them is a defect regardless of which is retired first. The divergence, not the missing field, is what let a correct manifest fail a deposit. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019TNtfRjLNhEz82G2ggeeNu
Asking "where else do the two assemblers disagree?" found a second divergence, and a worse one than the release tag. `varve layer-spec` correctly REFUSES an `sdk` payload — the encoding has no field for a layout, and the error says so and names the damage. It silently DROPPED `upstream-sums`, emitting an entry byte-identical to one that never declared it. That field is not a name, it is the mechanism that vouches for the release (REQ-UPSTREAMSUMS-001). Translating it away leaves an entry that assembles happily with less proof than the realm asked for: the shell assembler looks for a cosign bundle and an attestation, finds neither, and ingests the payload with no proof at all — while every other field survives the trip, so the entry looks perfectly ordinary and nothing reports a downgrade. The sdk case had already established the rule. This one predated it and was never brought in line. It now refuses the same way, naming the field, the consequence, and the command that reads it properly. This closes a live hazard rather than an outage: no workflow in varve or in pulseengine-layers runs `layer-spec`, and no realm manifest declares `upstream-sums` today. The next one to declare it would have been the first to find out. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019TNtfRjLNhEz82G2ggeeNu
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft — opened to get CI, not because it is ready. CI runs only on PRs and pushes to
main, so these commits have been verified locally only: fmt, 0 clippy, 17 suites,rivet validatePASS, andcargo mutantsrun per-file by hand on the files I touched. The fuzz targets, MSRV, the platform matrix and the full mutation shards have not seen any of it.Scope: 3 of 4
verifiedmeans connected in the graphWhat this release is about
Three gate-scope findings from v0.33.0, and the pattern behind them: each gate was real, worked, and had a boundary nothing checked.
varve-core. Every file is now gated or declared inmutation-scope.tomlwith one of three arguable reasons, and a file that is neither fails.not-yetis a legitimate reason, counted every run and capped at 22 with a cap that only ratchets down.varve-producerhad nine subcommands and zero topics while being the program other repos' CI runs. It now carries ten topics and its own coverage gate — and the check enumerates shipped binaries rather than naming the second one, so a third fails the build until it has one.varve-producer scanandnext-layerreplace it in Rust, tested against fixtures rather than a cron.What mutation testing caught, in this release, in my own new code
coverage_gapswas replaceable byvec![]— the only test asserted the real CLI has no gaps, which an empty list satisfies for everything.date -u -r, including all three leap rules.forge_envexists for exactly this; my handler passed&[]. On an enterprise instance every lookup fails — or worse, succeeds against a same-named repository on the public forge.The scope check earned its keep four times
It caught
mutationscope.rs,scan.rs,nextlayer.rs, andcli.rs+docs.rsas undeclared — each time before I remembered to gate them, each time a file its author had not thought about.Not done
REQ-TRACEGATE-001, and the note it carries: 38 requirements are graph orphans, and
validatereports its own central invariant as a warning among 212. Switching that to an error over an untriaged backlog would just teach people to write the exception, so the backlog is closed or excepted first.🤖 Generated with Claude Code
https://claude.ai/code/session_019TNtfRjLNhEz82G2ggeeNu