feat(asvs): the absence gate gains a --prove-absences mutation mode (BACKLOG #1006) - #249
Open
wshallwshall wants to merge 2 commits into
Open
feat(asvs): the absence gate gains a --prove-absences mutation mode (BACKLOG #1006)#249wshallwshall wants to merge 2 commits into
wshallwshall wants to merge 2 commits into
Conversation
…atches (BACKLOG #1006) `check_absences` admits an ASVS absence claim on `re.search(a.pattern, a.mutation)` -- one TOML field matched against another. That proves the mutation is well-formed; it never proves the mutation BITES. A reintroduction raised into a swallowing handler, written to a field nobody reads, or behind a flag nobody branches on satisfies every failure mode `check_absences` has and changes nothing observable. A green gate that is not evidence. Add an opt-in `--prove-absences` mode (`scripts/asvs/scorecard.py`) that executes the claim rather than grepping it: - Two optional `Absence` fields, `mutation_path` and `observable` (a pytest node id). When both are set the mode copies the tree to a scratch dir, runs the observable (baseline must be green), appends the mutation, and requires the observable to go RED -- and to fail as a test failure (exit 1). It fails closed on every other code: an already-red baseline, an uncollectable node, or a mutation that only breaks import is a PROVE-ERROR, never a proof. A claim that reddens nothing is UNPROVEN and fails the mode. - A coarse same-file static backstop screens claims carrying `mutation_path` but no `observable`: a `raise` landing in a file whose every handler swallows. It is a screen, not a proof (it cannot see a swallow in a caller), documented as such. - The whole pass runs in a TemporaryDirectory scratch copy, so it never mutates the tracked tree and never trips the committed-tree scan on itself. Both fields default empty and load without being refused: the vault's ~81 existing absence claims carry neither and must stay loadable (ADR 0156 §7). Absent means "not yet proven by execution", surfaced by the mode, never "proven vacuous". Review hardening carried in this change (the mode's own helpers): - `_scratch_ignore` refuses `.env*`, `*.db` (+ WAL sidecars) and `docs/security` when copying the tree. The vault runs this module against the REAL tree (ADR 0156 §7); a scratch copy carrying those would spill secrets / the local store / vault posture data into a world-default temp dir, which CLAUDE.md §9 forbids. The public-repo path never sees them; this is defence for the eventual vault run. - `_is_within_tree` refuses a `mutation_path` that is absolute or contains `..` before anything is applied, so an authored path cannot escape the scratch copy. Tests (tests/test_asvs_scorecard.py): eight fixture tests drive `prove_absences` directly (proved / UNPROVEN / already-red baseline / collection-error / two static backstop arms including a re-raise reach control / root-untouched / load round-trip), plus three that drive the CLI contract CI depends on -- `main([..., "--prove-absences"])` exit 0 on a biting fixture and 1 on a non-biting one, and `main([...])` without `--corpus` exit 2 -- plus the secrets-exclusion and path-traversal guards. Every new test was falsified (broken on purpose, watched red, restored). MessageFoundry is a not-deployed beta: the mode is opt-in, the default `verify` path is byte-unchanged, and no authored claim carries an `observable` yet, so nothing new is blocked by this alone today. Wiring the mode over the vault claims and backfilling their observables is the owner's follow-up.
…e behaviour (BACKLOG #1006) Flip the #1006 banner from filed to shipped. It is written as a capability claim, not a closure claim: the `--prove-absences` mode CAN catch a well-formed-but-vacuous reintroduction once a claim carries an `observable`, but the default `verify` path is byte-unchanged and no authored claim carries one yet, so nothing new is blocked by this alone today -- the honest present-tense state for a not-deployed beta. Banner lines of #1006 ONLY. The ranked table, the four census distribution lines, and every other item's banner are untouched. The status census was NOT recomputed.
wshallwshall
enabled auto-merge (squash)
August 6, 2026 14:16
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes BACKLOG #1006. scripts/asvs/scorecard.py. Public-repo only -- no vault content.
The ASVS absence gate asserted that certain items are absent, but nothing proved the gate could
DETECT a violation -- a green result and a blind result were indistinguishable.
--prove-absencesadds a mutation mode that introduces the absence it claims to catch and requires the gate to fail.
That is the same standard applied elsewhere tonight: a control never shown to fail is not evidence.
Verified in-lane: plan -> build -> 3-lens adversarial verify -> remediate. Banner gate OK.