diff --git a/.agents/skills/test-oracle-reviewer/SKILL.md b/.agents/skills/test-oracle-reviewer/SKILL.md index 8aea80f085..f6db475f99 100644 --- a/.agents/skills/test-oracle-reviewer/SKILL.md +++ b/.agents/skills/test-oracle-reviewer/SKILL.md @@ -8,8 +8,8 @@ description: Judge whether a test's expected value is justified independently of A test whose expected value came from the code under test detects change, not bugs. For each test, ask whether the expected value can be justified without running the code. -Independent sources: a spec example, an external capture, a hand-derived value, an input the test chose itself and asserts was honored, a named law such as round-trip, a property a document states, such as `encode` writing exactly `size()` bytes, or a reference model. -Self-derived sources: a snapshot of current output, a value computed through the code or a shared helper, a production constant or message copied into the test, a mock that returns what is then asserted, `is_ok()` or `!is_empty()` on an input that no requirement says must succeed, or a restatement of what a derive or the type system guarantees. +Independent sources: a spec example, an external capture, a hand-derived value, an input the test chose itself and asserts was honored, a named law such as round-trip, a property an intent, contract, or spec states, such as `encode` writing exactly `size()` bytes, or a reference model. +Self-derived sources: a snapshot of current output, a value computed through the code or a shared helper, a production constant or message copied into the test, documentation in or generated from the code under test, such as its rustdoc, a trait's included, or an OpenAPI file generated from its annotations, a mock that returns what is then asserted, `is_ok()` or `!is_empty()` on an input that no requirement says must succeed, or a restatement of what a derive or the type system guarantees. Verify instead of guessing. Check comments and `git log -p` for the origin, then check the value against the governing spec section: decode a byte vector field by field and show the breakdown, or cite the specification section that defines the behavior. @@ -30,7 +30,7 @@ Strong kinds name an independent source, and a test, case, or assertion marked w - `contract §
`: a `CONTRACT*.md`. - `intent §
`: a statement in an intent file. - `regression `: a bug report, as a Jira key such as `DGW-123` or a GitHub issue such as `#123` or `owner/repo#123`, or a fuzz crash file under `fuzz/`. -- `reference `: what another implementation or a reference model does, recorded or compared live. +- `reference `: what another implementation, a capture, or a reference model does, recorded or compared live; never the code under test or its own documentation, because documentation written with the code describes what it does rather than deciding what it must do. - `property `: a law of the code under test named by one of `round-trip`, `idempotent`, `involution`, `commutative`, `associative`, or `monotonic`. Write every path from the repository root, with forward slashes and exact case, such as `crates/gateway-db/INTENT.md`, because markers sit at every depth and a root-relative path reads and resolves the same from each. @@ -56,6 +56,7 @@ When it checks a property or intention no document states, ask whether and where Mark a strong source whenever one decides the value, so a test found through a mutant but decided by a spec is `spec`, because whether a test may go depends on whether a source still requires it, not on how it was found. An unmarked test starts as unknown: verify it as above, and turn a citation in another form into the marker it supports. A marker is a claim: verify it like any other origin. +A marker citing the code under test's own documentation is self-derived whatever its kind: re-cite the test to the intent, contract, or spec sentence that decides its value, keep it as `mutant` when `test-pruner` finds it alone catches a fault, or delete it. ## Review diff --git a/.agents/skills/test-pruner/SKILL.md b/.agents/skills/test-pruner/SKILL.md index 8502dec2bf..6f0febb96a 100644 --- a/.agents/skills/test-pruner/SKILL.md +++ b/.agents/skills/test-pruner/SKILL.md @@ -23,6 +23,7 @@ A reviewer's request for more mutation evidence falls under the same budget. ## Measure Invoke `test-oracle-reviewer` on every test in scope and record independent, self-derived, or unknown with evidence. +Record a test whose marker cites the code under test's own documentation, such as its rustdoc, as self-derived, so Decide flags it for removal unless an independent source decides its value or it alone catches a fault. The default mutant source is domain mutants written by a fresh-context sub-agent, because a few calibrated domain faults cost minutes where a generic run over the same code costs hours. Give it the applicable intent and contract files, the code under test, and `references/domain-mutants.md`, but not the tests, so the faults come from what the code must do rather than from what the tests already check. Ask for one patch per fault, each a single plausible mistake, weighted toward critical paths such as validation, cryptography, parsing, and authorization decisions, and as many as the estimate fits in the remaining budget.