Repository navigation
[quality] Gate proposal: shell-checker arms that survive mutation (mutation harness for scripts/*.sh) #296
Description
Finding
scripts/*.sh (check-links.sh, check-redirects.sh, make-redirects.sh) are the only gates in this repository without a "branch runs but is never asserted" gate. The JS gates got one in #254 (branch-coverage floor, scripts/coverage-floor.test.mjs); the shell checkers are outside it, and the only thing that exercises them in CI is their own self-test (links.yml test job).
This is now the third time a freshly-landed shell-checker change shipped with arms that no self-test assertion distinguishes from a broken one:
-
test: pin query-only self links in the check-links self-test #248 — first instance (check-links arm unasserted).
-
test: pin 410, soft-404 variants, fonts skip and html_escape in the shell self-tests #257 — 4 arms survived hand-written mutants at f930ed0 (check-links 410 handling, two
SOFT_404_REalternatives, fonts preconnect skip; make-redirectshtml_escapeof< > "). Self-tests stayed 106/28 green under every mutant. -
🐛 fix(check-links): match anchor ids on a real attribute boundary #295 @0dada63 —
has_idlanded with three arms the new self-test case does not pin. Each mutant below keepscheck-links.test.shat 119/119:- escape only
.instead of every ERE metacharacter →#a+bis satisfied byid="aab",#q(1)byid="q1"(false pass); - drop the
([^>]*[[:space:]])?group →<a class="x" href="#top" name="with-attrs">is reported missing (false failure on a legitimate named anchor); - use the unescaped id in the
name=arm →#n.mis satisfied by<a name="nzm">.
A PR pinning these three arms is open separately; it does not change the fact that nothing stops the fourth instance.
- escape only
Per the quality lane's three-of-a-class rule I am filing the missing gate instead of a third per-site finding.
Recommendation
Add a mutation gate for the shell checkers, mirroring how coverage-floor.test.mjs already gates the JS helpers in-repo with no new dependency.
Decision taken by hive: propose option A (declared-mutant harness) as the simplest additive, dependency-free option consistent with the repository's existing in-repo gate style; option B is listed in case the maintainers prefer true coverage over curated mutants.
Option A — declared-mutant harness (recommended)
scripts/shell-mutants.test.sh: a table of(script, sed-expression, description)rows, one per arm the self-tests are expected to pin. For each row it copies the script to a temp dir, applies the mutant, runs that script's self-test, and fails if the self-test still passes (or if the sed expression did not change the file — a stale mutant is itself a failure, so mutants cannot silently rot when the arm is refactored).- Seed rows: the 4 arms from test: pin 410, soft-404 variants, fonts skip and html_escape in the shell self-tests #257 and the 3 from 🐛 fix(check-links): match anchor ids on a real attribute boundary #295 (all currently killed, so the gate starts green).
- Wire it into
links.ymltestjob next toSelf-test check-links/Self-test check-redirects; add it toci-wiring.test.mjsso the step cannot be dropped. - Contributor rule (in the PR template "Checks run locally" list): a change to
scripts/*.shthat adds a branch adds a mutant row for it.
Option B — line/branch coverage floor for bash via kcov
kcov --include-path=scripts scripts/check-links.test.shstyle run in thetestjob, with a per-file ratcheted floor likecoverage-floor.test.mjs. True coverage rather than curated mutants, but adds a C++ tool install to CI (aptkcov, ~15s) and line coverage would not have caught 🐛 fix(check-links): match anchor ids on a real attribute boundary #295 (every line ran; the regex alternation inside one line is what was unasserted). Mentioned for completeness; A is the better fit for regex-heavy checkers.
Ask: accept or decline option A. If accepted, the quality lane will open the PR with the harness, the seven seed mutants, the links.yml step and the ci-wiring assertion. If declined, the lane resumes filing per-site findings in this class.
Risk: governance
Priority
- Impact: medium (false-pass anchors reach the live site unnoticed; false-fail anchors block unrelated PRs)
- Effort: low (option A) / medium (option B)
Filed by quality agent (ACMM L4/L6 — full mode)
🐝 Hive Agent: quality | Instance: hosted-available-oke-11-placeholder-r05x | SHA: 0dada63
— hive: agent=quality backend=copilot model=claude-fable-5.1 copilot=1.0.88
Activity
hivecommons-hive commentedon Oct 29, 2026
What to reply
/hive approve: go with the recommendation (option A)/hive decision B: go with option B/hive help: show these againIf what this issue waited on has been done (a merged fix, or an action taken outside this repository), a maintainer can reply
/fixedto close it as completed; Hive clears its park labels.Only a maintainer with write access can un-park; everything else is ignored. Full command reference: https://github.com/hivecommons/hive/blob/v5/src/docs/maintainer-commands.md.
hivecommons-hive commentedon Oct 29, 2026
A merged PR referenced this issue without closing it. Carrying the remainder back here so the issue timeline shows what is still open.
Remainder from the PR body:
(gate proposal for a shell mutation harness stays open for the maintainers' decision; this PR only pins the three arms that motivated it).
This comment is edited in place by the task-list sweep on every cycle; it is not duplicated.
hivecommons-hive commentedon Oct 29, 2026
Closed as won't fix by @clubanderson from the Hive dashboard.