Skip to content

[quality] Gate proposal: shell-checker arms that survive mutation (mutation harness for scripts/*.sh) #296

Closed as not planned

Description

@hivecommons-hive

Finding

scripts/*.sh (check-links.sh, check-redirects.sh, make-redirects.sh) are the only gates in this repository without a "branch runs but is never asserted" gate. The JS gates got one in #254 (branch-coverage floor, scripts/coverage-floor.test.mjs); the shell checkers are outside it, and the only thing that exercises them in CI is their own self-test (links.yml test job).

This is now the third time a freshly-landed shell-checker change shipped with arms that no self-test assertion distinguishes from a broken one:

  1. test: pin query-only self links in the check-links self-test #248 — first instance (check-links arm unasserted).

  2. test: pin 410, soft-404 variants, fonts skip and html_escape in the shell self-tests #257 — 4 arms survived hand-written mutants at f930ed0 (check-links 410 handling, two SOFT_404_RE alternatives, fonts preconnect skip; make-redirects html_escape of < > "). Self-tests stayed 106/28 green under every mutant.

  3. 🐛 fix(check-links): match anchor ids on a real attribute boundary #295 @0dada63 — has_id landed with three arms the new self-test case does not pin. Each mutant below keeps check-links.test.sh at 119/119:

    • escape only . instead of every ERE metacharacter → #a+b is satisfied by id="aab", #q(1) by id="q1" (false pass);
    • drop the ([^>]*[[:space:]])? group → <a class="x" href="#top" name="with-attrs"> is reported missing (false failure on a legitimate named anchor);
    • use the unescaped id in the name= arm → #n.m is satisfied by <a name="nzm">.

    A PR pinning these three arms is open separately; it does not change the fact that nothing stops the fourth instance.

Per the quality lane's three-of-a-class rule I am filing the missing gate instead of a third per-site finding.

Recommendation

Add a mutation gate for the shell checkers, mirroring how coverage-floor.test.mjs already gates the JS helpers in-repo with no new dependency.

Decision taken by hive: propose option A (declared-mutant harness) as the simplest additive, dependency-free option consistent with the repository's existing in-repo gate style; option B is listed in case the maintainers prefer true coverage over curated mutants.

Option A — declared-mutant harness (recommended)

  • scripts/shell-mutants.test.sh: a table of (script, sed-expression, description) rows, one per arm the self-tests are expected to pin. For each row it copies the script to a temp dir, applies the mutant, runs that script's self-test, and fails if the self-test still passes (or if the sed expression did not change the file — a stale mutant is itself a failure, so mutants cannot silently rot when the arm is refactored).
  • Seed rows: the 4 arms from test: pin 410, soft-404 variants, fonts skip and html_escape in the shell self-tests #257 and the 3 from 🐛 fix(check-links): match anchor ids on a real attribute boundary #295 (all currently killed, so the gate starts green).
  • Wire it into links.yml test job next to Self-test check-links / Self-test check-redirects; add it to ci-wiring.test.mjs so the step cannot be dropped.
  • Contributor rule (in the PR template "Checks run locally" list): a change to scripts/*.sh that adds a branch adds a mutant row for it.

Option B — line/branch coverage floor for bash via kcov

  • kcov --include-path=scripts scripts/check-links.test.sh style run in the test job, with a per-file ratcheted floor like coverage-floor.test.mjs. True coverage rather than curated mutants, but adds a C++ tool install to CI (apt kcov, ~15s) and line coverage would not have caught 🐛 fix(check-links): match anchor ids on a real attribute boundary #295 (every line ran; the regex alternation inside one line is what was unasserted). Mentioned for completeness; A is the better fit for regex-heavy checkers.

Ask: accept or decline option A. If accepted, the quality lane will open the PR with the harness, the seven seed mutants, the links.yml step and the ci-wiring assertion. If declined, the lane resumes filing per-site findings in this class.

Risk: governance

Priority

  • Impact: medium (false-pass anchors reach the live site unnoticed; false-fail anchors block unrelated PRs)
  • Effort: low (option A) / medium (option B)

Filed by quality agent (ACMM L4/L6 — full mode)


🐝 Hive Agent: quality | Instance: hosted-available-oke-11-placeholder-r05x | SHA: 0dada63

— hive: agent=quality backend=copilot model=claude-fable-5.1 copilot=1.0.88

Activity

  1. added
    testingCreated by Hive for agent-filed issue provenance
    agent/qualityCreated by Hive for agent-filed issue provenance
    stage/045-needs-decisionIssue parked on a person: needs-human or a maintainer decision is owed before the hive picks it up
    on Oct 29, 2026
  2. hivecommons-hive commented on Oct 29, 2026

    @hivecommons-hive
    ContributorAuthor

    What to reply

    • /hive approve: go with the recommendation (option A)
    • /hive decision B: go with option B
    • /hive help: show these again

    If what this issue waited on has been done (a merged fix, or an action taken outside this repository), a maintainer can reply /fixed to close it as completed; Hive clears its park labels.

    Only a maintainer with write access can un-park; everything else is ignored. Full command reference: https://github.com/hivecommons/hive/blob/v5/src/docs/maintainer-commands.md.

  3. hivecommons-hive commented on Oct 29, 2026

    @hivecommons-hive
    ContributorAuthor

    A merged PR referenced this issue without closing it. Carrying the remainder back here so the issue timeline shows what is still open.

    Remainder from the PR body:

    (gate proposal for a shell mutation harness stays open for the maintainers' decision; this PR only pins the three arms that motivated it).

    This comment is edited in place by the task-list sweep on every cycle; it is not duplicated.

  4. hivecommons-hive commented on Oct 29, 2026

    @hivecommons-hive
    ContributorAuthor

    Closed as won't fix by @clubanderson from the Hive dashboard.

  5. removed
    stage/045-needs-decisionIssue parked on a person: needs-human or a maintainer decision is owed before the hive picks it up
    on Oct 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent/qualityCreated by Hive for agent-filed issue provenancetestingCreated by Hive for agent-filed issue provenance

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions