Repository navigation
🌱 test(checker): pin normalisation and scoring rules the symmetric stemmer hides - #412
Merged
Merged
Conversation
…emmer hides The nightly advisory Stryker run leaves 27 mutants alive in checker.ts. Because text and keyword are reduced by the same stemmer, most rules only become observable where the two sides take different paths to a stem (commit/committed, add/ad, use/us), or where normalisation differs between pattern and text (curly double quotes, whitespace runs, filler-word gaps, regex metacharacters in a keyword). Each new case takes one such pair and asserts the outcome, plus the tie-break for matchedText, per-keyword hit counting for filler-only keywords, sort order across excuses, and the exact instruction text generatePromptBlock emits around the table. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: quality <280983584+hivecommons-hive[bot]@users.noreply.github.com>
kubestellar-prow
Bot
added
the
dco-signoff: yes
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
kubestellar-prow
Bot
added
the
size/M
hivecommons-hive
Bot
added
author/hive
Contributor
Author
|
Thank you for your contribution! Your PR has been merged. We'd love to hear how your experience was: share feedback |
hivecommons-hive
Bot
removed
role/trusted
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Test Improvement
Adds a
describe('normalisation and scoring rules')block undercheckand twogeneratePromptBlockcases insrc/checker.test.ts.Why
The nightly advisory mutation run on
main(562f1c4, run 40821869628,restshard) leavessrc/checker.tsat 83.13% with 27 surviving mutants. The behaviour-bearing ones are the stemmer's doubled-consonant / silent-e guards (L33, L35),normalizeText's curly-double-quote and whitespace-run folding (L40), the filler-gap collapse inreduceText(L47), regex-metacharacter escaping incontainsKeyword(L55), filler-only hit counting (L85), thematchedTexttie-break (L86), the confidence sort (L116) and the fixed instruction text ingeneratePromptBlock(L138, L147–L161).They survive because text and keyword go through the same stemmer: a rule that maps both sides to the same wrong stem still matches. Each new case picks a pair where the two sides take different paths (
committed→committ→commitvs. keywordcommit;advs.add;usvs.use), or where only one side is normalised (ASCII"done"pattern vs. curly text,out \t\n of scope,fix the bug latervs. keywordfix bug later,e.g.vs.eg), and asserts the outcome exactly.Re-running Stryker on
src/checker.tsalone with this branch (running onlydist/checker.test.js): 93.12% (149 killed, 11 survived). Of the 11, five (isStopWord/STOP_WORDS, L10–16) are killed by the cli/learner suites in the full nightly run, and the remaining six are equivalent mutants: the^anchors on L22/L23 (the stemmer is only ever applied to a whole[a-z]+token), the.trim()innormalizeText(every caller re-trims or matches with lookarounds), the unused fields of thematched: falseresult (L92) and theoptions?.onEventguard (L118, the mutant'sTypeErroris swallowed by the innertry).No shipped code changes; test-only, so no
CHANGELOG.mdentry.Filed by quality agent (ACMM L4/L6 — full mode)
— hive: agent=quality backend=copilot model=claude-fable-5.1 copilot=1.0.88