Repository navigation
Duplication check: keep one qualifying place per token run so the window cannot hide a real pair (#42) - #44
Conversation
…dow cannot hide a real pair (#42) Each run of min_tokens tokens keeps its 16 most recent places and, once it has left them, the most recent place that itself spans min_lines lines. That place is compared after the recent ones and is never evicted until a newer qualifying place replaces it, so 16 or more short copies between two real copies no longer drop the real pair. Two fixtures put 17 compact copies between two spread copies, across files and in one file; both fail on main and on a mutant that evicts the kept place. Signed-off-by: kikashy <35789537+kikashy@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review record: devkit PR #44 (issue #42), head fe2adb5 on base 1f02688The fix does what it claims and nothing else: the kept place is tried only after every recent place fails, so it can only add pairs main hid. Both new tests fail on main and on a mutant, and the stated limit is real but narrow.
Not hollow: hiding a pair takes 16 or more compact copies of the same 100+ tokens between the two real copies, which is rare in real code. Even then, any real copy with a stretch near the kit's median density (9 tokens/line) has a spanning run and is now reported. Only copies at the threshold that stay dense throughout (above about 10 tokens/line over every 100-token stretch) remain window-only. Checked in the diff:
What I ran (Python 3.12, memory cap and timeout on every run, scratch copies deleted after each run):
merge: yes |
|
🎉 Thank you for your contribution to Judgment Pack! If you’d like another task, choose a path that interests you: |
Closes #42.
What changed
lib/duplication.py: each run ofmin_tokenstokens still keeps its 16 most recent places and compares them nearest first. Besides them it now keeps the most recent place that itself spans at leastmin_lineslines: when that place leaves the 16 (and none of the 16 spansmin_lineslines) it is held apart, compared after the recent places, and never evicted until a newer place that spansmin_lineslines replaces it. Two such places always make a reported pair, so 16 or more short copies between two real copies no longer hide the real pair. Nearest-first comparison, the once-per-block examination, the same-file rule and the ignore blocks are unchanged.README: the sentence that states the 16-place bound now says what is kept beyond it.
Scope of the guarantee: it holds when a run of the earlier real copy spans
min_lineslines on its own. A copy longer thanmin_tokensin which no single run spansmin_lineslines is still covered by the 16-place window only.Fixture
test_a_copy_is_named_past_more_compact_places_than_the_recent_window: a spread copy of a 31-token function (7 lines), 17 files with the same tokens on 3 lines, and the spread copy again. Expected: exactlylib/a.js:2-8 and lib/c.js:2-8: 31 tokens...._in_one_file: the same in one file, each compact copy after a line holding one name of its own so no copy joins its neighbours.Both fail on main (exit 0, no pair) and pass on this branch.
Mutant
In a scratch copy, the kept place was evicted (the
kept[value] = earlier[0]assignment replaced bypass): both new tests fail, the other 41 pass.Cost
No synthetic-tree test in the suite; on the kit itself the check takes 0.13-0.15 s on main and 0.14-0.15 s on this branch, peak memory about 27.7 MB on both (three runs each).
Check summary
jpdev check --changed(lib/ and tests/ are unmapped, so the whole list ran): 8 passed, 0 failed, 0 skipped, 0 not run. diff-check, local-paths, agent-docs, bash-n, duplication (no duplicate block, 21 files), json, shellcheck, tests (430 passed, 0 failed).tests/duplication_test.py: 43 tests OK (41 existing + 2 new).🤖 Generated with Claude Code