Section 3a golden change tree (#144)

[houfu]

What and why

PRD § 3a calls the sample-pair change tree "the first test written and the last one allowed to fail". The first half landed earlier in this stack: eight named tests in tests/test_sample_pair_change_tree.py, one per row of tests/corpus/sample_pair/CHANGES.md, written as the specification redlines.comparison was built against. This PR is the second half.

tests/corpus/sample_pair/regenerate.py grows a second pairings table and a build_comparison/dump_comparison pair beside the existing build_tree/dump, so one command still regenerates the whole directory — now six files rather than four:

• tests/corpus/sample_pair/expected/change_tree.contract.json (313 KB)
• tests/corpus/sample_pair/expected/change_tree.markdown.json (341 KB)

Each is the whole JSON v2 document Comparison.to_dict() emits: both block trees, the ten change nodes, the alignment and the statistics. The four block-tree goldens are byte-identical after the run.

This branch carries a merge of origin/claude/m2-138-consumers (filters, statistics, and the schema's statistics/changeFilter definitions), on top of origin/claude/m2-143-metric. The merge was clean — no conflicts in pyproject.toml, README.md or anywhere else — and the diff is clean once that branch is merged in order. Without it the goldens would carry no statistics section, which is half of what makes them worth freezing.

The by-eye review: eight promises, ten nodes

Read against tests/corpus/sample_pair/CHANGES.md, node by node. The markdown twin is quoted; the contract twin is identical except promise 6.

• 1 — a definition whose text changed. modify at /section[1]/section[2]/list_item[4], 2.4 → 2.4, role: "definition", one inline insert src[133:133] → test[133:204] = "confidential, or that the receiving party ought reasonably to treat as ". span_types: [] — the defined_term span sits at the head of the definition, far from the edit, so a summary can say "a definition was widened" without claiming the defined term changed.
• 2 — a clause moved from section 7 to section 9, edited on the way. Two nodes, and their separation is the point. move /section[1]/section[7]/list_item[5] → /section[1]/section[9]/list_item[6], 7.5 → 9.6, matched_by: "move", confidence: 1.0, inline: [], source_text == test_text byte for byte. The edit rides in modify .../list_item[5]/paragraph[1] → .../list_item[6]/paragraph[1], one replace "three " → "five " at src[44:50], carrying both addresses — the node ADR-0033 cites as the reason every node carries both. Its matched_by is "fuzzy" (confidence 0.9333); the design refused to predict this one on paper and the golden records what the engine actually says.
• 3 — a renumbering caused by an inserted clause. One insert at /section[1]/section[3]/list_item[3], label 3.3, matched_by: "unmatched", confidence: 0.0, span_types: ["party"]. Then two renumber nodes, list_item[3] → list_item[4] (3.3 → 3.4) and list_item[4] → list_item[5] (3.4 → 3.5), both matched_by: "exact", confidence: 1.0, inline: []. Three nodes, not three inserts, and exact on both is what proves ADR-0032's pass order held: the surviving clauses were matched on their text before label could pair old 3.3 with the newly inserted 3.3.
• 4 — a cross-reference updated to follow the renumbering. modify at /section[1]/section[9]/list_item[2], 9.2 → 9.2, one replace src[164:168] "3.3." → "3.4.", and span_types: ["cross_reference"] — not ["cross_reference", "party"]. The block carries two party spans that the op does not overlap; ADR-0033's "touched" reading is what keeps the one signal PRD § 3a promises here from being buried.
• 5 — a deleted sub-clause. delete at /section[1]/section[5]/list_item[4]/list_item[3], source_label: "(c)", test_address: null, test_text: "", role: "sub_clause", source_text beginning "The Client pays the undisputed part of the invoice on the due date.". (a) and (b) keep their addresses and produce nothing: exactly one node under /section[1]/section[5]/list_item[4].
• 6 — an inserted table row (the one place the twins diverge). markdown: one insert at /section[3]/list_item[3]/table[1]/row[5], block_kind: "row", role: "schedule", with its three cell children emitting nothing — topmost-wins by construction, which is Table alignment for markdown pipe tables: row insert/delete, cell-level inline diff #134's stated bar met without a table special case. contract: insert at /section[3]/list_item[3]/paragraph[5], block_kind: "paragraph", test_text: "The training day is due by Week 9 and is owned by the Client.". The two goldens differ in this node and in nothing else that CHANGES.md did not predict.
• 7 — a whitespace-only change that is no change. Visible by absence. No node anywhere under /section[1]/section[11], and the statistics row for that section reads total: 0. The alignment shows why: ("/section[1]/section[11]/list_item[5]", same) is matched_by: "exact", confidence: 1.0. Both readers rejoined the hard wrap, so the change had disappeared before alignment ever saw it.
• 8 — an edit inside a repetitive schedule. modify at /section[4]/list_item[3], 3 → 3, role: "schedule", matched_by: "label", three replace ops: "four " → "two " at 60, "two " → "one " at 109, "Days." → "Day." at 122. Three rather than two because _merge_ops_split_by_punctuation merges only across punctuation-only equal runs and "Business " sits between them. Nothing else in /section[4] is reported, though seven near-identical siblings surround it — the ADR-0010 pathology, not reproduced.

Also checked, and also correct by absence: the "Go-live sign-off" row slides row[5] → row[6] and emits nothing, while the alignment carries it as matched_by: "exact", moved: false. "An address shift alone is never a change" (ADR-0033), demonstrated rather than asserted.

Two arithmetic checks on the rest of the payload. Totality: markdown 118 pairs + 5 inserted = 123 test blocks, 118 + 1 deleted = 119 source blocks; contract 101 + 2 = 103 and 101 + 1 = 102. Derived counts: chars_added 92 = 71 + 4 + 5 + 12 and chars_deleted 24 = 4 + 6 + 14, both summing the inline ops above exactly, so the filter's min_chars and the statistics agree by construction as D-4's resolution requires.

New tests

Six whole-tree tests per twin in tests/test_sample_pair_change_tree.py, beneath the eight named ones:

1. exactly ten change nodes, as an ordered (kind, source, test) list — the readable half of the golden, so an eleventh node names itself instead of printing 300 KB.
2. the whole comparison matches the golden, key section by key section (config, changes, statistics, alignment, source, test) and then on the stored text, which is what catches a golden reformatted rather than regenerated.
3. source/test are still byte-for-byte the M1 goldens. ADR-0033 states this as a conformance requirement; nothing checked it until now.
4. round trip through Comparison.from_dict.
5. determinism within one process (tests/test_determinism.py covers the hash-seed axis).
6. the two similarity backends agree on every node, pair, and statistic, differing only in the two fields that record which backend ran.

tests/test_json_v2.py gains a test that validates the two goldens as stored against comparison-v2.json — the freshly-built-output tests there cannot catch a hand-edited golden.

All of them compose the comparison through the public compare() rather than importing the regeneration script, so a drift between engine and script is a failure and not a silently regenerated golden — the discipline tests/test_sample_pair.py already applies to the block trees.

Decisions taken where the research document was silent

• The goldens are written with include_alignment=True. Comparison.from_dict refuses a payload without an alignment, so a golden without one could not be read back, and a golden that cannot round-trip is a weaker freeze than one that can. It also pins the correspondence set — the thing the benchmark scores and the thing the change tree structurally cannot express — which is where promise 7's proof actually lives. Cost is about 20 KB per file against 300 KB of block trees.
• The goldens are generated under an explicitly named difflib backend, not the "auto" default. auto resolves to rapidfuzz where the [fuzzy] extra is installed and difflib where it is not, and the resolved name goes on the wire in config.similarity and alignment.backend — so a golden generated under auto would fail on whichever CI leg was not the one that wrote it (3.10 installs no extras, 3.11+ installs --all-extras). difflib is the floor PRD § 12 runs the site on and the backend benchmark/REPORT.md quotes. It costs no coverage, and test 6 above is what says so rather than leaving it as an assumption. Every threshold is left at the shipped default.
• CHANGES.md was updated, because its closing paragraph said the change tree "is M2's golden, not this directory's" and that is now false. It records the six files, the eight-promises-to-ten-nodes arithmetic, and where the per-promise tests live.

Reinterpretation of the issue's wording

The issue says the golden shows "one of each thing the engine detects", listing eight items. The change tree has ten nodes, and that is not a discrepancy: change 3 is an insert plus the two renumbers it causes, and change 2 is a move plus the edit that rode along in its body — in both cases the extra node is the point of the row, not an artefact. Changes 7 and 8's promises are partly discharged by absence (no node at all for the whitespace rewrap; no spurious nodes among the seven repetitive siblings). Stated here so nobody reads "eight changes" against "ten nodes" and concludes the engine over-reported.

What was deliberately left out

• No strict-xfail mark was removed, because none of them passed. strict=True makes the suite say so: the run reports 7 xfailed, 0 xpassed. Four are the PRD § 10 gates in tests/test_benchmark_gates.py — synthetic F1 0.9412 (< 0.95), synthetic move recall 0.7391 (< 0.90), hand F1 not assessable while its 644 rows are status: proposed, and 3 unreviewed engine moves on the hand set. Three are the M1-scope reader hard cases in tests/test_text_reader.py, out of scope here. Removing any of them would have been weakening a test, not flipping a gate.
• ROADMAP.md is untouched: the maintainer records milestone completion.
• No benchmark/REPORT.md or results/latest.json diff. uv run python -m benchmark.run was re-run after the consumers merge and rewrote both files identically; --check reports up to date. Nothing to commit rather than an empty commit.
• No derived old→new label index and no by_block array — ADR-0033 declines both for 1.0; the golden would otherwise have been the first thing to freeze them.

M2 exit-criteria status

Criterion	Status
The section 3a golden change tree passes	Yes. Both twins, whole-tree and per-promise: tests/test_sample_pair_change_tree.py is 27 passed, 0 xfail.
Move gate met	No. Synthetic move recall 0.7391 against ADR-0009's ≥ 0.90, with precision 1.0000, 0 ruled wrong and 0 unreviewed — the engine is missing moves, not inventing them (17 reported, 17 hits, 23 labelled). The hand-set half of the gate is awaiting the maintainer's labelling and verdict passes: 644 correspondence rows are still status: proposed (engine-seeded drafts, so the 1.0000 F1 measures self-agreement) and 3 engine moves have no move_verdicts entry, which fails closed by design. Its xfail(strict=True) tests stay on and will fail the day they are earned.
Benchmark report exists	Yes. benchmark/REPORT.md and benchmark/results/latest.json, both tiers, both backends, current as of this branch.
JSON schema frozen	Yes. redlines/schemas/comparison-v2.json, draft-07, schema_version: "2.0" (#137), and as of this PR the two committed goldens validate against it as stored.

Test evidence

uv run pytest        1633 passed, 2 skipped, 7 xfailed
uv run mypy redlines tests benchmark    Success: no issues found in 86 source files
uv run python -m benchmark.run --check  up to date

The 14 new tests are 12 in tests/test_sample_pair_change_tree.py (six per twin) and 2 in tests/test_json_v2.py. The 2 skips are the two extras-absent paths, skipped because uv sync --all-extras --dev installs both (nupunkt is installed, the [fuzzy] extra is installed).

Closes #144

🤖 Generated with Claude Code