From ROADMAP.md § M2. PRD: D19 (ADR-0021)
Take real text and markdown documents and apply known mutations programmatically, keeping the ground truth: block moves, renumberings (insert a clause, relabel everything below, update cross-references), inline edits, and splits and merges (labelled now so the 1.1 metric has data, not scored in 1.0). Output is a pair plus a label file that the metric (#143) reads. Seeded and deterministic, so a run is reproducible.
Sources: the sample pair, the existing tests/corpus/ documents, and neurotic_docx_bench's 763 pairs as a text source (AGPL, so evaluation use only, never bundled; keep a fetch script rather than the files). Publish the generator alongside the labels: ADR-0021's answer to the self-marking risk.
ADR-0021: build the corpus before tuning alignment, not after. This should land before thresholds in #131 are settled.
From ROADMAP.md § M2. PRD: D19 (ADR-0021)
Take real text and markdown documents and apply known mutations programmatically, keeping the ground truth: block moves, renumberings (insert a clause, relabel everything below, update cross-references), inline edits, and splits and merges (labelled now so the 1.1 metric has data, not scored in 1.0). Output is a pair plus a label file that the metric (#143) reads. Seeded and deterministic, so a run is reproducible.
Sources: the sample pair, the existing
tests/corpus/documents, andneurotic_docx_bench's 763 pairs as a text source (AGPL, so evaluation use only, never bundled; keep a fetch script rather than the files). Publish the generator alongside the labels: ADR-0021's answer to the self-marking risk.ADR-0021: build the corpus before tuning alignment, not after. This should land before thresholds in #131 are settled.