Repository navigation
Measure source reads and graph outcomes along namesake adjudication paths - #1377
WaylandYang merged 5 commits into
Conversation
…nges the graph Signed-off-by: Virtue039 <virtue039@qq.com>
… behavior Signed-off-by: Virtue039 <virtue039@qq.com>
WaylandYang
left a comment
There was a problem hiding this comment.
Thanks — this is a clean, genuinely test-only measurement, and CI cost is negligible. One change before merging: please pin the outcome of each of the 48 rows the way namesake_scripted_cases.rs does. Name today's 20 wrong merges and 4 unmerged true pairs in a known-gold-error list (e.g. Batch/ordinary/forward/distinct), make listed rows required to show the error and all others required to match gold. That way a policy fix must update the list and a regression can't pass silently. Please also switch the new comments to Chinese per repo convention. #1425 doesn't overlap textually, but whichever lands second should rerun this.
Signed-off-by: Virtue039 <virtue039@qq.com>
Signed-off-by: Virtue039 <virtue039@qq.com>
WaylandYang
left a comment
There was a problem hiding this comment.
Thanks — the 24 gold errors are now pinned by name and the comments are in Chinese. #1425 has landed, so CI on the updated branch is the rerun on the integrated tree. Merging.
An exact-name
samecan reach an automatic merge without reading either source. This test-only measurement for #1193 records whether investigation runs, what each side actually receives, and the resulting graph/review state.Changes requested in review
Addresses review 5658119082:
Batch/ordinary/forward/distinct. Explicitly list today's 20 wrong merges and four unmerged true pairs inKNOWN_GOLD_ERRORS. Each listed row must still disagree with source gold; every other row must match it. Unexpected passes, unlisted errors, duplicate/missing rows and unused list names fail the test. There is no total-error budget.KNOWN_FAILURESlist intact. The new named list applies only to the new 48-row measurement. A future policy fix removes its corresponding names with the fix.The 48 cases cover ordinary adjudication/governance, both creation orders, source-defined same/distinct pairs, batch
same(0.8)andsame(0.9), a seeded ordinary verdict cache, facts/quotes on both sides and a finaldifferent(0.5). Governance does not read the ordinary cache. The assertions also check A/B ownership in real HTTP inputs/tool results, verbatim source delivery, graph/review agreement and no repeated work on a terminal pair.Scripted conclusions deliberately disagree with some gold labels. Green CI means exactly the named baseline errors remain and controls match gold; it does not mean these identity errors are fixed or estimate provider accuracy. No production gate, prompt, API, schema, model request or lifecycle state is added by this PR.
Calibration and validation
A temporary mock-reply calibration made
Batch/ordinary/forward/distinctcorrect and made the true-pair controlBatch/ordinary/forward/sameincorrect. All 48 ran and the gold-error total stayed 24, yet the test failed on both the unexpected pass and unlisted error. The temporary reply change was restored before final verification; source inputs, gold labels, lists and production code were unchanged by the calibration.At signed commit
f652023197b051fe3a27406da7ae0fce26eebd35, based on dev1142792979b61272d92cf322cbfd2f86615f6cb3(2026-10-27), on macOS arm64:-D warnings), workspace build and frozen frontend install/build pass.CI 40486272486 passes all six checks at signed commit
f652023197b051fe3a27406da7ae0fce26eebd35. The PR remains ready for the maintainer's re-review; the previous changes-requested review has not been dismissed.Landing order
#1425 was still open/unmerged when this revision was validated and submitted. Its implementation is not included here. As requested in review, whichever PR lands second must rerun this measurement on the integrated tree.
#1251 is already merged. The production evidence standard remains open under #1193; this PR establishes the required named baseline guard.
Refs #1193, #1136, #1202, #1220.
Integration preflight (2026-10-28)
A local merge of this head
f652023197b051fe3a27406da7ae0fce26eebd35with #1425 head2916e332150bc5baa1435c0a0fd14721bf85f899merges without conflicts. The tested local tree is80facb8ed2df69589d8dacf10b16a2e746eb7fb5(local preflight commita0dc25bd51fbdfb9a455ff10067325124bc91b2e, not pushed).-D warnings), workspace build, frozen frontend install, 338 web tests across 49 files, and frontend build pass.This pre-merge check does not replace the landing requirement: whichever PR lands second must rerun the measurement on the actual integrated upstream commit. Neither PR was merged by this preflight; the previous changes-requested review remains for the maintainer to re-review.