Skip to content

test(routing): add 302-case legacy parity baseline - #57

Merged
chubirka merged 1 commit into
mainfrom
test/work-routing-parity
Oct 14, 2026
Merged

chubirka merged 1 commit into
mainfrom
test/work-routing-parity

Conversation

@girish1430

@girish1430 girish1430 commented Oct 6, 2026 •

Copy link
Copy Markdown
Member

The work-router refactor needs a repeatable record of existing decisions. This PR imports the 302 SDK fixture cases into Traust and verifies them against the checked-out legacy Python worklist code, so subsequent extraction can expose behavior changes by case name.

This is the legacy-baseline portion of XWING-2216 and can be reviewed and merged independently of XWING-2215. It does not complete old/new router parity or change production routing policy.

Implementation

  • Preserve both SDK JSON fixtures byte for byte, including their original source provenance: 93 table, 20 risk-tier, 30 event-lane and 159 primary-selection cases.
  • Execute the actual legacy functions/constants and the event/table block in main(), using the original generators' adapter approach. Temporary JSONL/SQLite fixtures cover events and inventory bootstrap; network connections and subprocesses are blocked during these tests.
  • Verify fixture checksums/counts and compare all 302 cases against the current legacy code. A mutation check proves a changed churn threshold is detected by the saved decision expectation.
  • Document scope, provenance, and how to review drift. There is no whole-file source checksum gate, so unrelated edits to the monolith do not require updating the baseline record. The reviewed source revision remains historical provenance.

XWING-2216 progress and follow-up

  • Export and preserve all 302 cases.
  • Add offline legacy comparisons, named failures, and drift checks.
  • Add comparisons against the new router once XWING-2215 provides its implementation.
  • Agree the mapping to Route(lane, priority, reason), including rule IDs, helper cases, multiple event decisions and bootstrap; preserve coverage without inventing interface fields.
  • Verify the complete old/new comparison meets the five-second target.

Validation

  • python -m pytest -q tests/test_route_parity.py tests/test_build_rescan_worklist.py: 487 passed in 2.21s (306 baseline/guard tests and 181 existing worklist tests).
  • All 306 baseline/guard tests passed in 0.33s when replayed against the extracted legacy code on feat/per-repo-observation (feat(observe): expose Core repository observation through Traust CLI #58).
  • Ruff lint and formatting checks passed for both new Python files.
  • Skill alignment, skill security, content licensing, documentation consistency, staged estate-data and whitespace checks passed.
  • Both fixture exports compare byte for byte with the SDK originals.

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 6, 2026
@smith-xyz

Copy link
Copy Markdown
Contributor

Nice work on this, the approach of running the real legacy code against the 302 cases is solid.

One change before merge: the whole-file checksum on build_rescan_worklist.py (reviewed_legacy.sha256 in baseline.json, checked by test_legacy_source_matches_reviewed_baseline) will fail on every E2 PR. XWING-2213, 2214 and 2217 all edit that file, so the guard trips even when no decision changed.

The cases already catch real behaviour changes. Your test_guard_and_cases_detect_source_drift shows that: a changed churn threshold fails churn-lines-8000 without any help from the hash. So could you either:

  • drop the whole-file hash check, or
  • narrow it to just the definitions the adapter executes (e.g. hash ast.dump of the REFERENCE_NAMES nodes plus the event_rows→cc_urls slice of main()), so unrelated edits to the file don't trip it.

Either is fine with me; the second keeps a "you touched the decision code" signal if you want it.

@girish1430

girish1430 commented Oct 7, 2026 •

Copy link
Copy Markdown
Member Author

Addressed the whole-file checksum concern in commit 6a4ce77.

  • Removed the source checksum test and reviewed_legacy.sha256 from the baseline manifest. Unrelated edits to build_rescan_worklist.py no longer require repinning the baseline.
  • Kept both fixture checksums, the recorded source revision/provenance, and all 302 behavior cases unchanged.
  • Kept the mutation test: changing CHURN_FULL_LINES from 8000 to 9000 still disagrees with churn-lines-8000 through the behavior comparison itself.
  • Updated the adapter documentation, drift-review instructions, and PR description.

Validation: 487 routing/worklist tests passed; all 306 baseline/guard tests also passed when replayed against #58's extracted legacy code. Ruff and repository hooks passed.

@chubirka

Copy link
Copy Markdown
Member

/lgtm

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Oct 12, 2026
@chubirka

Copy link
Copy Markdown
Member

/approved

Import the 302 SDK fixture cases into Traust and verify them against
the checked-out legacy Python worklist code, so subsequent extraction
can expose behavior changes by case name.

- Preserve both SDK JSON fixtures byte for byte (93 table, 20 risk-tier,
  30 event-lane and 159 primary-selection cases)
- Execute legacy functions/constants using the original generators'
  adapter approach
- Verify fixture checksums/counts and compare all 302 cases against
  the current legacy code
- Mutation check proves a changed churn threshold is detected

XWING-2216

Co-authored-by: Cursor <cursoragent@cursor.com>
@girish1430
girish1430 force-pushed the test/work-routing-parity branch from 6a4ce77 to 5d401f9 Compare October 14, 2026 21:10
@openshift-ci openshift-ci Bot removed the lgtm Indicates that a PR is ready to be merged. label Oct 14, 2026
@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Oct 14, 2026
@chubirka

Copy link
Copy Markdown
Member

/lgtm

@openshift-ci

openshift-ci Bot commented Oct 14, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: chubirka, girish1430, smith-xyz

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:
  • OWNERS [chubirka,girish1430,smith-xyz]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@chubirka
chubirka merged commit 789f05e into main Oct 14, 2026
1 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants