Skip to content

test: add invariant baseline and reconciled command matrix - #201

Open
Yehonal wants to merge 11 commits into
mainfrom
pr/invariant-test-baseline
Open

Yehonal wants to merge 11 commits into
mainfrom
pr/invariant-test-baseline

Conversation

@Yehonal

@Yehonal Yehonal commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Requested by Joseph · project thread

Adds a synthetic invariant baseline without changing product behavior. Two defects are retained as it.fails: incomplete Git cache preservation (I2) and stale merge admission parity (I5). Each has an adjacent ordinary test that asserts the precise current defect, so an unrelated setup failure cannot satisfy the baseline. A subsequent fix must update both tests and remove the expected-failure marker.

An unreachable source during pin preflight warns by design; it is not classified as a defect here. The existing pin-preflight tests preserve that policy.

The effects helper compares bytes, modes, symlinks, manifest claims and optional Git index/refs with exact, documented exclusions. The command matrix loads the actual Commander tree and reconciles runner-owned Vitest IDs. The command-test map currently links only three commands. unattributedCommands means no declared link, not proof of absent test coverage. Unattributed tests remain listed rather than being assigned by imports.

The runner uses the default reporter with inherited stdout/stderr, alongside its JSON reports, so CI failures remain visible. Parser scratch files and the default result destination use the operating system temporary directory. CI uploads the complete matrix as an artifact; environment-specific results are not committed.

Validation: all nine focused local tests passed, one file at a time with one worker. CI on the current head is green: 1,232 tests passed with zero failures or skips, plus typecheck, build and both isolated smoke checks. The uploaded matrix confirms two expected failures, 58 command paths and 55 unattributed commands. Default reporter output is visible in the CI log. No product fixes, release, runtime or repository-policy changes. Merge is reserved for maintainer approval.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant