You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Layer V of spec 066 (§12.1, §13 row V). Test-only, no product change. It adds the harness that later layers use to prove they change behaviour only on purpose:
A seeded matrix generator. It covers all 378 Transform × Transition × Feature triples, with pairwise coverage of wrapper, flow direction, reduced motion, user code and sub-variant. The control group keeps every other dimension at its default.
A WinUI recorder that runs each case through mount, change, remove/add feature, remove modifier, unmount, rent plain and rent transformed. It records threw, pose, opacity (the settled opacity, so a lost update reads 1.00 instead of 0.80), active, motion, curve, running, residue, hit/uia/ttv, channel, pool and diag. Facade Translation drag cases also record screen, the PrintWindow pose (§12.2, Q1f). Motion is controller-driven, layout Offset included, so no verdict depends on wall-clock timing. A stale or drifting capture, a failed screen capture or a compositor that stops rendering makes a case inconclusive, and it reruns, instead of recording a wrong value. Under a facade Translation drag, curve is recorded but not compared, because the drag migrates at a varying moment.
Positive and negative controls for every oracle, run before anything is recorded or compared. A session with no compositor frames (locked or disconnected) fails them, so nothing gets recorded there.
A pool soak, platform probes (§5.2, §5.3, §6.3) and the behaviour ledger. The ledger was recorded on a hosted runner with main's behaviour. Record mode writes the ledger only when every check passes, and accepts a change only once two observations agree.
A Reactor.Tests gate that diffs the PR's ledger against the base commit's. Every change must be declared in transforms-ledger.expected.json with its §9 row or issue. The failure message includes a ready-to-paste entry. Expectations inherited unchanged from the base are ignored, so a merged layer's entries never fail an unrelated PR. The control group must stay byte-identical.
A Transforms matrix workflow that runs the full matrix on PRs labelled transforms-matrix and nightly. Label a PR transforms-matrix-record to record a new ledger on a hosted runner and download it as an artifact.
A TESTING.md section and §12.1 F in the spec describing what layer V measured. That includes a finding for §12.2: Q1f doesn't reproduce on hosted runners, so the screen oracle is gated on a write-and-reset control and Q1f is logged.
Linked issue / spec
Part of #1630. Implements docs/specs/066-transforms-on-composition-visual.md §12.1 (layer V). Filed while building it: #1651, #1652, #1660 and #1665.
Test plan
dotnet test tests/Reactor.Tests --filter-class "*TransformsMatrixTests*": 58 of 58 pass. These cover the generator, the model, the ledger format, the opacity reading, and the declaration gate, including near-miss selectors, inherited expectations, the byte-identical control group, NaN and skip handling, and the drag-case curve exemption.
Mutation checks on the gate: removing stale detection, removing control-group protection or loosening the tolerance each turns tests red.
Transforms matrix record run 44905273571 on a hosted runner: every instrument control passed (including Screen and SettledOpacity), 378 cases, 0 failures, and every known-difference rule matched. That ledger is the one committed here.
Transforms matrix compare run 44955107840 on the head: 378 cases, 0 failures.
dotnet build tests/Reactor.AppTests.Host -c Release -p:Platform=x64: clean. I couldn't run the full Reactor.slnx Release build locally because restore can't reach nuget.org (TLS), so the CI Build solution job is the check for that.
CI on the head: unit tests, selftests, packaged, AOT, E2E and Build solution all pass.
Risk / breaking changes
None for users. CI cost: the sample (9 cases plus a short soak, about a minute) runs in each selftest shard, including packaged and AOT. The full matrix runs only in its own label-gated or nightly workflow. One new step in the Unit Tests job fetches the base commit's ledger and expectations.
Spec 066 section 12.1: a seeded 378-case matrix of transform modifiers,
implicit transitions and compositor features (every triple, plus pairwise
coverage of wrapper, flow, reduced motion, user code and sub-variant), an
observation recorder, an analytic composition model, positive and negative
controls for every oracle, a pool soak, platform probes, and the behaviour
ledger captured on main. Test-only, no product change.
The normal selftest corpus checks a 9-case sample; the Transforms matrix
workflow runs the full matrix on labelled PRs and nightly. Reactor.Tests
checks that every ledger change from a PR's base is declared.
Part of #1630.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- Expectations inherited unchanged from the base are ignored, so a merged
layer's entries never fail an unrelated PR; CI passes the base copies.
- Known-difference rules must attribute each model disagreement exactly
once, with {a,b} alternatives; #1623 no longer covers user-code cases,
and drags lost on a handoff cite #1642.
- Motion is measured from where each animation starts, layout Offset is
seeked like every channel, and a curve field records the 25% shape.
- New residue field after removeModifier (#1626), no visual reads on
stagger renters, retries on inconclusive oracles, cheaper record reruns.
- Instrument controls for the curve, pool residue and layout Offset.
- The control group keeps default dims; the baselines are read from the
checkout; negative selector tests; clearer parse errors.
- Workflow: persist-credentials false, other labels don't cancel a run,
and a transforms-matrix-record label records on a hosted runner. The
ledger is re-recorded there, so it's empty in this commit.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Artifact sizes for 61c556a vs the base branch (28a8722).
Packages (compressed .nupkg)
Artifact
base
PR
Δ
Microsoft.UI.Reactor.nupkg
2.09 MB
2.09 MB
-13 B (0.00%)
≈
Microsoft.UI.Reactor.Advanced.nupkg
528.6 KB
528.6 KB
-9 B (0.00%)
≈
Microsoft.UI.Reactor.Devtools.nupkg
287.8 KB
287.8 KB
-12 B (0.00%)
≈
Assemblies in Microsoft.UI.Reactor
Artifact
base
PR
Δ
Reactor.Analyzers.dll
437.5 KB
437.5 KB
+0 B (0.00%)
≈
Reactor.dll
2.83 MB
2.83 MB
+0 B (0.00%)
≈
Reactor.Localization.Generator.dll
16.0 KB
16.0 KB
+0 B (0.00%)
≈
Reactor.Wrappers.Abstractions.dll
10.5 KB
10.5 KB
+0 B (0.00%)
≈
Reactor.Wrappers.Generator.dll
164.0 KB
164.0 KB
+0 B (0.00%)
≈
Assemblies in Microsoft.UI.Reactor.Advanced
Artifact
base
PR
Δ
Reactor.Advanced.dll
1.08 MB
1.08 MB
+0 B (0.00%)
≈
Assemblies in Microsoft.UI.Reactor.Devtools
Artifact
base
PR
Δ
Microsoft.UI.Reactor.Devtools.dll
787.0 KB
787.0 KB
+0 B (0.00%)
≈
No size change beyond the noise floor. ✅
✅ smaller / ⚠️ larger / ≈ within noise. Sizes come from a Release dotnet pack on the CI runner: packages are the compressed .nupkg download size, assemblies the uncompressed DLL inside it. workflow run.
Coverage for 61c556a vs the base branch (28a8722) — unit + selftest merged.
Metric
base
PR
Δ
Line
88.70%
88.71%
+0.01 pp
≈
Branch
45.68% (18380/40239)
45.68% (18381/40239)
0.00 pp
≈
No coverage change beyond the noise floor. ✅
✅ higher / ⚠️ lower / ≈ within noise. Δ is in percentage points; coverage is unit + selftest merged (Debug x64) on the CI runner. Cobertura reports attached to the workflow run as artifacts.
Recorded by the Transforms matrix workflow in record mode (run 44879140336)
on main's behaviour plus this branch's harness: every instrument control
passed, 378 cases, no failures, and every known-difference rule matched.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The reason will be displayed to describe this comment to others. Learn more.
🟡 Changes recommended
The ledger gate can miss one-DIP changes to protected control fields, while recording can persist unstable samples or observations that failed validation.
Adds spec 066 layer V’s validation infrastructure for Reactor’s planned move to composition-visual transforms. It establishes a behavior baseline for later implementation layers without changing product code.
Changes:
Adds a seeded 378-case matrix, analytic model, WinUI recorder, instrument controls, pool soak, and platform probes.
Introduces ledger comparison and change-declaration checks, with resources shared across test hosts.
Adds label-gated and nightly matrix runs, plus documentation of usage, measurements, and remaining coverage limits.
- Write the ledger only when every check passes; otherwise write transforms-ledger.rejected.json.
- Record mode requires two agreeing observations before a change is recorded.
- Compare the control group byte-for-byte in the gate.
- Epsilon float comparisons, LINQ filters, narrower catch in the pool soak.
- Upload rejected ledgers from the matrix workflow.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
The tests can accept stale or screen-inaccurate captures and commit new cases without confirming reproducibility, undermining the reliability of their verdicts.
No open issues 🎉
3 previously missed issues
In code that hasn't changed since last review
Require two matching observations before recording new cases
When a case has no committed record, the expected is null branch stops after one observation, and the confirmation check then accepts it unconditionally. New matrix cases can therefore enter the ledger without demonstrating that their results reproduce. Require two matching observations for new cases; a single observation should suffice only when it matches an existing baseline.
Use guarded screen captures for translation drag observations
TranslationDrag cases derive their post-write pose and model verdict from RenderTargetBitmap, which can show movement that is absent on screen after translation is enabled (spec 066 §12.2, Q1f). These cases can therefore report model agreement while the visible drag is broken. Use a guarded screen capture for these observations, validate its Q1f control, and skip the screen-dependent checks when no interactive desktop is available.
Treat frame timeouts as inconclusive, not successful captures
If rendering stops after the instrument controls pass, the two-second timeout makes Frames return as if the requested frames arrived. Later seeks can therefore compare or record stale captures without reporting an instrument failure. Throw on timeout and ensure the recorder and pool soak treat it as an inconclusive observation, not as a product threw value.
- Facade Translation drag cases record the PrintWindow pose (screen), guarded on an interactive
desktop, with a Screen instrument whose positive control must reproduce Q1f (spec 066 §12.2).
- A frame timeout throws and makes the observation inconclusive instead of reading stale captures.
- Record mode needs two agreeing observations for a case the ledger doesn't have yet.
- Narrower catch clauses and Path.Join for baseline paths.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The hosted runner has no interactive desktop, so drag cases record screen=skip (run 44884004263);
every other field is unchanged. A skip now matches any value either way, and recording replaces
a recorded skip with a real pose.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The reason will be displayed to describe this comment to others. Learn more.
🟡 Changes recommended
Active findings can cause false ledger failures from screen-only differences and locale-dependent formatting, while NaN coordinates can incorrectly match finite values and hide real differences.
…reeing recordings
- Known rule for #1660: a one-shot keyframe entry added while .Opacity() declares 0.8 settles at
its last value, 1 (record run 44901927811). Every other opacity disagreement follows a step that
threw, and an existing rule already covers it.
- Record mode accepts a change once any two conclusive observations agree, so one outlier between
two agreeing runs doesn't reject it (ScaleRotationCenter.Opacity.Animate in that run).
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Hosted record run 44905273571: every rest step gains opacity, and model gains its opacity part
where they disagree. All of those follow a step that threw, except the #1660 keyframe cases. The
soak counts are unchanged. No other field changed.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
Pending Windows JIT, packaged, and NativeAOT validation leaves the state-mutating compositor harness’s baseline unconfirmed, while live WinUI verification and independent cross-family review remain unavailable.
The handoff migrates the drag at a moment that varies between runs, so the curve is measured from a
different start and flips (layer 2: Translation.Rotation.Animate 0.2 or 0.0,
Translation.Translation.Stagger 0.4 or 0.0). Comparisons skip curve for user=TranslationDrag cases
and recording keeps the committed value. The motion verdict is still compared, and the ledger is
unchanged.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
Human assessment is needed because exhausted stabilization attempts can leave drifting measurements unflagged, allowing unreliable comparisons or recordings to be accepted.
If two captures never agree within five tries, the sample is drifting, so the case is now
inconclusive and reruns instead of using the last reading. The ledger is unchanged.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
Human assessment is needed because failed screen captures can be recorded as valid empty results, potentially overwriting valid baselines or causing false regression failures.
PrintWindow failing now throws, so a drag case's screen observation becomes inconclusive and
reruns instead of recording screen=none. The instrument guard and the probes treat it as no
screen. The ledger is unchanged.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
Human assessment is needed because catch-all baseline rules conflate keyed-Memo exit-animation failures with separate defects, obscuring whether the attributed bugs are actually resolved.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ready-for-reviewAgent work and technical checks complete; awaiting review, not approval or mergetransforms-matrixRun the spec 066 transforms matrix against the ledger (Transforms matrix workflow)
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Layer V of spec 066 (§12.1, §13 row V). Test-only, no product change. It adds the harness that later layers use to prove they change behaviour only on purpose:
threw,pose,opacity(the settled opacity, so a lost update reads 1.00 instead of 0.80),active,motion,curve,running,residue,hit/uia/ttv,channel,poolanddiag. FacadeTranslationdrag cases also recordscreen, the PrintWindow pose (§12.2, Q1f). Motion is controller-driven, layoutOffsetincluded, so no verdict depends on wall-clock timing. A stale or drifting capture, a failed screen capture or a compositor that stops rendering makes a case inconclusive, and it reruns, instead of recording a wrong value. Under a facadeTranslationdrag,curveis recorded but not compared, because the drag migrates at a varying moment.transforms-ledger.known.json: [Bug] XAML implicit transitions crash with any compositor modifier (LayoutAnimation, Animate, Transition, InteractionStates, Keyframes) #1623, [Bug] UIElement.Translation writes are silently ignored after Reactor enables visual translation #1642, [Bug] A declared .Opacity() change overrides a looping .Keyframes opacity animation #1651, A finished one-shot .Keyframes() opacity entry overrides the declared .Opacity() #1660, §9 row 1 or §9 row 3.Reactor.Testsgate that diffs the PR's ledger against the base commit's. Every change must be declared intransforms-ledger.expected.jsonwith its §9 row or issue. The failure message includes a ready-to-paste entry. Expectations inherited unchanged from the base are ignored, so a merged layer's entries never fail an unrelated PR. The control group must stay byte-identical.Transforms matrixworkflow that runs the full matrix on PRs labelledtransforms-matrixand nightly. Label a PRtransforms-matrix-recordto record a new ledger on a hosted runner and download it as an artifact.Linked issue / spec
Part of #1630. Implements docs/specs/066-transforms-on-composition-visual.md §12.1 (layer V). Filed while building it: #1651, #1652, #1660 and #1665.
Test plan
dotnet test tests/Reactor.Tests --filter-class "*TransformsMatrixTests*": 58 of 58 pass. These cover the generator, the model, the ledger format, the opacity reading, and the declaration gate, including near-miss selectors, inherited expectations, the byte-identical control group, NaN andskiphandling, and the drag-casecurveexemption.dotnet build tests/Reactor.AppTests.Host -c Release -p:Platform=x64: clean. I couldn't run the fullReactor.slnxRelease build locally because restore can't reach nuget.org (TLS), so the CIBuild solutionjob is the check for that.Risk / breaking changes
None for users. CI cost: the sample (9 cases plus a short soak, about a minute) runs in each selftest shard, including packaged and AOT. The full matrix runs only in its own label-gated or nightly workflow. One new step in the Unit Tests job fetches the base commit's ledger and expectations.