Analyze delegated coverage once per test run - #1891
Draft
wenytang-ms wants to merge 2 commits into
Draft
Conversation
A delegated runner previously received a bare output directory and had to supply its own JaCoCo agent. That left the agent free to be a different version than the analyzer, and left one directory shared between every run. Hand it a versioned artifact request instead. It names the artifact, a directory that belongs to a single test run, and the agent this extension ships, which is the only one guaranteed to match the analyzer. Creation, retention and removal of the directory stay on this side, so a runner never deletes data another run may still be writing. The capability becomes `jacoco-exec@1`, pinning the revision of the contract rather than the feature, so a later incompatible revision can be advertised under a name of its own. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8b7a64f2-3b9f-4328-a173-f4728a1fa95c
Coverage was collected and published inside the loop that runs each batch of tests. A run split over several batches therefore published a report per batch, and because file coverage is stored rather than merged, the last batch hid what the batches before it had covered. The pre-run cleanup sat in the same loop, so an explicit `appendResult: false` also threw away the data this very run had just produced. Collect once per project and analyze after every batch has finished. For a delegated run, `appendResult` becomes a matter of how wide to analyze -- this run's directory, or the root that still holds the runs before it -- rather than of deleting data, which is what made concurrent runs able to erase each other. Analyzing no execution data at all reported every line as uncovered, which reads exactly like a run that genuinely covered nothing. Report it as the failure it is, so a run whose agent never attached cannot be mistaken for 0%. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8b7a64f2-3b9f-4328-a173-f4728a1fa95c
wenytang-ms
force-pushed
the
wenyt/delegate-test-p0
branch
from
July 27, 2026 05:57
ec906c1 to
d0e7e25
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Coverage for a delegated test run was collected and published from inside the loop that runs each batch of tests, one batch per project and per set of classes. A run split over several batches therefore published a report per batch. Because file coverage is stored rather than merged, the last batch hid everything the batches before it had covered.
The pre-run cleanup lived in the same loop, which made it worse in two ways: an explicit
appendResult: falsethrew away the data the run itself had just produced, and because the directory was fixed per project rather than per run, two runs started close together could erase each other.Analyzing no execution data at all reported every line as uncovered — indistinguishable from a run that genuinely covered nothing. A run whose agent never attached looked like a real 0%.
What
One directory per run, analyzed once. A delegated run gets a directory of its own, and every project's data is analyzed after all batches have finished rather than after each one.
appendResultbecomes a matter of scope, not deletion. Analyze only this run's directory, or the root that still holds the runs before it. Nothing is deleted while a run might be writing into it, which is what let concurrent runs erase each other. Old run directories are collected on a time bound instead, and a directory younger than that bound is never touched.No execution data is an error. A run that produced nothing reports the failure rather than a fabricated 0%.
The contract
Delegated coverage now goes through an explicit request rather than a bare output path:
.execfiles into it and must not clear it or write withappend=false.IRunTestContext.cancellationToken, not part of the coverage request.Runners advertise support with the
jacoco-exec@1capability, so the version is negotiated rather than assumed.Testing
tscandeslintclean.mvnw clean verifybuilds the Java plugin clean, including checkstyle.CoverageHandlerTestcase covers the no-execution-data failure.