Repository navigation
evetest: reduce PR noise from CI comments - #7045
Merged
Merged
Conversation
milan-zededa
requested review from
eriknordmark,
rene,
shjala and
yash-zededa
as code owners
November 5, 2026 14:30
Post the failure analysis on a PR only for failures Claude judges related to the PR (RELATED: yes|likely), based on the PR diff and the nightly ledger. Unrelated failures stay visible through the red check and the job summary, which still hold the full analysis. Shorten the skipped-tests comment to a count and a link; the full list of skipped tests and the reasoning moves to the run's job summary. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Signed-off-by: Milan Lenco <milan@zededa.com>
milan-zededa
force-pushed
the
evetest-stop-spamming
branch
from
November 5, 2026 14:33
1b6d808 to
b15a6a1
Compare
rene
approved these changes
Nov 5, 2026
Merged
6 of 9 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The evetest CI jobs post a lot of comments on PRs (see #7037 for an example):
a failure analysis after every failed suite run, and a long comment listing
every skipped test with its reasoning. This reduces the noise.
at least one failure of the suite as related to the PR (
RELATED: yesorlikely). The judgment is made from the PR diff and the nightly ledger,and Claude is told to default to
unlikelyunless it can name the changedcode that explains the failure. The comment lists only the related
failures. Unrelated failures stay visible through the red check, the job
summary and the
claude-analysis-<suite>artifact, which still containthe full analysis. Collapsing of stale analysis comments is unchanged.
names, a link to the run summary and the
/evetestusage hint. The fulllist of skipped tests and the reasoning moved to the job summary of the
run. The comment is still re-created at the end of the conversation on
each run.
How to test and validate this PR
Not tested end-to-end yet; the workflow changes only run on a PR with the
evetest gate enabled. Locally checked that the YAML parses and that
live-investigation-watcher.shpassesbash -n. To validate, observe theevetest runs of this PR and of the next PRs:
run summary.
unrelated to the PR gets none.
Changelog notes
None. CI-only change, no user-facing changes.
PR Backports
This PR is not going to be backported on its own. All evetest workflow files
will be backported together once they work reliably.
Checklist
For backport PRs (remove it if it's not a backport):
And the last but not least:
check them.
This is a CI-only change: there is no device-facing code to test on amd64 or
arm64, and no documentation covers these PR comments.
🤖 Generated with Claude Code