Repository navigation
evetest: show failure analysis in the Allure report - #7002
Merged
Merged
Conversation
Clicking a failed test in the nightly Allure report showed only gotestsum's generic "Failed" message plus the raw log trace; the real failure message and Claude's conclusion ended up only in the nightly ledger and the job's step summary. Inject both into the JUnit XML before it is uploaded for the report: the new inject-junit-analysis.py rewrites each failing testcase's failure message with the failure message and conclusion that live-investigation-watcher.sh recorded in live-investigation.md, so Allure's test overview page displays them. The suite-level testcase fails whenever any of its subtests do, so it gets every recorded analysis, each prefixed with the subtest's name. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Milan Lenco <milan@zededa.com>
milan-zededa
requested review from
eriknordmark,
rene,
shjala and
yash-zededa
as code owners
October 31, 2026 16:44
rene
approved these changes
Oct 31, 2026
Merged
6 of 9 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Clicking a failed test in the nightly Allure report showed only gotestsum's
generic "Failed" message plus the raw log trace; the real failure message and
Claude's conclusion ended up only in the nightly ledger and the job's step
summary, so the report itself carried no information about why a test failed.
Inject both into the JUnit XML before it is uploaded for the report: the new
inject-junit-analysis.pyrewrites each failing testcase's failure messagewith the failure message and conclusion that
live-investigation-watcher.shrecorded in
live-investigation.md, so Allure's test overview page displaysthem. The suite-level testcase fails whenever any of its subtests do, so it
gets every recorded analysis, each prefixed with the subtest's name.
PR dependencies
None.
How to test and validate this PR
Tested on my fork: a nightly run restricted to a single suite
(
TestTelemetrySuite) with two intentionally broken assertions. The publishedAllure report shows, for each failed test (and aggregated on the suite-level
entry), the real assertion failure plus Claude's conclusion on the test
overview page:
https://milan-zededa.github.io/eve/test/master/runs/57/
To validate locally, run the injector on a gotestsum-generated JUnit XML and a
live-investigation.mdfrom a failed suite run:and inspect the rewritten
<failure message="...">attributes, or feed theresult to
allure generate.Changelog notes
None. CI and test tooling only.
PR Backports
Not to be backported as an individual PR: the evetest CI workflow files will
be backported as a whole once everything works well.
Checklist
I've provided a proper description
I've added the proper documentation
I've tested my PR on amd64 device
I've tested my PR on arm64 device
I've written the test verification instructions
I've set the proper labels to this PR
I've checked the boxes above, or I've provided a good reason why I didn't
check them.
Unchecked items do not apply: CI and test tooling only, no device impact.
🤖 Generated with Claude Code