Skip to content

evetest-pr: run one evetest per PR head, supersede stale runs - #6970

Merged
rene merged 1 commit into
lf-edge:masterfrom
milan-zededa:evetest-pr-dedupe
Oct 29, 2026
Merged

rene merged 1 commit into
lf-edge:masterfrom
milan-zededa:evetest-pr-dedupe

Conversation

@milan-zededa

@milan-zededa milan-zededa commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Description

The concurrency group of evetest-pr was built from head_branch of the
triggering PR Gate run. That is the PR branch for a review-triggered gate but
the default branch for a workflow_run-triggered one. As a result:

  • the two triggers of a single PR (approval and a finished/re-run PR build)
    landed in different groups, so both ran Claude to decide the skips and then
    the whole suite, occupying the evetest runners twice;
  • all workflow_run-triggered gates of unrelated PRs shared one group and
    cancelled each other.

The workflow-level group is dropped. Instead, discover-suites decides before
Claude is asked anything, using the PR number from the gate's run-context:

  • a trigger for a commit that another active run already tests is ignored;
  • a trigger for a new head commit of the PR cancels the active run of the older
    commit;
  • a trigger for a commit that is no longer the PR head is ignored.

Active runs are recognised by an evetest-active-<pr>-<sha> artifact that each
run uploads once it has passed the check. Unlike a per-commit status, it can be
found after a force-push.

A cancelled or timed-out run leaves the suites that never got a runner with
their "Queued" status, so a new cleanup job turns the statuses of that run
that are still pending into errors.

discover-suites now has actions: write, needed to cancel a superseded run.

How to test and validate this PR

On a PR that is approved and has a finished PR build:

  1. Trigger a second gate for the same commit (re-run the PR build, or comment
    /evetest) while the first evetest-pr run is active: the second run must
    stop in discover-suites without calling Claude, and the first run must
    continue.
  2. Push a new commit to the PR and get it approved/built: the run for the old
    commit must be cancelled and the new one must start.
  3. After the cancel, the Evetest / <suite> statuses of the old commit that
    were still pending must turn into errors.

This is a CI workflow change that can only be exercised on GitHub itself.

Changelog notes

None. No user-facing changes (CI workflow only).

PR Backports

  • 17.0-stable: Backported in [17.0-stable] evetest: add the evetest CI workflows and the fixes made to them on master #7111.
  • 16.0-stable: Not on its own. It will be backported alongside all evetest workflow files together in one PR once it works well.
  • 14.5-stable: Not on its own. It will be backported alongside all evetest workflow files together in one PR once it works well.
  • 13.4-stable: Not on its own. It will be backported alongside all evetest workflow files together in one PR once it works well.

Checklist

  • I've provided a proper description
  • I've added the proper documentation
  • I've tested my PR on amd64 device
  • I've tested my PR on arm64 device
  • I've written the test verification instructions
  • I've set the proper labels to this PR
  • I've checked the boxes above, or I've provided a good reason why I didn't
    check them.

The documentation and device-test boxes do not apply: this only changes a CI
workflow, and it has not been run on GitHub yet.

🤖 Generated with Claude Code

The concurrency group of evetest-pr was built from the head_branch of the
triggering PR Gate run. That is the PR branch for a review-triggered gate
but the default branch for a workflow_run-triggered one, so the two
triggers of a single PR landed in different groups (and both ran Claude
to decide the skips and then the whole suite), while all
workflow_run-triggered gates of unrelated PRs shared one group and
cancelled each other.

Drop the group and instead decide in discover-suites, before Claude is
asked anything, using the PR number from the gate's run-context:
- a trigger for a commit that another active run already tests is
  ignored;
- a trigger for a new head commit of the PR cancels the active run of the
  older commit;
- a trigger for a commit that is no longer the PR head is ignored.

Active runs are recognised by an evetest-active-<pr>-<sha> artifact that
each run uploads once it has passed the check; unlike a per-commit status
it can be found after a force-push.

A cancelled or timed out run leaves the suites that never got a runner
with their "Queued" status, so add a cleanup job that turns the statuses
of this run that are still pending into errors.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Milan Lenco <milan@zededa.com>
@rene
rene merged commit 389299a into lf-edge:master Oct 29, 2026
42 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

testing New test or test framework changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants