Skip to content

[17.0-stable] evetest: sync with master and adapt the tests to EVE 17.0 - #6833

Merged
eriknordmark merged 3 commits into
lf-edge:17.0-stablefrom
milan-zededa:evetest-17.0
Oct 19, 2026
Merged

eriknordmark merged 3 commits into
lf-edge:17.0-stablefrom
milan-zededa:evetest-17.0

Conversation

@milan-zededa

@milan-zededa milan-zededa commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Description

Bring evetest to 17.0-stable and make it work with EVE 17.0.

The PR has three commits:

  1. evetest: add the image publishing workflow to 17.0-stable: the workflow
    that builds and publishes the evetest images from this branch.
  2. Copy evetest from master: first, evetest is made to be in sync with
    master, including all the enhancements from evetest: run several tests in parallel with a dashboard UI #6783 (not yet merged to master), plus a few small fixes to make some tests less sensitive to
    timing.
  3. evetest: adapt tests to EVE 17.0: in the follow-up commit, some tests
    are removed or modified to align with 17.0, and evetest/go.mod is
    updated to point to the pillar from 17.0-stable:
    • pkg/pillar is bumped to the 17.0-stable tip (da1f21294a; the
      eve-api version is unchanged, 17.0 pillar uses the same one).
    • Framework: on 17.0 a change of the adapters of a deployed app is
      rejected unless a purge is requested, so UpdateApplication now bumps
      the purge counter instead of the restart counter for such changes.
    • Removed tests of features that are not in 17.0: the konvert suite,
      vault key mode recovery, deferred queue metrics, native Kubernetes app
      networking, the USB hub chain test, and the NIC count, ordered
      interface, staged NIC and passthrough NIC change tests. The dynamic NIC
      addition part of TestNetworkAdapterPassthrough and the usbaddr
      wildcard phase of TestUSBFlashDriveHotplug are removed as well.
    • Adjusted tests to the 17.0 behavior: TestDNSFunctionality and the LPS
      network test do not rely on the management dnsmasq (DNS servers are
      read from /etc/resolv.conf), and the NTP servers reported for an app
      VIF are only the NI-level ones. The upgrade test starts EVE-K from
      17.0.1-rc2, the first 17.0 tag with the longhorn disk path fix.

The only open question is TestInfoRetriedAfterServerError, which currently
fails because #6304 is still not backported to 17.0-stable.

How to test and validate this PR

Run the tests against a build of 17.0-stable, for example:

EVETEST_EVE_VERSION=<17.0-stable build> make evetest NAME=<TestFunctionName>

The tests have been run with the Proxmox and the QEMU providers.

Changelog notes

No user-facing changes (evetest only).

Checklist

  • I've provided a proper description
  • I've added the proper documentation
  • I've tested my PR on amd64 device
  • I've tested my PR on arm64 device
  • I've written the test verification instructions
  • I've set the proper labels to this PR

And the last but not least:

  • I've checked the boxes above, or I've provided a good reason why I didn't
    check them.

@milan-zededa milan-zededa added the testing New test or test framework changes label Oct 16, 2026
Comment thread evetest/broker/provider/proxmox_qmp.go Dismissed
@milan-zededa
milan-zededa force-pushed the evetest-17.0 branch 5 times, most recently from 1d634aa to e3cfd85 Compare October 16, 2026 15:44
@milan-zededa

milan-zededa commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor Author

About the remaining Yetus failures (latest run: https://github.com/lf-edge/eve/actions/runs/38925121993). None of them is relevant for this PR:

  • bufcompat (6 incompatible protobuf changes in evetest/grpcapi/proto/sdn.proto): this PR syncs evetest with master, and before that evetest was not really used on 17.0. The protobuf API is internal to evetest (the test process, the broker and the SDN are built from the same tree), so there is no compatibility to preserve with the older version that was on this branch.
  • golangcilint (2 "new" issues): these are not real findings. They are the lines evetest/testapps/volverify/0 issues. and evetest/sdn/vm/0 issues., which golangci-lint prints for modules without any finding. Yetus counts every output line as an issue, and these two modules did not exist on 17.0-stable before. There are no new golangci-lint findings compared to the base.

@codecov

codecov Bot commented Oct 17, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 23.30%. Comparing base (4e587ba) to head (c536778).
⚠️ Report is 4 commits behind head on 17.0-stable.

Additional details and impacted files
@@               Coverage Diff               @@
##           17.0-stable    #6833      +/-   ##
===============================================
+ Coverage        22.94%   23.30%   +0.35%     
===============================================
  Files              513      523      +10     
  Lines            94855    96553    +1698     
===============================================
+ Hits             21768    22497     +729     
- Misses           71248    72034     +786     
- Partials          1839     2022     +183     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

milan-zededa and others added 3 commits October 18, 2026 19:32
Publish the evetest, broker, SDN and testapps images from 17.0-stable
the same way as from master.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Milan Lenco <milan@zededa.com>
Start with a copy of evetest from master. Test changes required
to match EVE 17.0 will be done in the follow-up commits.

The copy also includes a few fixes that are not in master yet: more
generous timeouts in tests that depended on the periodic connectivity
test or on slow app start, the port label used to isolate a node in
TestClusterPVCRedownload, and the issues reported by yetus.

It also carries the evetest changes of lf-edge#6781 (TestAppTeardownWithRemovedAdapter)
and lf-edge#6794 (TestMgmtProxy CDI HTTP DataVolume cases and timeouts), which are
in master but were missing from the first copy.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Milan Lenco <milan@zededa.com>
Point evetest at the pillar from 17.0-stable (the eve-api version it
uses is unchanged) and make the tests match what 17.0 supports.

Framework: on 17.0 a change of the adapters of a deployed app is
rejected unless a purge is requested, so UpdateApplication now bumps the
purge counter instead of the restart counter for such changes.

Removed tests of features that are not in 17.0: the konvert suite,
vault key mode recovery, deferred queue metrics, native Kubernetes app
networking, the USB hub chain test, and the NIC count, ordered
interface, staged NIC and passthrough NIC change tests. The dynamic NIC
addition part of TestNetworkAdapterPassthrough and the usbaddr wildcard
phase of TestUSBFlashDriveHotplug are removed as well.

Adjusted tests to the 17.0 behavior: TestDNSFunctionality and the LPS
network test do not rely on the management dnsmasq (DNS servers are read
from /etc/resolv.conf), and the NTP servers reported for an app VIF are
only the NI-level ones. The upgrade test starts EVE-K from 17.0.1-rc2,
the first 17.0 tag with the longhorn disk path fix.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Milan Lenco <milan@zededa.com>

@eriknordmark eriknordmark left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we not need a evetest/VERSION file any more?

@eriknordmark

Copy link
Copy Markdown
Contributor

No need to wait for eden run for these evetest-only changes.

@eriknordmark
eriknordmark merged commit f29503c into lf-edge:17.0-stable Oct 19, 2026
46 of 49 checks passed
@milan-zededa

Copy link
Copy Markdown
Contributor Author

Do we not need a evetest/VERSION file any more?

@eriknordmark
No, see #6773 (and in particular the "Why" section of the PR description)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

testing New test or test framework changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants