Skip to content

[16.0-stable] evetest: sync with master and adapt the tests to EVE 16.0 - #6842

Merged
milan-zededa merged 4 commits into
lf-edge:16.0-stablefrom
milan-zededa:evetest-16.0
Oct 26, 2026
Merged

milan-zededa merged 4 commits into
lf-edge:16.0-stablefrom
milan-zededa:evetest-16.0

Conversation

@milan-zededa

@milan-zededa milan-zededa commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

Description

Bring evetest to 16.0-stable and make it work with EVE 16.0.

The PR has four commits:

  1. evetest: add the image publishing workflow to 16.0-stable: the workflow
    that builds and publishes the evetest images from this branch.
  2. Copy evetest from master: evetest is made to be in sync with master,
    including the evetest changes of USB passthrough: fixes for device model changes at runtime #6781 and Fix CDI HTTP DataVolume imports through the mgmtproxy cni0 TLS proxy #6794, plus a few small fixes
    that make some tests less sensitive to timing.
  3. evetest: adapt tests to EVE 17.0: the same adaptation as in [17.0-stable] evetest: sync with master and adapt the tests to EVE 17.0 #6833
    (17.0-stable), kept as a separate commit so that the 16.0 changes on top of
    it are easy to review. It removes the tests of features that are not in
    17.0, adjusts the tests to the 17.0 behavior and points evetest/go.mod to
    the 17.0-stable pillar.
  4. evetest: adapt tests to EVE 16.0: on top of that, the tests are made to
    match what 16.0 supports, and evetest/go.mod is updated to point to the
    pillar from 16.0-stable:
    • The kubevirt hypervisor (eve-k) is not supported in 16.0-stable, so it is
      removed from the framework together with everything that exists only for
      it (cluster configuration builders, cluster-info gRPC calls, kubectl
      helpers, RemoteConsole, DialViaSSH, the K3s registry mirror setup).
    • Removed the tests that cover eve-k only (the cluster and kuberegistration
      suites, the VMIRS purge test, the vault zvol trim test and the volume size
      alignment test). Tests that run on both flavors are kept without their
      eve-k specific parts.
    • Removed tests of features that are not in 16.0: TestPNAC (no SCEP
      enrollment), TestHardwareInventory (HwInventorySupport is not
      reported) and TestContainerdCPUs (no ctrd_max_vcpus confinement of
      containerd).
    • The 16.0 sshd has a broken Subsystem sftp path, so scp over SFTP never
      works there. scp now falls back to the legacy protocol (-O), escaping
      the remote path with backslashes.
    • The PCI access tracer builds eve-tools/bpftrace-compiler from its
      vendor directory only if the checkout has one, which 16.0-stable does
      not.

Some tests need fixes that are not in 16.0-stable yet and are backported in
separate PRs:

How to test and validate this PR

Run the tests against a build of 16.0-stable (with the backports above for
the tests listed there), for example:

EVETEST_EVE_VERSION=<16.0-stable build> make evetest NAME=<TestFunctionName>

The tests have been run with the QEMU provider.

Changelog notes

No user-facing changes (evetest only).

Checklist

  • I've provided a proper description
  • I've added the proper documentation
  • I've tested my PR on amd64 device
  • I've tested my PR on arm64 device
  • I've written the test verification instructions
  • I've set the proper labels to this PR

And the last but not least:

  • I've checked the boxes above, or I've provided a good reason why I didn't
    check them.

Comment thread evetest/utils/tar.go Dismissed
Comment thread evetest/utils/tar.go Dismissed
@codecov

codecov Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 26.89%. Comparing base (c4cf53c) to head (1920752).
⚠️ Report is 17 commits behind head on 16.0-stable.

Additional details and impacted files
@@               Coverage Diff               @@
##           16.0-stable    #6842      +/-   ##
===============================================
- Coverage        29.12%   26.89%   -2.24%     
===============================================
  Files               10       25      +15     
  Lines             1133     4332    +3199     
===============================================
+ Hits               330     1165     +835     
- Misses             768     2938    +2170     
- Partials            35      229     +194     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

milan-zededa and others added 4 commits October 22, 2026 16:10
Publish the evetest, broker, SDN and testapps images from 16.0-stable
the same way as from master.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Milan Lenco <milan@zededa.com>
Start with a copy of evetest from master. Test changes required
to match EVE 17.0 will be done in the follow-up commits.

The copy also includes a few fixes that are not in master yet: more
generous timeouts in tests that depended on the periodic connectivity
test or on slow app start, the port label used to isolate a node in
TestClusterPVCRedownload, and the issues reported by yetus.

It also carries the evetest changes of lf-edge#6781 (TestAppTeardownWithRemovedAdapter)
and lf-edge#6794 (TestMgmtProxy CDI HTTP DataVolume cases and timeouts), which are
in master but were missing from the first copy.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Milan Lenco <milan@zededa.com>
Point evetest at the pillar from 17.0-stable (the eve-api version it
uses is unchanged) and make the tests match what 17.0 supports.

Framework: on 17.0 a change of the adapters of a deployed app is
rejected unless a purge is requested, so UpdateApplication now bumps the
purge counter instead of the restart counter for such changes.

Removed tests of features that are not in 17.0: the konvert suite,
vault key mode recovery, deferred queue metrics, native Kubernetes app
networking, the USB hub chain test, and the NIC count, ordered
interface, staged NIC and passthrough NIC change tests. The dynamic NIC
addition part of TestNetworkAdapterPassthrough and the usbaddr wildcard
phase of TestUSBFlashDriveHotplug are removed as well.

Adjusted tests to the 17.0 behavior: TestDNSFunctionality and the LPS
network test do not rely on the management dnsmasq (DNS servers are read
from /etc/resolv.conf), and the NTP servers reported for an app VIF are
only the NI-level ones. The upgrade test starts EVE-K from 17.0.1-rc2,
the first 17.0 tag with the longhorn disk path fix.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Milan Lenco <milan@zededa.com>
Point evetest at the pillar from the tip of 16.0-stable. The eve-api
version used by that pillar is the one evetest already referenced.

The kubevirt hypervisor (eve-k) is not supported in 16.0-stable, so
remove it from the framework together with everything that exists only
for it: HypervisorKubevirt, the EdgeCluster and cluster config builders,
the cluster-info gRPC calls and "evetest cluster" command, the kubectl
helpers, RemoteConsole and DialViaSSH, and the K3s registry mirror setup.

Remove the tests that cover eve-k only (the cluster and kuberegistration
suites, the VMIRS purge test, the vault zvol trim test and the volume
size alignment test). Tests that run on both flavors are kept without
their eve-k specific parts.

TestPNAC is dropped as well, since SCEP enrollment is not implemented in
16.0-stable. The OCI datastore upgrade variant starts from the latest
16.0.x release (16.0.2-lts); if it fails because of the missing
downloader fix, backport the fix and bump the version.

TestHardwareInventory and TestContainerdCPUs are dropped too: 16.0-stable
does not report HwInventorySupport (lf-edge#5535) and has no ctrd_max_vcpus
confinement of containerd (lf-edge#6771).

The 16.0 sshd has a broken "Subsystem sftp" path, so scp over SFTP never
works there. scp now falls back to the legacy protocol (-O) when the SFTP
attempt fails, escaping the remote path with backslashes. The PCI access
tracer builds eve-tools/bpftrace-compiler from its vendor directory only
when the checkout has one, which 16.0-stable does not.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Milan Lenco <milan@zededa.com>
@milan-zededa
milan-zededa merged commit 5052aca into lf-edge:16.0-stable Oct 26, 2026
65 of 72 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants