Repository navigation
[13.4-stable] evetest: sync with master and adapt the tests to EVE 13.4 - #6899
Merged
Merged
Conversation
Publish the evetest, broker, SDN and testapps images from 13.4-stable the same way as from master. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Signed-off-by: Milan Lenco <milan@zededa.com>
milan-zededa
requested review from
deitch,
eriknordmark and
yash-zededa
as code owners
October 22, 2026 08:37
This was referenced Oct 22, 2026
milan-zededa
force-pushed
the
evetest-13.4-pr
branch
from
October 22, 2026 09:08
2ec0f7d to
a31ff19
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## 13.4-stable #6899 +/- ##
===============================================
+ Coverage 24.78% 32.66% +7.88%
===============================================
Files 8 9 +1
Lines 1138 1090 -48
===============================================
+ Hits 282 356 +74
+ Misses 788 654 -134
- Partials 68 80 +12 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Start with a copy of evetest from master. Test changes required to match EVE 17.0 will be done in the follow-up commits. The copy also includes a few fixes that are not in master yet: more generous timeouts in tests that depended on the periodic connectivity test or on slow app start, the port label used to isolate a node in TestClusterPVCRedownload, and the issues reported by yetus. It also carries the evetest changes of lf-edge#6781 (TestAppTeardownWithRemovedAdapter) and lf-edge#6794 (TestMgmtProxy CDI HTTP DataVolume cases and timeouts), which are in master but were missing from the first copy. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Signed-off-by: Milan Lenco <milan@zededa.com>
Point evetest at the pillar from 17.0-stable (the eve-api version it uses is unchanged) and make the tests match what 17.0 supports. Framework: on 17.0 a change of the adapters of a deployed app is rejected unless a purge is requested, so UpdateApplication now bumps the purge counter instead of the restart counter for such changes. Removed tests of features that are not in 17.0: the konvert suite, vault key mode recovery, deferred queue metrics, native Kubernetes app networking, the USB hub chain test, and the NIC count, ordered interface, staged NIC and passthrough NIC change tests. The dynamic NIC addition part of TestNetworkAdapterPassthrough and the usbaddr wildcard phase of TestUSBFlashDriveHotplug are removed as well. Adjusted tests to the 17.0 behavior: TestDNSFunctionality and the LPS network test do not rely on the management dnsmasq (DNS servers are read from /etc/resolv.conf), and the NTP servers reported for an app VIF are only the NI-level ones. The upgrade test starts EVE-K from 17.0.1-rc2, the first 17.0 tag with the longhorn disk path fix. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Signed-off-by: Milan Lenco <milan@zededa.com>
Point evetest at the pillar from the tip of 16.0-stable. The eve-api version used by that pillar is the one evetest already referenced. The kubevirt hypervisor (eve-k) is not supported in 16.0-stable, so remove it from the framework together with everything that exists only for it: HypervisorKubevirt, the EdgeCluster and cluster config builders, the cluster-info gRPC calls and "evetest cluster" command, the kubectl helpers, RemoteConsole and DialViaSSH, and the K3s registry mirror setup. Remove the tests that cover eve-k only (the cluster and kuberegistration suites, the VMIRS purge test, the vault zvol trim test and the volume size alignment test). Tests that run on both flavors are kept without their eve-k specific parts. TestPNAC is dropped as well, since SCEP enrollment is not implemented in 16.0-stable. The OCI datastore upgrade variant starts from the latest 16.0.x release (16.0.2-lts); if it fails because of the missing downloader fix, backport the fix and bump the version. TestHardwareInventory and TestContainerdCPUs are dropped too: 16.0-stable does not report HwInventorySupport (lf-edge#5535) and has no ctrd_max_vcpus confinement of containerd (lf-edge#6771). The 16.0 sshd has a broken "Subsystem sftp" path, so scp over SFTP never works there. scp now falls back to the legacy protocol (-O) when the SFTP attempt fails, escaping the remote path with backslashes. The PCI access tracer builds eve-tools/bpftrace-compiler from its vendor directory only when the checkout has one, which 16.0-stable does not. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Signed-off-by: Milan Lenco <milan@zededa.com>
Point evetest at the pillar from the tip of 14.5-stable. The eve-api version used by that pillar is the one evetest already referenced. Vector-based log processing (vector.config) is not present in 14.5, so the Vector tests of the telemetry suite are removed together with their helpers and test data. timer.deviceinfo.interval is not configurable either; the periodic device info is published every 10 minutes +/- 20%. TestIntermittentConnectivity, TestDeviceIPv6Connectivity and the app-routed management path test now use the periodic device metrics (every 20s) as the "EVE still reaches the controller" signal, while the deferred info test waits up to 13 minutes for the next device info. Bond status reporting and 802.3ad bond creation, LPS network configuration and the HTTP/protobuf vcomlink API are not available in 14.5, so remove TestActiveBackupBond, TestLACPBond, TestVLANSubinterfacesOnTopOfLAGs, TestBootstrapWithLACPBond, TestNetworkLocalChanges and TestVCom. TestSwitchNIPortConfigRace is removed too, since it needs the IfInstanceID tracking of NIM interfaces and bridges, whose backport would be very large. TestInfoRetriedAfterServerError, TestDownloadBurstWithDeadMgmtPorts, TestHaltUnresponsiveGuest and TestSwitchNIDeletedWithAppsAttached are removed as well, because they need fixes that are not backported. The network models only these tests used are removed too. TestDeviceInfo waits up to 13 minutes for a device info that carries the storage size, which the info sent right after the config can lack. TestAppLocalInfo discards buffered app updates before issuing an LPS command, so that a stale HALTING does not end the wait early. DnsConfigExclusively is not honored by 14.5, so TestDNSFunctionality expects the DHCP and static DNS servers of ethernet2 combined, its /etc/resolv.conf check expects only the DHCP-provided servers (14.5 does not write static ones there), and its broken-DNS fallback phase uses ethernet3 alone. The upgrade test starts from 14.5.0-lts. The OCI datastore variant starts from the 14.5-stable snapshot 0.0.0-14.5-stable-42a70aa5, because the downloader fix it needs is not part of any 14.5.x release yet. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Signed-off-by: Milan Lenco <milan@zededa.com>
milan-zededa
force-pushed
the
evetest-13.4-pr
branch
from
October 22, 2026 09:51
a31ff19 to
946eb1a
Compare
Point evetest at the pillar from the tip of 13.4-stable. The eve-api version used by that pillar is the one evetest already referenced. debug.kernel.remote.loglevel and debug.syslog.remote.loglevel do not exist in 13.4, so TestRemoteLogLevelNone is removed and TestDeviceLogs only sets the local kernel log level. The following tests are removed because the features they check are not present in 13.4: TestDeviceIPv6Connectivity and TestApplicationIPv6Connectivity (IPv6 configuration received from dhcpcd), and TestDeviceNTPConfig and TestApplicationNTPConfig (multiple and domain-based NTP servers, NtpServerExclusively). The network models only these tests used are removed too. EVE 13.4 keeps the ControllerCert publication in /persist/status instead of in memory, so the certificate change tests read it from there and the post-reboot check no longer proves a refetch. 13.4 hands out DHCP leases with an all-ones netmask, so TestApplicationGateway accepts the propagated route via the NI bridge as well as via the app gateway. 13.4 replaces the static DNS servers of a port with the ones learned from dhcpcd, so TestVGAPassthroughNoHostAccess does not wait for the added DNS server to be reported. The upgrade test starts from 13.4.3-lts. The OCI datastore variant starts from the 13.4-stable snapshot 0.0.0-13.4-stable-e7581dbd, because the downloader fix it needs is not part of any 13.4.x release yet. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Signed-off-by: Milan Lenco <milan@zededa.com>
milan-zededa
force-pushed
the
evetest-13.4-pr
branch
from
October 22, 2026 10:14
946eb1a to
72812da
Compare
rene
approved these changes
Oct 25, 2026
4 of 7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Bring
evetestto13.4-stableand make it work with EVE 13.4.The PR has six commits:
evetest: add the image publishing workflow to 13.4-stable: the workflowthat builds and publishes the evetest images from this branch.
Copy evetest from master:evetestis made to be in sync with master.evetest: adapt tests to EVE 17.0: the same adaptation as in the 17.0-stableand 16.0-stable PRs, kept as a separate commit so that the changes on top of
it are easy to review. It removes the tests of features that are not in 17.0,
makes
UpdateApplicationrequest a purge for adapter changes (as 17.0requires) and points
evetest/go.modto the17.0-stablepillar.evetest: adapt tests to EVE 16.0: the same adaptation as in [16.0-stable] evetest: sync with master and adapt the tests to EVE 16.0 #6842(16.0-stable): eve-k and everything that exists only for it is removed from
the framework and the test suites, and the tests of features missing from
16.0 are dropped (
TestPNAC,TestHardwareInventory,TestContainerdCPUs).evetest: adapt tests to EVE 14.5: the same adaptation as in [14.5-stable] evetest: sync with master and adapt the tests to EVE 14.5 #6873(14.5-stable). Vector-based log tests, bond and LACP tests,
TestVCom,TestSwitchNIPortConfigRaceand the tests that need fixes not backportedthere are removed, the periodic device info is no longer used as the "EVE
still reaches the controller" signal, and the DNS tests expect the DHCP and
static DNS servers combined.
evetest: adapt tests to EVE 13.4: on top of that, the tests are made tomatch what 13.4 supports, and
evetest/go.modis updated to point to thepillar from
13.4-stable:servers are not in 13.4, so
TestDeviceIPv6Connectivity,TestApplicationIPv6Connectivity,TestDeviceNTPConfigandTestApplicationNTPConfigare removed, together with the network modelsthat only these tests used.
debug.kernel.remote.loglevelanddebug.syslog.remote.logleveldo notexist in 13.4, so
TestRemoteLogLevelNoneis removed andTestDeviceLogssets only the local kernel log level.
13.4.3-lts. The OCI datastore variant startsfrom the
13.4-stablesnapshot0.0.0-13.4-stable-e7581dbd, because thedownloader fix it needs is not part of any 13.4.x release yet.
How some tests had to be adapted specifically for 13.4
ControllerCertpublication in
/persist/status/zedagentinstead of in memory (/run).WaitUntilHasControllerCertand the cert-change tests(
TestControllerCertChange,TestEncryptedControllerCertChange) read itfrom the persistent path. The post-reboot step of these tests can no longer
prove that zedagent refetched the certificates (the publication survives the
reboot), so it only checks that both certificates are still published.
/32leases andrewrites application-to-application gateway routes to the NI bridge (dropped
in later releases).
TestApplicationGatewaytherefore accepts the propagatedroute via the NI bridge as well as via the app gateway
(
default via 172.28.2.[12]).DNS servers of a port with the ones learned from dhcpcd. In
TestDNSFunctionalitythe expected DNS servers of the first phase are theDHCP-provided ones only, and
TestVGAPassthroughNoHostAccessno longer waitsfor the added DNS server to be reported.
treats
sftp://host:portas a bare host name and fails to resolve it(
lookup sftp on ...: no such host). The framework (SFTPStorage.toProto)therefore sends the datastore FQDN as plain
host:port. This is a frameworkchange, not a test change, and it works with newer EVE versions as well.
NOTICE-severity error while the datastore config is still being applied.The storage tests'
volumeHasErrorhelper ignores it.Some tests need fixes that are not in
13.4-stableyet and are backported inseparate PRs:
TestZVolProvisionedSizeReported: [13.4-stable] zfs: report zvol provisioned size (volsize) in volume metrics #6902TestPurgeNeverActivatedApp: [13.4-stable] zedmanager: fix purge stuck when app was never activated #6901 (depends on [13.4-stable] zedmanager: drive purge state machine to completion in doUpdate #6900)TestVMAppPurgeAfterPowerCycle: [13.4-stable] pillar: keep purges from leaving the old app generation behind #6905TestExternalPortScanKVM: [13.4-stable] debug: let sshd offer only publickey login #6904TestBootstrapWithNetworkBoot: [13.4-stable] Fixes newc initrd filename null termination #6906TestNetworkConfigFallback: [13.4-stable] dpcreconciler: pass the IP version mode to dhcpcd --release and --exit #6907test,
TestSwitchNIDeletedWithAppsAttached, is not included here yet.PR dependencies
None.
How to test and validate this PR
Run the tests against a build of
13.4-stable(with the backports above forthe tests listed there), for example:
Changelog notes
No user-facing changes (evetest only).
Checklist
And the last but not least:
check them.