Skip to content

evetest: cover what only eden tests covered so far - #7084

Merged
milan-zededa merged 3 commits into
lf-edge:masterfrom
milan-zededa:evetest-close-eden-gaps
Nov 12, 2026
Merged

milan-zededa merged 3 commits into
lf-edge:masterfrom
milan-zededa:evetest-close-eden-gaps

Conversation

@milan-zededa

@milan-zededa milan-zededa commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Description

We want to replace eden with evetest without losing test coverage. To find
what eden still covers that evetest does not, all eden CI suites were run
against a coverage-instrumented EVE (COVER=y) and compared with the evetest
nightly coverage (pillar / zedbox only). Eden covered 579 statements (0.9% of
pillar) that evetest never reached. Most of them come from a few test steps
that were lost when the eden tests were rewritten to evetest, or from eden
running the storage tests on ZFS. This PR adds them to evetest:

  • evetest: run TestVolumes also on ZFS: TestVolumes takes the FILESYSTEM
    parameter and TestStorageSuite runs it once more on ZFS
    (TestVolumesOnZFS), where volumes are backed by zvols. On ZFS the test
    also sets timer.gc.vdisk to its minimum, reboots the device, checks that
    the zvols survive the reboot, and checks that the volume GC destroys an
    unused zvol while it keeps the zvols of configured volumes.
  • evetest: check that a new controller epoch republishes all info:
    TestDeviceInfo re-applies its config with ControllerEpoch incremented and
    waits for a new ZInfoHardware (otherwise published only once after boot).
  • evetest: check network.json served by the metadata server:
    TestAppInstanceMetadata fetches /eve/v1/network.json from inside the app
    and checks the app and device identity and the external IPv4 address.

ZFS variants of TestMountedVolumes and TestExtraDiskAttach were tried as well,
but they added only one statement beyond TestVolumesOnZFS, so they were left
out.

With coverage collected from these three tests, 230 of the 579 eden-only
statements are now covered, including the zvol volume handler, ZFS dataset
GC, the metadata server's network.json handler and the controller-epoch
republish. Most of the rest is the local datastore test (being rewritten
separately), SATA/SMART disk info (ZTAF is more suitable to cover that) and
eden's environment specifics.

How to test and validate this PR

These are automated tests. Run them (with coverage, to see what they reach):

export EVETEST_COLLECT_COVERAGE=true EVETEST_COLLECT_ARTIFACTS=/tmp/evetest-artifacts
make evetest NAME=TestDeviceInfo,TestAppInstanceMetadata
EVETEST_FILESYSTEM=zfs make evetest NAME=TestVolumes

All three pass on amd64 (KVM) against a COVER=y build. The new phases end in
the checkpoints controller-epoch-republish, network-metadata-served,
rebooted-with-zvols, standalone-volumes-survived-reboot and
unused-zvol-garbage-collected. make evetest NAME=TestStorageSuite runs the
new TestVolumesOnZFS variant as part of the suite.

Changelog notes

No user-facing changes.

PR Backports

Checklist

  • I've provided a proper description

  • I've added the proper documentation

  • I've tested my PR on amd64 device

  • I've tested my PR on arm64 device

  • I've written the test verification instructions

  • I've set the proper labels to this PR

  • I've checked the boxes above, or I've provided a good reason why I didn't
    check them. (Documentation is in the test descriptions; arm64 not tested,
    the changes are architecture-independent test logic.)

🤖 Generated with Claude Code

milan-zededa and others added 3 commits November 11, 2026 15:41
TestVolumes always ran on ext4, where volumes are files. On ZFS,
volumes are backed by zvols and go through a different volume handler
(zvol creation and destruction, dataset cleanup, iSCSI/vhost block
devices), which only the eden storage tests covered so far.

TestVolumes now takes the FILESYSTEM parameter, and TestStorageSuite
runs it once more on ZFS (TestVolumesOnZFS), grouped with the other ZFS
tests so they can share a device. On ZFS, the test also reboots the
device once the standalone volumes are created and checks that all of
them are reported as created again, i.e. that the zvols survive a
reboot and volumemgr picks them up at boot. Before the reboot, it sets
timer.gc.vdisk to its minimum (60s): volumemgr garbage-collects ZFS
datasets every timer.gc.vdisk/10 (6 minutes by default), and reads the
interval only at boot. After the reboot the GC thus runs every 6
seconds while the zvols exist. The test then creates an unused zvol
directly on the device, named like a volume EVE has no config for, and
checks that the GC destroys it while it keeps the zvols of all
configured volumes. The ext4 run keeps its name and does not reboot.

TestMountedVolumes and TestExtraDiskAttach were not given ZFS variants:
with coverage collection, they reached only one statement on ZFS that
TestVolumesOnZFS did not.

Signed-off-by: Milan Lenco <milan@zededa.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A new controller epoch tells EVE that the controller may have lost its
state, so EVE must republish all of its info messages. Nothing in
evetest covered this.

TestDeviceInfo now re-applies its configuration with ControllerEpoch
incremented and waits for a new ZInfoHardware. EVE otherwise publishes
the hardware info only once after boot, so its arrival proves that the
full republish happened.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Milan Lenco <milan@zededa.com>
Apps read /eve/v1/network.json from the metadata server to learn the
external IP under which they are reachable, along with their own and the
device's identity. TestAppInstanceMetadata did not use this endpoint.

The test now fetches network.json from inside the app and asserts the
app instance UUID, the device UUID and name, and that external-ipv4
contains the IPv4 address of ethernet0, the port used by the app's
network instance.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Milan Lenco <milan@zededa.com>
@milan-zededa milan-zededa added stable Should be backported to stable release(s) testing New test or test framework changes labels Nov 11, 2026
@codecov

codecov Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 29.58%. Comparing base (de2c99f) to head (c60e740).
⚠️ Report is 6 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #7084      +/-   ##
==========================================
+ Coverage   29.09%   29.58%   +0.48%     
==========================================
  Files         549      560      +11     
  Lines       99421   101016    +1595     
==========================================
+ Hits        28925    29882     +957     
- Misses      68212    68896     +684     
+ Partials     2284     2238      -46     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions

Copy link
Copy Markdown
Contributor

Evetest: Claude chose to skip 58 test/suite name(s) as unrelated to or redundant for this PR. Which ones and why: run summary.

Comment /evetest all to run the full suite, or /evetest <name> <name> ... to run only specific tests/suites.

@milan-zededa

Copy link
Copy Markdown
Contributor Author

This is evetest-only PR, so whatever happened to eden is irrelevant.

@milan-zededa
milan-zededa merged commit 6a1353e into lf-edge:master Nov 12, 2026
73 of 83 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

stable Should be backported to stable release(s) testing New test or test framework changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants