Repository navigation
evetest: put /persist on a disk of its own - #6540
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #6540 +/- ##
==========================================
+ Coverage 26.26% 26.55% +0.29%
==========================================
Files 515 525 +10
Lines 94872 96588 +1716
==========================================
+ Hits 24916 25650 +734
- Misses 68036 68829 +793
- Partials 1920 2109 +189 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
|
||
| // appImageName is the container image the escripts use as their test app | ||
| // (eden's eclient equivalent for evetest). | ||
| appImageName = "milan4zededa/evetest-ubuntu-ctr" |
There was a problem hiding this comment.
Should be lfedge image repo (we do publish this: https://hub.docker.com/r/lfedge/evetest-ubuntu-ctr)
There was a problem hiding this comment.
Fixed — lfedge/evetest-ubuntu-ctr:1.0. (It lives in #6539, so the fix is there.)
| // device boots from, which the broker can only do while the file stays | ||
| // where it put it. The overlay strategy is exactly that case; a standalone | ||
| // disk may since have been handed to a node the broker cannot reach. | ||
| if diskStrategy == provider.DiskImageOverlay { |
There was a problem hiding this comment.
Unfortunately this means that tests requiring CAPABILITY_EDIT_DEVICE_DISK will not be exercised in the github CI where we intend to use the Proxmox provider.
Are there going to be many tests in the konvert suite that require this?
There was a problem hiding this comment.
Two of eleven tests in the konvert suite: TestKvmToKRepartition (it has to leave a free tail past the last partition, which the image generator will not do at creation) and TestPersistWipeRestore (it destroys the /persist filesystem while the device is off — TestBackupCorruptRestore damages the backup from inside EVE and needs no capability). Plus TestPersistOnSeparateDisk in #6540. The rest run anywhere.
So on Proxmox the suite still covers the conversion itself, the app/volume/blob carry-over and the declined-repartition paths; what it loses is the grow-into-slack route and the /persist-loss recovery.
|
Given all the API and other changes in evetest, I think it makes sense to also increase the version number in evetest/VERSION |
| // own, not a lookalike built by the harness. | ||
| // | ||
| // The device must be powered off (PowerOff), and the test must have declared | ||
| // RequireCapabilities{CAPABILITY_EDIT_DEVICE_DISK}. |
There was a problem hiding this comment.
If these new methods are part of CAPABILITY_EDIT_DEVICE_DISK, then it should be part of the same PR (and the increased evetest version) as the rest of this capability (#6539). Otherwise we may end up with evetest broker image which claims CAPABILITY_EDIT_DEVICE_DISK (after 6539 merge), but actually does not yet implement CreatePersistPartition and DeletePartition, failing (instead of skipping) tests that depend on them.
There was a problem hiding this comment.
Correction: that move did not land. #6539 merged without these two edits, so
CreatePersistPartition/DeletePartition, the disk_index field and their
broker side stay in this PR, and evetest/VERSION is bumped here to 1.8 — so
the broker image that advertises CAPABILITY_EDIT_DEVICE_DISK at 1.8 is the
one that serves these edits, and 1.7 keeps meaning what it means in master.
Happy to split the framework half into its own PR if you would rather review it
separately from the test.
There was a problem hiding this comment.
OK, no need to split.
Since we do not run evetest in the CI yet, it does not matter. Only once we start using evetest from github actions and across multiple stable branches, we will need to be little more careful with versioning and capabilities reported.
9487828 to
6787069
Compare
6787069 to
0177b2f
Compare
|
VERSION bumped to 1.7, in #6539 with the rest of the framework change. |
0177b2f to
094e391
Compare
@milan-zededa given a number of pending evetest PRs, in which one (the first to merge, the last?) should we increase the version number? |
A device whose /persist lives on a separate disk exercises a different path through EVE's storage setup than the usual single-disk layout, and the in-field repartition reaches its grow branch there: with no /persist on the boot disk to shrink, the freed space is the only place to grow into. The harness could not arrange that shape, so those topologies were out of reach. Adds two offline disk edits. One writes a GPT on a blank extra disk with a single partition typed and named as EVE's /persist; the other removes a named partition from the boot disk. Used together they move /persist to its own disk, because EVE locates it by partition label across every block device rather than by which disk it sits on. The new partition is left unformatted on purpose: storage-init finds no filesystem and formats it on the next boot exactly as it would a factory-fresh disk, encryption feature included. Building the filesystem in the harness instead would mean reproducing EVE's mkfs by hand, and a test would not notice where the two had drifted apart. Disk edits now carry the index of the disk to act on, so an edit can reach an extra disk; the existing edits keep addressing the boot disk. Signed-off-by: eriknordmark <erik@zededa.com> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NZEoFvMK2UHByU3LAeXsxA
Covers a storage shape the single-disk layout does not reach: /persist on a dedicated disk rather than on the boot disk. The device is brought up normally, powered off while a persist partition is created on its extra disk and the boot disk's is removed, and then has to come back using the new one. That EVE adopts it is only half of what is asserted. The partition is handed over empty, so a device that comes back with /persist formatted ext4 is one that formatted it itself. The device also has to still take configuration from its controller: /persist carries the identity and the last configuration, and one that came back with a fresh /persist would answer over SSH exactly like a recovered one while being unable to be told anything. The freed space is checked against what the removed partition actually occupied, read from the partition table before and after, rather than against a size derived from the disk. The two are not the same question, and only the partition table's answer says what EVE can allocate. Signed-off-by: eriknordmark <erik@zededa.com> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NZEoFvMK2UHByU3LAeXsxA
094e391 to
1eefc47
Compare
@eriknordmark |
Description
Lets a test give a device a
/persiston a disk of its own, rather than on theboot disk where EVE creates it.
That shape exercises a different path through EVE's storage setup, and it is
where the in-field repartition reaches its grow branch: with no
/persistonthe boot disk to shrink, the space freed there is the only place to grow into.
The harness could not arrange it, so the
twodisk-ext4andtwodisk-zfstopologies from lf-edge/eden#1209 were out of reach.
Two offline disk edits are added. One writes a GPT on a blank extra disk with a
single partition typed and named as EVE's
/persist; the other removes a namedpartition from the boot disk. Used together they move
/persistto its owndisk, because EVE locates it by partition label across every block device rather
than by which disk it sits on. Disk edits now carry the index of the disk to act
on, so an edit can reach an extra disk; the existing edits keep addressing the
boot disk.
The new partition is deliberately left unformatted. storage-init finds no
filesystem and formats it on the next boot exactly as it would a factory-fresh
disk — with the encryption feature the vault needs. Eden's escripts run
mkfs.ext4 -L PERSISTthemselves, but EVE's ownstorage-init.shusesmkfs -t ext4 -F -F -O encryptfollowed bytune2fs -O encrypt; reproducingthat by hand in the harness would risk a filesystem that differs from EVE's in
ways a test never thinks to check. Leaving it to EVE also keeps mkfs tooling out
of the broker image.
The second commit adds
TestPersistOnSeparateDisk, which drives the sequenceend to end.
PR dependencies
None. The
EditDeviceDiskbroker RPC and theCAPABILITY_EDIT_DEVICE_DISKcapability these two edits extend are in master, from #6539.
How to test and validate this PR
Automated, by
TestPersistOnSeparateDisk(part ofTestStorageSuite):It brings a device up normally, confirms
/persistis on the boot disk, powersit off, creates a persist partition on the extra disk and removes the boot
disk's, then powers it back on. Passes at ~256s.
What the assertions are actually pinned on, for a reviewer validating by hand:
/persiststarts on/dev/vda9and ends on/dev/vdb1.reporting a formatted
/persistis one that formatted it itself — this is thecheck that the "let EVE do the mkfs" decision above actually holds.
/persistcarriesthe identity and the last configuration, and a device that came back with a
fresh one would answer over SSH exactly like a recovered one while being
unable to be told anything.
partition actually occupied, read from the partition table before and after,
rather than against a size derived from the disk. In the passing run those
match exactly: free space goes from 2079232 to 42944395264 bytes, a gain of
42942316032, which is precisely what P3 held.
Changelog notes
No user-facing changes. Test-only, plus the evetest framework support that test
needs. That support extends the broker API, so
evetest/VERSIONgoes to 1.8 —a broker image advertising
CAPABILITY_EDIT_DEVICE_DISKat 1.8 is one thatserves these two edits.
PR Backports
stable branches.
Checklist
two new RPC messages carry documentation of what they do and why.
device through the evetest broker on an amd64 host.
🤖 Generated with Claude Code
https://claude.ai/code/session_01NZEoFvMK2UHByU3LAeXsxA