Repository navigation
evetest: upgrade a three-node EVE-k cluster - #6673
Conversation
|
@andrewd-zededa do you have a similar cluster upgrade test underway? If so we can merge the two rather than land both. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #6673 +/- ##
==========================================
+ Coverage 28.10% 28.38% +0.27%
==========================================
Files 534 545 +11
Lines 97717 99456 +1739
==========================================
+ Hits 27464 28231 +767
- Misses 68019 68797 +778
- Partials 2234 2428 +194 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
2508e0c to
fbcce1f
Compare
|
Claude finished @rene's task in 3m 41s —— View job Review complete
I reviewed Correctness — holds up:
Material gap (posted inline at the "Suite placement" doc):
Minor (not blocking, not posted inline):
I could not run |
| // Suite placement | ||
| // --------------- | ||
| // - Standalone: its devices run a different EVE version than | ||
| // TestNodeClusterSuite's and cannot be reused. |
There was a problem hiding this comment.
Confirming milan-zededa's point with the concrete consequence: TestThreeNodesUpgrade is not referenced from testsuite_test.go's RunTestSuite nor any other suite, so as written it will never be scheduled by CI — it only runs when invoked explicitly via make evetest NAME=TestThreeNodesUpgrade. The "Standalone" rationale here (devices run a different EVE version and can't be reused by TestNodeClusterSuite) is valid for reuse, but it doesn't register the test for execution. Either add it to an existing/new suite entry point, or move it under tests/upgrade as suggested, so it actually gets exercised.
Note there is precedent for standalone-but-unrun cluster tests here (e.g. vmapp_test.go, pvcredownload_test.go are "not in TestNodeClusterSuite yet"), so this is an accepted interim state — but worth resolving before relying on this test as a regression guard.
There was a problem hiding this comment.
@eriknordmark Still waiting for this one to be addressed. Otherwise it would be good to go..
There was a problem hiding this comment.
Addressed: the test is registered in TestUpgradeSuite (see the thread on line 201).
fbcce1f to
4370107
Compare
Form a three-node cluster on an earlier EVE-k, deploy a container app, and upgrade the nodes one at a time to the version under test, bootstrap node last. After each upgrade a fresh cluster report must show every node Ready, and at the end the app must still answer. The test runs in TestUpgradeSuite and by default starts from the same pre-17.0 master build as the suite's EVE-k upgrade variant, since 17.0-stable lacks a fix needed for reliable app deployment. The initial version and repo are parameters; the target defaults to the checked-out HEAD. Signed-off-by: eriknordmark <erik@zededa.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ULraDivKMXrdxJxycz6g6h
4370107 to
7a7d70b
Compare
|
Local run results, before the move to
The new default baseline, |
| // which is needed for reliable app deployment. | ||
| // For the time being, we will use EVE version from the master, at the commit just | ||
| // before the big rewrite of kube-init from shell scripts to Go. | ||
| const initialEVEVersionForKubevirt = "0.0.0-master-66475696" |
There was a problem hiding this comment.
Btw. we can now use:
git tag --contains 3f95383ccf76b7803736122065967de220450402
17.0.1-rc1
17.0.1-rc2
17.0.1-rc3
But this can be changed later, when 17.0.1 is finally released.
There was a problem hiding this comment.
FWIW We refer to the same master branch hash in another cluster test. Let's fix both when 17.0.1 is out.
|
@eriknordmark Once #6833 is merged, you could backport this to 17.0 (I think we should have this test there as well and find out why it is failing). |
|
No need to wait for eden for this evetest PR. |
Description
Adds
TestThreeNodesUpgrade, an evetest that forms a three-node EVE-k cluster on an earlier EVE, deploys a container app, and upgrades the nodes one at a time to the version under test, bootstrap node last. After each node's upgrade, a fresh cluster report must show all three nodes Ready with healthy storage before the next one starts, so the cluster runs mixed versions for most of the test. At the end the app must still answer over its port forward.EVETEST_INITIAL_EVE_VERSION(default0.0.0-master-66475696, the same baseline asTestEVEUpgradeKubevirtToKubevirt) andEVETEST_INITIAL_EVE_REPO(defaultlfedge/eve) pick the baseline.EVETEST_EVE_VERSIONpicks the target and defaults to the checked-out HEAD, so a run on master upgrades to master.EVETEST_EVE_LIVE_IMAGEupgrades to the local build instead, as inTestEVEUpgrade.The test runs in
TestUpgradeSuite. It builds the same cluster asTestThreeNodesCluster, with its own copy ofnewThreeNodeClusterConfigbecause theclustertest package cannot be imported.Not covered: a node joining for the first time while the cluster runs mixed versions, which needs a fourth device. The App-Tracker check on each node is left out because it needs #6665's App-Tracker on 127.0.0.1, and will be added once both PRs merge.
How to test and validate this PR
Run the test on its own, or as part of
TestUpgradeSuite:It passes when every
node-<name>-upgradedcheckpoint is reached and the app answers after the last upgrade. On a failure, the last checkpoint reached shows which phase timed out and how far the rolling upgrade got.Changelog notes
None, test only.
PR Backports
Checklist
check them.
🤖 Generated with Claude Code