Repository navigation
Conversation
Fix app instance getting permanently stuck in HALTING state with PurgeInprogress=BringDown during a purge operation. The purge state machine has the following transitions: DownloadAndVerify → BringDown → RecreateVolumes → BringUp The BringDown → RecreateVolumes transition was only implemented in removeAIStatus(), which is called from updateAIStatusUUID() when it observes PurgeInprogress==BringDown. However, doUpdate() also drives the DownloadAndVerify → BringDown transition and calls doRemove() inline. When doRemove() completes synchronously (no domain or network to tear down), doUpdate() falls through with PurgeInprogress still set to BringDown. If no subsequent event triggers updateAIStatusUUID() (e.g. no DomainStatus, AppNetworkStatus, or VolumeRefStatus changes), removeAIStatus() is never called and the state machine stalls. This happens in practice when: 1. A purge counter mismatch is detected on reboot 2. handleCreateAppInstanceStatus calls doUpdate directly 3. The low-priority app boot path (checkLowPriorityApps) calls doUpdate 4. doRemove completes instantly because nothing is running 5. Nothing triggers another status re-evaluation After doRemove() completes, drive the state machine forward to RecreateVolumes and call purgeCmdDone() to persist the purge counter. This ensures the purge completes regardless of which code path invoked doUpdate(), and prevents the purge from being re-detected on subsequent reboots. Signed-off-by: Mikhail Malyshev <mike.malyshev@gmail.com> (cherry picked from commit b446019) Signed-off-by: Milan Lenco <milan@zededa.com>
When a purge is triggered for an app that never started (e.g. its image failed to download), doUpdate transitions PurgeInprogress from DownloadAndVerify to BringDown and calls doRemove. Because no domain was ever created, doRemove returns done=true immediately. Without this fix the code fell through with PurgeInprogress still set to BringDown, skipped the RecreateVolumes check, and entered doPrepare/doActivate without ever calling purgeCmdDone. This left the app stuck at state LOADED with VerifyOnly=true on the VolumeRefConfig. During DownloadAndVerify, doInstall publishes VolumeRefConfig with VerifyOnly=true so that volumemgr downloads and verifies the image without yet creating the volume. The transition to VerifyOnly=false is only made by doInstall when PurgeInprogress is NotInprogress or RecreateVolumes. Because the RecreateVolumes→BringUp transition fired in the same doUpdate call before doInstall ran with RecreateVolumes, VerifyOnly was never cleared, volumemgr never created the volume, and the app remained stuck at LOADED indefinitely. Fix by using BringDown only as an internal sentinel for doRemove/doInactivate (so it treats the operation as a purge rather than an uninstall). If doRemove completes immediately there is nothing to halt, so skip persisting BringDown and transition directly to RecreateVolumes, calling purgeCmdDone to save the purge counter and clean up stale volume refs. Then re-run doInstall with RecreateVolumes so that VerifyOnly is cleared and volumemgr is told to create the volume before the RecreateVolumes→BringUp transition fires. HALTING is now only recorded when a domain is genuinely being waited on. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: eriknordmark <erik@zededa.com> (cherry picked from commit a1582bb) Signed-off-by: Milan Lenco <milan@zededa.com>
This was referenced Oct 18, 2026
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## 16.0-stable #6847 +/- ##
===============================================
- Coverage 29.12% 26.89% -2.24%
===============================================
Files 10 25 +15
Lines 1133 4332 +3199
===============================================
+ Hits 330 1165 +835
- Misses 768 2938 +2170
- Partials 35 229 +194 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
rene
approved these changes
Oct 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Backport of #5742 to
16.0-stable.Clean cherry-pick with
-xofa1582bb40f, no conflicts, but it applies onlyon top of the backport of #5618.
It is one of three zedmanager purge fixes (together with the backports of
#5618 and #6257) that
TestVMAppPurgeAfterPowerCycleneeds.PR dependencies
The backport of #5618 (#6846). The first commit of this PR is that backport;
it will drop out when that PR is merged and this one is rebased.
How to test and validate this PR
Covered by the evetest test
TestVMAppPurgeAfterPowerCycle(added to16.0-stableby #6842):The test passes only with this PR together with the backports of #5618 (#6846) and #6257 (#6848).
Changelog notes
Fixed application purge getting stuck when the application was never activated.
Checklist
For backport PRs (remove it if it's not a backport):
And the last but not least:
check them.