Skip to content

[14.5-stable] zedmanager: fix purge stuck when app was never activated - #6875

Merged
eriknordmark merged 2 commits into
lf-edge:14.5-stablefrom
milan-zededa:14.5-backport-5742-purge-never-activated
Oct 26, 2026
Merged

eriknordmark merged 2 commits into
lf-edge:14.5-stablefrom
milan-zededa:14.5-backport-5742-purge-never-activated

Conversation

@milan-zededa

@milan-zededa milan-zededa commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Description

Backport of #5742 to 14.5-stable.

Cherry-picked with -x from a1582bb40f. The cherry-pick is clean on top of the backport of #6874 (the code it changes is introduced there); on its own it conflicts in zedmanager/updatestatus.go.

Without this fix TestPurgeNeverActivatedApp fails: the purge of an application that was never activated leaves it stuck in LOADED.

PR dependencies

Depends on #6874 (backport of #5618; this branch contains its commit until it is merged).

How to test and validate this PR

Covered by the evetest test TestPurgeNeverActivatedApp (added to 14.5-stable by #6873):

EVETEST_EVE_VERSION=<14.5-stable build with this PR> make evetest NAME=TestPurgeNeverActivatedApp

Changelog notes

Fixed a purge of an application that was never activated (e.g. its image failed to download) getting stuck.

Checklist

  • I've provided a proper description
  • I've added the proper documentation
  • I've tested my PR on amd64 device
  • I've tested my PR on arm64 device
  • I've written the test verification instructions
  • I've set the proper labels to this PR

For backport PRs (remove it if it's not a backport):

  • I've added a reference link to the original PR
  • PR's title follows the template

And the last but not least:

  • I've checked the boxes above, or I've provided a good reason why I didn't
    check them.

rucoder and others added 2 commits October 22, 2026 16:16
Fix app instance getting permanently stuck in HALTING state with
PurgeInprogress=BringDown during a purge operation.

The purge state machine has the following transitions:

  DownloadAndVerify → BringDown → RecreateVolumes → BringUp

The BringDown → RecreateVolumes transition was only implemented in
removeAIStatus(), which is called from updateAIStatusUUID() when it
observes PurgeInprogress==BringDown. However, doUpdate() also drives
the DownloadAndVerify → BringDown transition and calls doRemove()
inline. When doRemove() completes synchronously (no domain or network
to tear down), doUpdate() falls through with PurgeInprogress still set
to BringDown. If no subsequent event triggers updateAIStatusUUID()
(e.g. no DomainStatus, AppNetworkStatus, or VolumeRefStatus changes),
removeAIStatus() is never called and the state machine stalls.

This happens in practice when:
1. A purge counter mismatch is detected on reboot
2. handleCreateAppInstanceStatus calls doUpdate directly
3. The low-priority app boot path (checkLowPriorityApps) calls doUpdate
4. doRemove completes instantly because nothing is running
5. Nothing triggers another status re-evaluation

After doRemove() completes, drive the state machine forward to
RecreateVolumes and call purgeCmdDone() to persist the purge counter.
This ensures the purge completes regardless of which code path invoked
doUpdate(), and prevents the purge from being re-detected on subsequent
reboots.

Signed-off-by: Mikhail Malyshev <mike.malyshev@gmail.com>
(cherry picked from commit d66fc44)
When a purge is triggered for an app that never started (e.g. its image
failed to download), doUpdate transitions PurgeInprogress from
DownloadAndVerify to BringDown and calls doRemove.  Because no domain
was ever created, doRemove returns done=true immediately.  Without this
fix the code fell through with PurgeInprogress still set to BringDown,
skipped the RecreateVolumes check, and entered doPrepare/doActivate
without ever calling purgeCmdDone.

This left the app stuck at state LOADED with VerifyOnly=true on the
VolumeRefConfig.  During DownloadAndVerify, doInstall publishes
VolumeRefConfig with VerifyOnly=true so that volumemgr downloads and
verifies the image without yet creating the volume.  The transition to
VerifyOnly=false is only made by doInstall when PurgeInprogress is
NotInprogress or RecreateVolumes.  Because the RecreateVolumes→BringUp
transition fired in the same doUpdate call before doInstall ran with
RecreateVolumes, VerifyOnly was never cleared, volumemgr never created
the volume, and the app remained stuck at LOADED indefinitely.

Fix by using BringDown only as an internal sentinel for
doRemove/doInactivate (so it treats the operation as a purge rather than
an uninstall).  If doRemove completes immediately there is nothing to
halt, so skip persisting BringDown and transition directly to
RecreateVolumes, calling purgeCmdDone to save the purge counter and
clean up stale volume refs.  Then re-run doInstall with RecreateVolumes
so that VerifyOnly is cleared and volumemgr is told to create the
volume before the RecreateVolumes→BringUp transition fires.  HALTING is
now only recorded when a domain is genuinely being waited on.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: eriknordmark <erik@zededa.com>
(cherry picked from commit a1582bb)
@milan-zededa
milan-zededa force-pushed the 14.5-backport-5742-purge-never-activated branch from 954efb3 to b544771 Compare October 22, 2026 14:16
@eriknordmark
eriknordmark merged commit a784085 into lf-edge:14.5-stable Oct 26, 2026
56 of 65 checks passed
@eriknordmark eriknordmark mentioned this pull request Oct 26, 2026
3 of 7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants