Skip to content

[14.5-stable] Fix bugs in unit tests - Race when accessing stats of drona request - #118

Merged
rene merged 1 commit into
lf-edge:14.5-stablefrom
christoph-zededa:backport-51-14.5-stable
Oct 1, 2026
Merged

rene merged 1 commit into
lf-edge:14.5-stablefrom
christoph-zededa:backport-51-14.5-stable

Conversation

@christoph-zededa

@christoph-zededa christoph-zededa commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Description

Backport of #51 to 14.5-stable: the one commit of that PR, cherry-picked with
-x. It takes the request's read lock in DronaRequest.GetDoneParts and
locks the stats updates in the S3 and Azure transports, so that a reader of
the transfer statistics no longer races the stats goroutine that writes them.

Why now: EVE's backport of its downloader fixes to this branch (lf-edge/eve#6686)
adds unit tests that run a real HTTP transfer through the transport under the
race detector, and they fail on this branch with GetDoneParts reading
doneParts while the stats goroutine writes it. 16.0-stable and main have
had this fix since August 2025; this branch and 13.4-stable never received it.
The race existed here all along, it was just never exercised under -race.

One conflict in zedUpload/datastore.go: the lock around the write of
doneParts that the commit adds to reqPostSize is already present on this
branch inside recordStats, from the backport of #103, so this branch's
version is kept there; everything else applied cleanly.

How to test and validate this PR

go test -race ./zedUpload/ passes on this branch. With the EVE backport's
drain_test.go vendored against this change the race report disappears.

PR Backports

Checklist

  • I've provided a proper description
  • I've added the proper documentation — none needed
  • I've tested my PR on amd64 device — library change, unit tests with the race detector
  • I've tested my PR on arm64 device — not applicable
  • I've written the test verification instructions
  • I've set the proper labels to this PR — left to the maintainers
  • I've checked the boxes above, or I've provided a good reason why I didn't check them

For backport PRs:

  • I've added a reference link to the original PR
  • PR's title follows the template

🤖 Generated with Claude Code

- Adding calculation of md5checksum if not present in resp.ContentMD5 for Azure metadata

Signed-off-by: Ioannis Sfakianakis <jsfakas@gmail.com>
(cherry picked from commit de07a00)
@rene
rene merged commit 7752b89 into lf-edge:14.5-stable Oct 1, 2026
4 checks passed
christoph-zededa added a commit to christoph-zededa/eve that referenced this pull request Oct 5, 2026
Update eve-libs from 82b21ffd5536 to 7752b891f3f8, the tip of its
14.5-stable branch, which adds two commits: lf-edge/eve-libs#116 and
lf-edge/eve-libs#118, the backports of lf-edge/eve-libs#113 and
lf-edge/eve-libs#51.

DronaRequest.WithContext lets the caller hand the request the context
it selects on itself, so that the transport observes the same
cancellation; WithCancel and Cancel, which always derived a context of
their own, are gone. The downloader fix backported next builds on it.

The race fix makes GetDoneParts take the request's read lock, which the
stats updater already holds for writing when it records the parts, and
has the AWS and Azure transports update the request's size and status
under the lock as well. Pillar runs its unit tests under the race
detector, and the downloader test backported at the end of this series
drives a request through a real transport whose stats updater keeps
writing those fields while the downloader reads them from the progress
messages.

Pillar is the only module on this branch that pins eve-libs; it moves to
the new version, and only the six zedUpload files that carry the two
changes are re-vendored. No transitive dependency moves.

Signed-off-by: Christoph Ostarek <christoph@zededa.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
rene pushed a commit to lf-edge/eve that referenced this pull request Oct 5, 2026
Update eve-libs from 82b21ffd5536 to 7752b891f3f8, the tip of its
14.5-stable branch, which adds two commits: lf-edge/eve-libs#116 and
lf-edge/eve-libs#118, the backports of lf-edge/eve-libs#113 and
lf-edge/eve-libs#51.

DronaRequest.WithContext lets the caller hand the request the context
it selects on itself, so that the transport observes the same
cancellation; WithCancel and Cancel, which always derived a context of
their own, are gone. The downloader fix backported next builds on it.

The race fix makes GetDoneParts take the request's read lock, which the
stats updater already holds for writing when it records the parts, and
has the AWS and Azure transports update the request's size and status
under the lock as well. Pillar runs its unit tests under the race
detector, and the downloader test backported at the end of this series
drives a request through a real transport whose stats updater keeps
writing those fields while the downloader reads them from the progress
messages.

Pillar is the only module on this branch that pins eve-libs; it moves to
the new version, and only the six zedUpload files that carry the two
changes are re-vendored. No transitive dependency moves.

Signed-off-by: Christoph Ostarek <christoph@zededa.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants