Skip to content

TEST Claude PR review fix - #252

Closed
christoph-zededa wants to merge 229 commits into
rene:masterfrom
christoph-zededa:ubuntu
Closed

christoph-zededa wants to merge 229 commits into
rene:masterfrom
christoph-zededa:ubuntu

Conversation

@christoph-zededa

Copy link
Copy Markdown

Description

Provide a clear and concise description of the changes in this PR and
explain why they are necessary.

If the PR contains only one commit, you will see the commit message above:
fill free to use it under the description section here, if it is good enough.

For Backport PRs, a full description is optional, but please clearly state
the original PR number(s). Use the #{NUMBER} format for that, it makes it easier
to handle with the scripts later. For example:

Backport of lf-edge#1234, lf-edge#5678, #91011

Title of a backport PR must also follow the following format:

"[x.y-stable] Original PR title".

where x.y-stable is the name of the target stable branch, and
Original PR title is the title of the original PR.

For example, for a PR that backports a PR with title Fix the nasty bug to
branch 13.4-stable the title should be:
[13.4-stable] Fix the nasty bug.

PR dependencies

List all dependencies of this PR (when applicable, otherwise remove this
section).

How to test and validate this PR

Please describe how the changes in this PR can be validated or verified. For
example:

  • If your PR fixes a bug, outline the steps to confirm the issue is resolved.
  • If your PR introduces a new feature, explain how to test and validate it.

This will be used

  1. to provide test scenarios for the QA team
  2. by a reviewer to validate the changes in this PR.

The first is especially important, so, please make sure to provide as much
detail as possible.

If it's covered by an automated test, please mention it here.

Changelog notes

Text in this section will be used to generate the changelog entry for
release notes. The consumers of this are end users, not developers.
So, provide a clear and short description of what is changed in the PR from
the end user perspective. If it changes only tooling or some internal
implementation, put a note like "No user-facing changes" or "None".

PR Backports

For all current LTS branches, please state explicitly if this PR should be
backported or not. This section is used by our scripts to track the backports,
so, please, do not omit it.

Here is the list of current LTS branches (it should be always up to date):

  • 17.0-stable
  • 16.0-stable
  • 14.5-stable
  • 13.4-stable

For example, if this PR fixes a bug in a feature that was introduced in 17.0,
you can write:

- 17.0-stable: To be backported.
- 16.0-stable: No, as the feature is not available there.
- 14.5-stable: No, as the feature is not available there.
- 13.4-stable: No, as the feature is not available there.

Also, to the PRs that should be backported into any stable branch, please
add a label stable.

Checklist

  • I've provided a proper description
  • I've added the proper documentation
  • I've tested my PR on amd64 device
  • I've tested my PR on arm64 device
  • I've written the test verification instructions
  • I've set the proper labels to this PR

For backport PRs (remove it if it's not a backport):

  • I've added a reference link to the original PR
  • PR's title follows the template

And the last but not least:

  • I've checked the boxes above, or I've provided a good reason why I didn't
    check them.

Please, check the boxes above after submitting the PR in interactive mode.

Comment thread pkg/ubuntu/Dockerfile Resolved
Comment thread pkg/ubuntu/recurse-debs.pl Resolved
Comment thread pkg/ubuntu/Dockerfile Resolved
Comment thread pkg/ubuntu/eve-ubuntu-deploy.sh Resolved
Comment thread pkg/ubuntu-example/Dockerfile Resolved
@codecov-commenter

Copy link
Copy Markdown

Welcome to Codecov 🎉

Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.

Thanks for integrating Codecov - We've got you covered ☂️

FROM ubuntu AS build

ENV PKGS="curl"
ENV BUILD_PKGS="gcc openssh-client"

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@claude, are these packages enough?

@rene

rene commented Aug 25, 2026

Copy link
Copy Markdown
Owner

@claude please review

@github-actions

github-actions Bot commented Aug 25, 2026 •

Copy link
Copy Markdown

Claude finished @rene's task in 2m 2s —— View job


Claude review

  • Read changed files
  • Analyze for bugs, quality, security
  • Answer @rene's question about package sufficiency
  • Post inline review comments

I reviewed the new pkg/ubuntu / pkg/ubuntu-example packages and posted inline comments.

Answer to @rene ("are these packages enough?" on ubuntu-example): Not quite for a working curl:

  • curl only Depends on libcurl4; ca-certificates is a Recommends, and eve-ubuntu-deploy.sh installs with --no-install-recommends, so HTTPS will fail (cert verification).
  • Worse, the chroot's only apt source is the pre-seeded file:///mirror/archives/ mirror, which contains only the transitive Depends closure of the builder's PKGS. ca-certificates isn't in that set, so it can't be installed downstream even if added to PKGS — it has to be seeded in the eve-ubuntu builder first (or recurse-debs.pl taught to follow Recommends). gcc/openssh-client are fine.

Key findings (inline):

  • recurse-debs.pl:18 — re-queries the entire growing @pkgs each pass (O(n²)); should track only newly-discovered packages. Plus the :21 alt-regex \[^ >]+ is malformed, so virtual/alternative deps are silently dropped from the mirror.
  • Dockerfile:46/50-53 — the mirror + locked-down sources.list hard-limits all downstream images to the builder's package closure; document or extend this.
  • Dockerfile:39 — purging bash can break maintainer scripts of later-installed packages that use #!/bin/bash.

Still open from the prior review round (not yet addressed in the latest commit):

  • Dockerfile:34 — --arch=amd64 hardcoded despite multi-arch intent in eve-ubuntu-deploy.sh.
  • Dockerfile:39 — duplicate login.defs (×2, and not a package) and procps (already purged on :37).
  • eve-ubuntu-deploy.sh:5 — header still describes Alpine, not Ubuntu/debootstrap.
  • ubuntu-example/Dockerfile:8-10 — commented-out experiment lines to remove before merge.

register-sbom-pkg.sh looks solid — clear docs, proper validation, and correct use of the distroless status.d convention.

FROM ubuntu AS build

ENV PKGS="curl"
ENV BUILD_PKGS="gcc openssh-client"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rene For a bare curl this is not quite enough in practice, for two reasons:

  1. HTTPS will fail. curl only Depends on libcurl4; ca-certificates is a Recommends. eve-ubuntu-deploy.sh installs with --no-install-recommends --no-install-suggests, so ca-certificates is never pulled in, and any https:// request will fail with a cert-verification error. If the example is meant to demonstrate a working curl, add ca-certificates to PKGS.

  2. It may not even be installable. The chroot's only apt source is file:///mirror/archives/ (see pkg/ubuntu/Dockerfile:50-53), and that mirror only contains the transitive Depends hull of the PKGS list baked into the eve-ubuntu builder image. ca-certificates is not in that list, so it isn't in the mirror and can't be installed here even if you add it to PKGS. To make it work you'd have to add ca-certificates to the builder's PKGS first (or have recurse-debs.pl follow Recommends).

gcc / openssh-client are fine — gcc is covered by build-essential and openssh-client is explicitly in the builder PKGS.

Comment thread pkg/ubuntu/Dockerfile Outdated
RUN rm -Rf /out/usr/share/bash-completion /out/usr/share/fish /out/usr/share/zsh

COPY recurse-debs.pl /root
RUN for i in ${PKGS}; do apt-get -yd install --reinstall -o dir::cache=/mirror $(/root/recurse-debs.pl $i); done

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The mirror is populated only with the transitive Depends closure of the builder-image PKGS (via recurse-debs.pl, which follows Depends only — not Recommends). Combined with sources.list being rewritten to only file:///mirror/archives/ (lines 50-53), this hard-limits downstream images (e.g. ubuntu-example) to exactly this package set. Two consequences worth documenting or fixing:

  • Packages needed by consumers but missing here (e.g. ca-certificates) cannot be installed at all.
  • The download here runs against the host builder's apt (ubuntu:26.04), while installs happen in the resolute chroot. This only works because 26.04 == resolute; if they ever diverge, resolved versions won't match what the chroot expects.

Consider either documenting this constraint prominently or extending recurse-debs.pl to follow Recommends for commonly-needed runtime deps.

Comment thread pkg/ubuntu/recurse-debs.pl Outdated
my $is_added=1;
while ($is_added) {
$is_added=0;
open(my $dh, "-|", 'apt-cache', 'depends', @pkgs)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Each iteration re-runs apt-cache depends on the entire accumulated @pkgs list rather than only the packages newly discovered in the previous round. It still terminates (thanks to %seen), but it re-resolves the whole growing set every pass — O(n²) work. Track a separate @todo/frontier of newly-added packages and query only those:

my @todo = @pkgs;
while (@todo) {
    open(my $dh, "-|", 'apt-cache', 'depends', @todo) or die ...;
    @todo = ();
    while (<$dh>) {
        if (m/^\s+(?:Pre)?Depends:\s+([^\s<]\S*)/) {
            next if $seen{$1}++;
            push @pkgs, $1;
            push @todo, $1;
        }
    }
    close($dh) or die ...;
}

Also note the second alternative regex m/^<?(\[^ >]+)/ (line 21) is malformed — \[ matches a literal [, so virtual/alternative dependencies are silently dropped and may be missing from the mirror.

Comment thread pkg/ubuntu/Dockerfile Outdated
RUN chroot /out apt-get update
RUN chroot /out apt-get purge --allow-remove-essential -y procps
# findutils sed libbsd0 sensible-utils are still needed to install further packages
RUN chroot /out apt-get purge --allow-remove-essential -y gawk login login.defs bsdutils readline-common libsemanage2 mount bash gzip e2fsprogs login.defs ncurses-base ncurses-bin libext2fs2t64 sysvinit-utils procps logsave passwd

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Purging bash from the chroot (in addition to the duplicate login.defs/procps noise already noted) is risky: Debian/Ubuntu maintainer scripts (postinst/preinst) of packages installed later by consumers frequently start with #!/bin/bash. With bash removed and only dash as /bin/sh, those installs can fail at configure time. Worth verifying that every package a consumer might install (curl, openssh-*, etc.) has POSIX-sh-compatible maintainer scripts, or keep bash in the chroot.

rene and others added 5 commits October 12, 2026 11:12
Replace the hadolint shipped with the Yetus image (2.12.0) by the
upstream v2.15.1 release. Release assets are named after x86_64/arm64
rather than TARGETARCH, so both binaries are fetched (with checksum
verification) and the one matching the target architecture is
installed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Renê de Souza Pinto <rene@renesp.com.br>
When every layer of an application's image fails to download from every
management port, nothing bounds the error on its way up: the downloader
appends the failure of each source address, volumemgr joins the errors of
all blobs and prefixes them into the volume's error, and zedmanager embeds
that twice in the AppInstanceStatus. Once the status exceeds the 64 KiB a
pubsub message may carry, the socket driver's Fatalf kills zedbox and the
watchdog reboots the node, as it did on a customer node with an 8-layer
image, four management addresses and an unresolvable registry hostname.

TestLayerDownloadErrorsBounded (apps suite, TwoMgmtPorts) deploys an app
whose 24-layer image is published into evetest's own registry without its
layer blobs, so EVE resolves the tag and fetches the manifest, then fails
every layer on both ports with BLOB_UNKNOWN. It expects the app to be
reported in error with a bounded description while the device stays up,
proven by a metrics heartbeat after the last layer failed, and stops at
once on an unexpected reboot or a fatal log. The sizes are measured: with
two ports, 24 layers put the unbounded status a third over the limit while
the content tree's and the volume's own statuses stay a quarter under it,
so the unfixed code fails in zedmanager as the field node did. The port
retest timer is shortened to a minute because verifying the two-port
configuration leaves a stale error on a port that nim only clears at the
next retest.

Framework additions: PushImageWithMissingLayersToLocalRegistry pushes an
image's manifest and config without its layer blobs, and
EdgeDevice.UnexpectedRebootCount exposes the reboot audit Close already
performs, so a test waiting for something a crash would prevent can fail
immediately with the reboot reason.

On unfixed EVE the test stops within minutes with "Too large message
(72310 bytes) sent to zedmanager/AppInstanceStatus" as the reboot reason;
with MISSING_LAYERS=4, where nothing overflows, it passes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Christoph Ostarek <christoph@zededa.com>
When every layer of an application's image failed to download from every
management port, the error grew without bound on its way up: the
downloader appended the failure of each source address, volumemgr joined
the errors of all blobs and prefixed them into the volume's error, and
zedmanager embedded that twice in the AppInstanceStatus, once per volume
reference and once more as the app-level error. Past the 64 KiB a pubsub
message may carry, the socket driver's Fatalf killed zedbox and the
watchdog rebooted the node; an 8-layer image, four management addresses
and a registry hostname that did not resolve were enough.

Bound the error at the three levels where it is built, and make the
publish safe regardless:

- downloader: collect the failure of each source address in addrErrors,
  which reports the same failure once with the addresses it came from and
  keeps the report within 1 KiB, counting what it leaves out.
- volumemgr: join the blob errors of a content tree with JoinErrors within
  2 KiB; which blobs failed is in the error entities regardless.
- zedmanager: the app-level error is the bounded join of the volume errors
  (MaxErrorLen) rather than a full copy of each, since every volume's own
  error already travels in VolumeRefStatusList.
- types: SetErrorDescription cuts any error down to MaxErrorLen (4 KiB)
  with TruncateError, which keeps the head and the tail of the string
  around a note of how many bytes were dropped, so no status type can
  carry an unbounded error whoever produces it.
- The publishers of AppInstanceStatus, VolumeStatus, VolumeRefStatus and
  ContentTreeStatus check the serialized size with CheckMaxSize when the
  status carries an error, and cut the errors down to
  OversizedStatusErrorLen rather than publish what the socket driver
  would reject; the fatal remains for anything else that is oversized.

The pubsub limit itself is left alone: the subscriber reads one message
per socket read into a fixed buffer, and a silently dropped status would
leave agents stale, which is worse for manageability than a visible crash.

Unit tests cover the helpers, the collector and the guard; the full pillar
suite passes. TestLayerDownloadErrorsBounded, which reproduced the crash
on the unfixed code, passes on a build with this change: the application
reports a 1476-byte error for its 24 failed layers and the device stays
up.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Christoph Ostarek <christoph@zededa.com>
zizmor reports template-injection in rerun-ci.yml and
request_codeowners_review.yml. Pass the event fields to the scripts
through step env instead of expanding them in run blocks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: eriknordmark <erik@zededa.com>
The CODEOWNERS review request read .github/CODEOWNERS from the PR
head instead of the base branch, so it was not using the correct
CODEOWNERS file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: eriknordmark <erik@zededa.com>
eriknordmark and others added 26 commits October 27, 2026 15:27
Building eve-cross-compilers from source takes 4h15m to 4h43m on the
amd64 runners, so a master run that rebuilds it leaves the rest of
pkgs-amd64 and coverage under two hours of the default 360-minute job
limit, and those jobs have been cancelled at 6h. Give them the same
1440 minutes as pkgs-arm64 and the eve jobs.

Signed-off-by: eriknordmark <erik@zededa.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Move cross-compilers to the eve-alpine built from the signed
eve-alpine-base, and the packages built with it to its new tag. Its
current base, an older eve-alpine, carries no signature, so the next
rebuild of cross-compilers would fail the base-image check once
EVE_COSIGN_ENFORCE is set.

Signed-off-by: eriknordmark <erik@zededa.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TestWalkUSBPorts compares ghw's USB device lookup against a reference
implementation that uses extractPCIaddress. Its regex captured the
first PCI address after pci0000:00/, which is the root port when the
USB controller sits behind a PCIe bridge, whereas ghw walks the sysfs
path upwards and reports the controller itself. On a runner with such
a topology (xHCI at 0000:02:00.0 behind 0000:00:02.1) every Go Tests
job failed deterministically.

Capture the last PCI address in the path instead, i.e. the PCI device
closest to the USB root hub. This matches ghw's FindPCIAddress and also
covers controllers attached to a PCI device through intermediate
platform devices (dwc3-pci). extractPCIaddress has no callers outside
the test, so production behaviour is unchanged.

Fixes lf-edge#6938

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Christoph Ostarek <christoph@zededa.com>
FindDeviceHostingApp returned the host named by the first device whose
stored cluster info mentioned the app. That report can be stale: only
the stats leader publishes cluster info, and a node that loses the
lease stops publishing without retracting its last report. A report can
also list the VMI being replaced next to its successor.

FindDeviceHostingApp now trusts only the newest stored report across
the cluster's devices, then waits for new reports as before. Finished
VMIs are not counted.

Add EdgeCluster.WaitUntilAppIsHostedOn for checks after an app moves:
it waits until the newest report places the app on the given node and
on no other.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Andrew Durbin <andrewd@zededa.com>
TestDNIDandBackupDNID failed after vm2 failed back to edge-dev3: the
cluster ran vm2 on edge-dev3, but the placement check read a stored
cluster info report that still placed it on edge-dev1, its backup node.
The check after activation on the backup node has the same race.

Use WaitUntilAppIsHostedOn for both checks, and point the HTTP
readiness check at the device it returns. On the backup node, expect
edge-dev1 itself: edge-dev2 is the tie-breaker and edge-dev3 is down.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Andrew Durbin <andrewd@zededa.com>
Adds evetest-pr.yml (per-PR run, against a freshly built-but-
unpublished PR image, gated on its build having succeeded for that
exact commit AND either the PR being approved or a CODEOWNER having
requested a run with an "/evetest" comment -- checked live by the PR
Gate, since a new build, a new review or a new comment can come first)
and evetest-nightly.yml (full run against a branch's tip, on a schedule/
manual dispatch, against whatever publish.yml already published for
that commit -- the full-coverage safety net for evetest-pr.yml's own
PR-time test skipping).

Shared per-suite execution lives in the evetest-run-suite composite
action: checkout/image loading, running the suite, formatting/
uploading results, and (when a suite fails) an EVETEST_PAUSE_ON_FAILURE
live investigation -- live-investigation-watcher.sh polls the suite
over evetest's own gRPC control plane and has Claude inspect the
still-running device/SDN directly (status, eve ssh/logs, collect-info,
sdn ssh) rather than post-mortem from collected artifacts, which often
can't find a root cause. Findings land in one claude-analysis.md per
suite, posted as a PR comment and prepended into a nightly failure
ledger (published alongside an Allure report on GitHub Pages) so a
recurring failure is recognized as such instead of re-investigated
from scratch every time.

On a PR, Claude is also asked up front which suites/tests can be
skipped as clearly unrelated or redundant with another already-kept
test, given the diff and each suite's own doc comments -- skipped
suites still get their own (immediately-resolved) status check rather
than never appearing at all.

A CODEOWNER controls what runs with a PR comment (rerun-ci.yml re-runs
the PR Gate, which reads the comment and tells evetest-pr.yml):
  /evetest               - run now, with Claude deciding what to skip
  /evetest all           - run every suite
  /evetest <name> ...    - run only the named suites/tests/variants
                           (select-tests.sh turns names into the
                           matrix and an EVETEST_SKIP list)

Signed-off-by: Milan Lenco <milan@zededa.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
New collect-coverage workflow_dispatch input on evetest-nightly.yml:
when set, builds a COVER=y EVE image on the builder fleet (COVER=y
changes pillar's own package build, so it's a separate image, not a
toggle on the one publish.yml already published) and has every suite
collect coverage (EVETEST_COLLECT_COVERAGE), merging it all into one
overall per-package and total summary alongside each suite's own.

The line-by-line HTML report (source + coverage highlighting) is
published to gh-pages next to the Allure report; the raw merged
covdata is also uploaded as its own artifact for combining with other
coverage locally (see "make coverage-merge"). Left off by default,
since it rebuilds the image instead of pulling the published one.

Signed-off-by: Milan Lenco <milan@zededa.com>
Add trigger-evetest-nightly.sh, which dispatches evetest-nightly.yml via
the GitHub API so it can be driven by any external scheduler with
Internet access, instead of relying on GitHub's best-effort schedule
trigger.

The target repository and branch are mandatory arguments, and the token
is read from the GH_TOKEN environment variable rather than stored in the
script.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Milan Lenco <milan@zededa.com>
Add --registration-token, so a runner can be installed by someone who
was handed a short-lived registration token (from the GitHub UI or API)
instead of a PAT with repository Administration permission. --gh-pat is
now required only when no registration token is given.

With --destroy, --gh-pat becomes optional: without it the VM and its
cloud-init snippet are removed, but the runner stays registered on
GitHub and has to be removed there manually.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Milan Lenco <milan@zededa.com>
Name the GitHub runner exactly as given by --name, and the VM after it
by default, instead of prefixing both with "gh-runner-". A new optional
--vm-name sets the Proxmox VM name separately; pass it again with
--destroy. The cloud-init snippet file keeps its prefix so it cannot
clash with other snippets.

Without the prefix, a VM name could match the evetest broker VM or its
per-test "ev-*" VMs, which --destroy would then remove, so reject those
names.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Milan Lenco <milan@zededa.com>
A publish job that signs what it pushed fails when Docker Hub's
referrers listing has not shown a new signature within a minute, and
the packages it has not signed yet stay unsigned until that run is
re-run. Every later build that pulls them reports an unsigned package,
which fails once verification is enforced. A master publish run hit
this on an arm64 package whose signature did become visible later, so
re-check for up to five minutes before failing.

Signed-off-by: eriknordmark <erik@zededa.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Rekor outage longer than cosign's own two attempts failed package
signing on master: publish run 40594931115 pushed 41 amd64 packages,
got a 502 from Rekor on the second one, and left the remaining 40 on
Docker Hub unsigned. Package signing in eve-cosign.sh and the image,
attestation and blob signing in release-sign.yml now retry every 30 s,
five attempts in all, before failing the job. Each attempt signs with
new ephemeral keys, so a Rekor entry from a failed attempt goes unused.

Signed-off-by: eriknordmark <erik@zededa.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zededa/ghw's eve/v0.26.0 branch now reports, for each online logical
processor, its package, core, NUMA node, core type (performance,
efficiency or uniform) and frequencies; every cache instance with the
logical processors sharing it; the offline logical processors; and the
CPUs local to each PCI device. Pillar's CPU placement and hardware
inventory will use them in place of reading sysfs themselves.

The CPU inventory pillar reports today is unchanged on every ghw
snapshot, x86 and arm64. It changes only on boards whose device tree
numbers cores per cluster inside one package: there ghw used to merge
the cores of different clusters that share a core_id, and now reports
each core.

Signed-off-by: Mikhail Malyshev <mike.malyshev@gmail.com>
The TPM test step in Go Tests writes coverage profiles such as
evetpm.coverage.txt and swtpm-vtpm.coverage.txt into pkg/pillar and
pkg/vtpm. linuxkit treats untracked files in a package directory as
local changes, so the later make test computes -dirty tags for
eve-pillar and eve-vtpm. Docker Hub never has a dirty tag, so every
run rebuilds eve-vtpm from source even when pkg/vtpm is unchanged.
The pattern goes in the root .gitignore so no package hash changes.

Signed-off-by: eriknordmark <erik@zededa.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bumps [github.com/go-jose/go-jose/v4](https://github.com/go-jose/go-jose) from 4.1.4 to 4.1.6.
- [Release notes](https://github.com/go-jose/go-jose/releases)
- [Commits](go-jose/go-jose@v4.1.4...v4.1.6)

---
updated-dependencies:
- dependency-name: github.com/go-jose/go-jose/v4
  dependency-version: 4.1.6
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
A package job that pushed and then failed to sign left the packages
unsigned for good: a re-run found them on Docker Hub and skipped them,
and signed only those that happened to land in its linuxkit cache.

Each job now records the packages it built, as a per-attempt artifact,
before pushing them, and signs what every attempt's record lists, so a
re-run signs what an earlier attempt pushed. The manifest job then
verifies every package manifest and index the run pushed or used, and
the bases of those packages, failing the run on any unsigned one
regardless of EVE_COSIGN_ENFORCE. On a release tag that stops the
release.

Signed-off-by: eriknordmark <erik@zededa.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Publish run 40594931115 pushed 7 amd64 packages and failed to sign them
on a Rekor outage, before re-runs signed what earlier attempts pushed,
and an older run left the eve-node-exporter index unsigned. Without
their signatures the new audit fails every build that uses them.

The manifest job now signs the indexes listed in the temporary
tools/eve-cosign-backfill, and their platform manifests, skipping with a
warning a tag that no longer resolves or names the listed index. A
later PR deletes the list, its step and the sign-backfill command once
master has signed them.

Signed-off-by: eriknordmark <erik@zededa.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
eve-rust 1.93.1-2 is the first eve-rust release signed with keyless
cosign. It is built from the same Dockerfile with Rust 1.93.1 as
1.93.1-1, six months later, so its Alpine packages may be newer. Build
the installer, monitor and vector packages, and the Rust tests, on it.

Signed-off-by: eriknordmark <erik@zededa.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The steps that publish Eden results as commit statuses call the GitHub
API with no retries, so a single 5xx fails the job. When the per-suite
render fails, the previous attempt's statuses stay on the PR, so a
suite that passed on re-run still shows failure. The PR then stays red
until someone re-runs the job by hand. On one run with several attempts,
the jobs listing returned 502 after about 10 s, twice in a row.

Retry each status step up to three times. Fetch the job list in pages
of 50, so each request is lighter and a run with more than 100 jobs is
not silently truncated.

Signed-off-by: eriknordmark <erik@zededa.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Dockerfile scanner treated a comment line ending in the escape
character as the start of a continuation, so it folded the next
instruction into the comment and dropped it. buildkit strips comment
lines before it looks for a continuation, so that instruction is
built. In the tree this hides pkg/qemu's COPY --from=uefi-build; no
FROM is affected, so check-docker-hashes-consistency reports the same.

Signed-off-by: eriknordmark <erik@zededa.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A native build of pillar, qemu or bsp-imx made Make build
eve-cross-compilers first, although buildkit never builds the stage
that uses it. When that image is not on Docker Hub for the host arch,
Go Tests and Fuzz Tests compile the gcc cross toolchains for hours.

get-deps now follows the FROM, COPY --from and RUN --mount from=
references from the last stage, for the target arch on the build
host's arch, and lists only the images those stages use. When a
stage selection does not resolve, as for riscv64 pillar, it keeps
every stage. The deps file is per host and target arch, and covers
the host arch too for packages built for both.

Signed-off-by: eriknordmark <erik@zededa.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When a push changes a package that pillar depends on, the coverage
job's make test finds the new tags missing from the registry and
rebuilds pillar's dependency chain from source, because it starts
alongside pkgs-amd64, which is what pushes them. A cross-compilers
change made the job take 7.6 hours, duplicating pkgs-amd64's build on a
second runner, and on a branch without a raised timeout it hits the
6-hour default.

Wait for pkgs-amd64 so make test pulls what it pushed. Run even when
pkgs-amd64 failed, since that can happen after its push (a signing
error); in the worst case the job rebuilds as before.

Signed-off-by: eriknordmark <erik@zededa.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A remote broker pulls the SDN image itself, so an SDN image built from
an unmerged change was unavailable to it and every test failed.

When SetupDevices can neither find nor pull the SDN image, the broker
still fails the request with the pull error, now as a FailedPrecondition
status carrying an ErrorInfo detail with a dedicated reason. A harness
that recognizes it uploads its local copy through the new
PushSDNContainerImage RPC and repeats the request, as it already does
for the EVE container image. An older harness just fails with the pull
error. The broker-side receive logic and the harness-side upload loop
are shared between the two image types.

make evetest now ensures the local SDN image also with a remote broker.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Milan Lenco <milan@zededa.com>
Init takes testM before checking EVETEST_SKIP for a suite subtest, but
the skip path called t.Skipf without releasing it. Skipf ends the
goroutine via runtime.Goexit, leaving testM held, so the next
testM.Lock() in RunTestSuite blocked forever and the run hung.

Unlock testM before skipping, like every other exit from this branch.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Milan Lenco <milan@zededa.com>
Allow the nightly trigger to set the collect-coverage workflow input,
so a cron can enable coverage collection on selected days only.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Milan Lenco <milan@zededa.com>
lfedge/eve-ubuntu is the Ubuntu 26.04 counterpart of lfedge/eve-alpine:
a builder stage with a minimal /out rootfs and a local apt mirror under
/mirror, which consumers bind-mount into their build, so they can
install debs with network: no and get the same package set on every
rebuild.

The image is put together as follows:

- Resolve the builder, the debootstrap run and the /out chroot against
  snapshot.ubuntu.com/ubuntu/${UBUNTU_SNAPSHOT} instead of the moving
  archive: the same timestamp yields the same package set on every
  rebuild, and /out and /mirror are resolved against the same archive
  state. All three pockets (release/-updates/-security) are pinned in
  the builder because the base image already contains newer packages
  from -updates/-security than the frozen release pocket. The snapshot
  must not be older than the base image either: apt refuses to install
  perl (and thus dpkg-dev) on top of a newer preinstalled perl-base.
  ca-certificates must be installed first because the snapshot service
  is https-only.
- Bootstrap /out (minbase) from the pinned mirror. debootstrap can only
  draw from the release pocket, so afterwards point /out at all three
  pinned pockets and dist-upgrade to reach the snapshot's current
  state. ca-certificates is --include'd so apt inside /out can talk to
  the https-only snapshot until its sources are switched to the local
  mirror. Retry the bootstrap up to 50 times: the snapshot service
  intermittently answers 5xx and debootstrap, unlike apt, does not
  retry failed downloads; --cache-dir carries already-fetched debs
  over to the next attempt.
- Slim /out down: purge packages EVE containers do not need (procps,
  bash, login, e2fsprogs, ncurses, ...) and drop locales, man pages,
  docs and shell completions.
- Fill /mirror/archives with the transitive dependency closure
  (recurse-debs.pl, via apt-cache depends) of a fixed set of build
  tools (build-essential, cmake, git, gdb, curl, openssh, ...), index
  it with dpkg-scanpackages and switch both the builder and /out to
  that file:// mirror.

Two helper scripts are shipped in /bin for consumers:

- eve-ubuntu-deploy.sh, the counterpart of eve-alpine-deploy.sh:
  installs BUILD_PKGS into the builder and PKGS into /out, with
  per-architecture variants of both.
- register-sbom-pkg.sh: records a source-built package in
  /out/var/lib/dpkg/status.d plus a copyright file, so syft lists it
  in the SBOM while dpkg and apt never read the entry.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Christoph Ostarek <christoph@zededa.com>
A minimal consumer of lfedge/eve-ubuntu: it bind-mounts the /mirror
of the base image into /out/mirror and runs eve-ubuntu-deploy.sh with
BUILD_PKGS for the builder stage and PKGS for the /out rootfs, then
copies /out into a FROM scratch image. The package builds with
network: no, so every deb must come from the pinned local mirror.

The README shows how to export the resulting image and inspect its
SBOM with syft.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Christoph Ostarek <christoph@zededa.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.