From 20ec8a3cdb94fc1cd63934b69f5eb0a83d4750ea Mon Sep 17 00:00:00 2001 From: Wynne Pirini Date: Wed, 14 Oct 2026 17:23:27 +1300 Subject: [PATCH 1/2] packaging(ns8): fix manifest add arity, drop bad rm flag, and selftest the path Second failure of the main-branch publish, both in push-manifest.sh: - 'buildah manifest add' was passed three arguments (list, image, instance name). Buildah 5.x takes exactly two; it failed with 'too many arguments: expected list and image add to list'. - 'buildah manifest rm --all' does not exist; rm takes no flags. The stale manifest cleanup was silently doing nothing behind '|| true'. Reproduced both locally in a buildah container rather than burning release runs: buildah 5.39.3, 'manifest add list image' yields an OCI image index with amd64 and arm64, and 'manifest rm ' works. Also add an invariant to the script itself: after assembly, inspect the list and refuse to push unless it holds exactly the requested number of arches. An arm64 node receiving an amd64-only image fails at podman run, so a partial manifest is worse than no publish. And add a CI selftest that runs the real push-manifest.sh against scratch images and an oci: destination (no registry, no credentials). PR builds push nothing, so the manifest step is skipped on PRs: that is why two broken versions both reached main. Verified locally that the selftest passes and that the new invariant fires. PUSH_DESTINATION was added as an override so the script can be driven without a registry; it defaults to the docker:// target. --- .github/workflows/ns8-module.yml | 20 ++++++++++++++++++++ packaging/ns8/push-manifest.sh | 23 +++++++++++++++++------ 2 files changed, 37 insertions(+), 6 deletions(-) diff --git a/.github/workflows/ns8-module.yml b/.github/workflows/ns8-module.yml index 76ef09f1..092889a3 100644 --- a/.github/workflows/ns8-module.yml +++ b/.github/workflows/ns8-module.yml @@ -47,6 +47,26 @@ jobs: exit 1 fi done + + # Run the real push-manifest.sh end to end against a scratch storage and + # an oci: destination, so no registry and no credentials are needed. PR + # builds push nothing, which means the manifest step is skipped on PRs and + # a green run says nothing about it. Both earlier publish failures (a bad + # subcommand, then a bad argument count) lived in this step and only + # surfaced on a release push, so exercise it here instead. + # push-manifest.sh refuses to publish a list that is missing an arch, so a + # clean exit is the assertion. + - name: Selftest the multi-arch manifest assembly + run: | + set -euo pipefail + tmp=$(mktemp -d) + buildah from --name ocr-selftest-amd64 --arch amd64 scratch >/dev/null + buildah commit ocr-selftest-amd64 ocr-selftest/ns8-calnode:build-amd64 >/dev/null + buildah from --name ocr-selftest-arm64 --arch arm64 scratch >/dev/null + buildah commit ocr-selftest-arm64 ocr-selftest/ns8-calnode:build-arm64 >/dev/null + REPOBASE=ocr-selftest TARGETARCHES="amd64 arm64" IMAGETAGS=selftest \ + PUSH_DESTINATION="oci:${tmp}/out" bash packaging/ns8/push-manifest.sh + echo "manifest selftest OK" # No build-time placeholders may remain: the unit and configure read # the app ref from ${CALNODE_IMAGE} at install time. ! grep -rn '__CALNODE_APP_IMAGE__' imageroot/ diff --git a/packaging/ns8/push-manifest.sh b/packaging/ns8/push-manifest.sh index f879ca14..b8d3dfd7 100755 --- a/packaging/ns8/push-manifest.sh +++ b/packaging/ns8/push-manifest.sh @@ -18,6 +18,9 @@ # REPOBASE module image namespace, default ghcr.io/calnode # TARGETARCHES space-separated arches to assemble, required # IMAGETAGS space-separated tags to publish, default "latest" +# PUSH_DESTINATION push target override, default docker://. Set to an +# oci: path to exercise the whole script with no registry, +# which is what the CI selftest does. # # Example: # TARGETARCHES="amd64 arm64" IMAGETAGS="1.2.3 1.2 latest" bash push-manifest.sh @@ -48,15 +51,23 @@ done for imagetag in "${imagetags[@]}"; do manifest="${repobase}/${reponame}:${imagetag}" # Drop any stale manifest so a retry cannot inherit old entries. - buildah manifest rm --all "${manifest}" 2>/dev/null || true + buildah manifest rm "${manifest}" 2>/dev/null || true buildah manifest create "${manifest}" for arch in "${arches[@]}"; do - buildah manifest add \ - "${manifest}" \ - "${repobase}/${reponame}:build-${arch}" \ - "docker.io/calnode/ns8-calnode:${imagetag}-${arch}" + # Exactly two arguments: buildah 5.x takes only the list and the + # image, with no instance-name argument. + buildah manifest add "${manifest}" "${repobase}/${reponame}:build-${arch}" done - buildah manifest push --all "${manifest}" "docker://${manifest}" + # Refuse to publish a partial manifest. If any arch silently failed to + # join the list, an arm64 node would get an image it cannot run. + count=$(buildah manifest inspect "${manifest}" | grep -c '"architecture"') + if [ "$count" -ne "${#arches[@]}" ]; then + echo "Refusing to publish ${manifest}: it holds ${count} arches," \ + "expected ${#arches[*]}" >&2 + buildah manifest rm "${manifest}" 2>/dev/null || true + exit 1 + fi + buildah manifest push --all "${manifest}" "${PUSH_DESTINATION:-docker://${manifest}}" buildah manifest rm "${manifest}" echo "Published ${manifest} (${arches[*]})" done \ No newline at end of file From 52ae44b99337c1f9ff9bceebe28a2be462c46db2 Mon Sep 17 00:00:00 2001 From: Wynne Pirini Date: Wed, 14 Oct 2026 17:36:45 +1300 Subject: [PATCH 2/2] packaging(ns8): restore the validate checks to the right step The previous commit's YAML edit spliced the new selftest step into the middle of the existing validate step's script, so the placeholder and exec-bit checks were orphaned into a step that runs from the repo root instead of packaging/ns8. The selftest itself passed on the runner (Published ocr-selftest/ns8-calnode:selftest (amd64 arm64)); the step then failed on the orphaned 'grep imageroot/'. Both sets of checks now sit in the step whose working-directory they need. --- .github/workflows/ns8-module.yml | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/.github/workflows/ns8-module.yml b/.github/workflows/ns8-module.yml index 092889a3..5e83009f 100644 --- a/.github/workflows/ns8-module.yml +++ b/.github/workflows/ns8-module.yml @@ -47,12 +47,26 @@ jobs: exit 1 fi done + # No build-time placeholders may remain: the unit and configure read + # the app ref from ${CALNODE_IMAGE} at install time. + ! grep -rn '__CALNODE_APP_IMAGE__' imageroot/ + # Action scripts must be committed executable: the agent silently + # skips non-executable files, and fileMode=false checkouts do not + # record chmod, so assert the indexed modes, not just the worktree. + # The non-empty assertion matters: the awk below exits 0 on empty + # input, so a pathspec that stops matching would pass vacuously. + scripts=$(git ls-files -s -- imageroot/actions | grep -v 'validate-.*\.json$') + if [ -z "$scripts" ]; then + echo "::error::no action scripts matched, the exec-bit check below would pass vacuously" + exit 1 + fi + echo "$scripts" | awk '$1!="100755" {print "not executable in index:", $4; bad=1} END{exit bad}' # Run the real push-manifest.sh end to end against a scratch storage and # an oci: destination, so no registry and no credentials are needed. PR # builds push nothing, which means the manifest step is skipped on PRs and # a green run says nothing about it. Both earlier publish failures (a bad - # subcommand, then a bad argument count) lived in this step and only + # subcommand, then a bad argument count) lived in that step and only # surfaced on a release push, so exercise it here instead. # push-manifest.sh refuses to publish a list that is missing an arch, so a # clean exit is the assertion. @@ -67,20 +81,6 @@ jobs: REPOBASE=ocr-selftest TARGETARCHES="amd64 arm64" IMAGETAGS=selftest \ PUSH_DESTINATION="oci:${tmp}/out" bash packaging/ns8/push-manifest.sh echo "manifest selftest OK" - # No build-time placeholders may remain: the unit and configure read - # the app ref from ${CALNODE_IMAGE} at install time. - ! grep -rn '__CALNODE_APP_IMAGE__' imageroot/ - # Action scripts must be committed executable: the agent silently - # skips non-executable files, and fileMode=false checkouts do not - # record chmod, so assert the indexed modes, not just the worktree. - # The non-empty assertion matters: the awk below exits 0 on empty - # input, so a pathspec that stops matching would pass vacuously. - scripts=$(git ls-files -s -- imageroot/actions | grep -v 'validate-.*\.json$') - if [ -z "$scripts" ]; then - echo "::error::no action scripts matched, the exec-bit check below would pass vacuously" - exit 1 - fi - echo "$scripts" | awk '$1!="100755" {print "not executable in index:", $4; bad=1} END{exit bad}' build: needs: validate