ci: version resolute-qcom-devel builds by distance from their Canonical base - #108
ci: version resolute-qcom-devel builds by distance from their Canonical base#108Bjordis Collaku (bjordiscollaku) wants to merge 3 commits into
Conversation
Pre-merge validation only, for PR #108. The permanent value is main, matching the docker-pkg-build checkout convention; this override lets the actual new script run in a real dispatch before merge, mirroring the same temporary-dispatch-input pattern already used for controlled distro validation testing. Will be reverted before this PR is marked ready for review. Signed-off-by: Bjordis Collaku <bcollaku@qti.qualcomm.com>
Pre-merge validation only, for PR #108. Permanent value is main, matching the docker-pkg-build checkout convention; this override lets the new script run in a real dispatch before merge. Revert before marking this PR ready for review. Signed-off-by: Bjordis Collaku <bcollaku@qti.qualcomm.com>
0c02136 to
cb3f30a
Compare
Pre-merge validation only, for PR #108. Permanent value is main, matching the docker-pkg-build checkout convention; this override lets the new script run in a real dispatch before merge. Revert before marking this PR ready for review. Signed-off-by: Bjordis Collaku <bcollaku@qti.qualcomm.com>
cb3f30a to
fbe974a
Compare
1f146b9 to
47c132b
Compare
|
one suggestion : instead of using .g can we have . truncated to 7 chars, so that user may not confuse to search commit sha with ‘g’ as prepend. Thanks. +qcom47.e2d8ff8 |
|
Phanendra Yarra (@phanendra-qc) Thanks for the suggestion. I agree that removing the I propose using I would prefer to retain 12 SHA characters rather than reduce it to 7. A shortened SHA remains unique only while no other Git object shares the same prefix. Given the size and continued growth of the kernel repository, 12 characters provide stronger long-term traceability while remaining concise. For additional context, with the proposed format, The resulting values on a running target would be:
The on-target output shared earlier confirms that the suffix is present in
I will update the PR with the proposed format so we can generate a build and review the resulting package and on-target version strings. |
Packages built from resolute-qcom-devel currently carry the same
version as whatever Canonical tag they were last rebased onto,
regardless of how many Qualcomm commits have landed on top since. A
build with five Qualcomm patches and a build with eight are versioned
identically, so the version string alone cannot distinguish them, and
apt/dpkg have no way to see a newer patched build as an upgrade over
an older one.
- Adds scripts/apply-local-version-suffix.sh, which appends
+qcom<N>.g<sha> to the topmost debian/changelog entry, where N is
the commit count past the last Canonical sync tag and <sha> is the
short commit hash. N sorts numerically ahead of the SHA, so ordering
between successive builds is always correct, and the leading + sorts
above the bare Canonical version per Debian Policy's convention for
local modifications.
- Only applies to resolute-qcom-devel builds with commits past the
last sync tag. Skipped for resolute-qcom mirror builds and exact
kernel_version-pinned builds, which must stay byte-identical to the
real Canonical upload, and skipped when HEAD is exactly a sync tag.
- Finds the nearest Ubuntu-qcom-* tag through the GitHub Compare API
rather than local git tags: the checkout is shallow and never
carries tag refs, deepening commit history alone does not fetch
tags, and this repository mirrors Ubuntu's full upstream history, so
a blanket "git fetch --tags" is unsafe on every build. Verified
empirically: a real depth-1 clone fails git describe outright ("No
names found"), while the Compare API gives an identical, correct
answer with no local tag fetch at all.
- Never fails the kernel build itself. The script writes a full
replacement changelog to a temp file and swaps it in with mv at the
end, so a failure partway through leaves the real changelog
untouched, and the calling workflow step downgrades any non-zero
exit to a warning.
- Adds a "Checkout CI scripts" step, its own path (matching kernel-src
and docker-pkg-build, since checking out to the workspace root would
clean the whole workspace and delete kernel-src), pinned to main:
this repository's own CI scripts must always come from the trusted
default branch, never from whatever ref is being built, including
when this workflow is called via workflow_call from premerge-pr.yml,
where github.sha reflects the caller's resolute-qcom-devel merge
commit rather than main.
- Sets persist-credentials: false on this and the two other checkout
steps in this file (kernel source, docker-pkg-build), per zizmor's
credential-persistence finding. Neither needs the checkout token
afterward: the devel-PR merge step already authenticates its own
git remote independently.
- Surfaces the resolved version in the existing build summary table.
Verified against the real repository before writing any workflow
change: constructed the actual changelog prepend against the real
fetched debian.qcom/changelog content and confirmed dpkg-parsechangelog
parses the full result cleanly, including the untouched older entry
beneath it; confirmed abi_release and uname -r are unaffected, since
both derive only from the portion of the version before the appended
suffix; confirmed worst-case filename length stays well under
filesystem limits; and confirmed no script in qcom-distro-images parses
package versions via a fragile filename regex that this could break,
since all of it goes through dpkg-deb metadata instead.
Signed-off-by: Bjordis Collaku <bcollaku@qti.qualcomm.com>
The suffix step ran green on every build while silently doing nothing. Build 33690758951 shows why: apply-local-version-suffix.sh: line 68: gh: command not found gh is not installed on the self-hosted runner. The existing handler caught it, warned, emitted KERNEL_LOCAL_VERSION=unmodified and exited 0, so the step passed and an hour-long build shipped the plain Canonical version. gh was only the first missing tool. jq and dpkg-parsechangelog run on the runner host as well, not inside the build container, so the container's dpkg-dev does not cover them. dpkg-parsechangelog is reached after every guard in this script, where a failure aborts under set -euo pipefail and produces the same silent "unmodified" outcome from a different cause. Check all three up front and name every missing one in a single warning, so a runner that is short a tool costs one line of log rather than one build per tool. The check sits after the suite early-exit, so builds that skip the suffix entirely still need nothing installed. This does not by itself install anything; build-kernel.yml has to provide gh and dpkg-dev on the runner. Signed-off-by: Bjordis Collaku <bcollaku@qti.qualcomm.com>
Completes the preflight check in the prior commit: gh, jq, and dpkg-dev (for dpkg-parsechangelog) are not present on the self-hosted runner host, only inside the build container, which the local version suffix step never touches. Installs all three before that step runs, matching the existing "Install GitHub CLI" pattern already used in premerge-distro-validation.yml. Signed-off-by: Bjordis Collaku <bcollaku@qti.qualcomm.com>
47c132b to
5998504
Compare
Problem
Every package built from
resolute-qcom-develcarries the version of whateverCanonical tag it was last rebased onto. A build with five Qualcomm commits on
top and a build with fifty are both
7.0.0-1013.16. The version string cannottell them apart, so
dpkg -lon a device says nothing about which downstreamcontent it actually has, and apt has no basis for ordering two devel builds.
Change
Appends a Debian local version suffix,
+qcom<N>.<sha>, whereNis thenumber of commits past the nearest
Ubuntu-qcom-*sync tag:Applied only to
resolute-qcom-develbuilds carrying commits past a sync tag.Mirror builds and
kernel_version-pinned builds stay byte-identical toCanonical.
Nis derived from the GitHub Compare API rather than local git tags. Thebuild checks out shallow, deepening history does not fetch tags, and this
repository's tag namespace is too large to fetch on every build. The script
selects the tag with the smallest
ahead_byamong those that are genuineancestors of HEAD, which is the nearest sync point regardless of the order the
API returns tags in.
Failure is never fatal to the kernel build. Every path emits a warning, sets
KERNEL_LOCAL_VERSION=unmodifiedand exits 0. The rewritten changelog iswritten to a temp file and validated with
dpkg-parsechangelogbefore beingmoved into place, so a partial write cannot reach the build.
Version ordering
Ordering is dominated by the Canonical base version, which advances on every
sync, so a resync always supersedes a devel build of the previous tag
regardless of
N. Within a single base version,Nonly grows as commitsland. dpkg splits on the last hyphen and compares the revision left to right,
so the upload number decides before
Nis ever reached:Verified with
dpkg --compare-versionson both Ubuntu (dpkg 1.22.6) andDebian trixie (dpkg 1.22.22), all five passing on each:
The last two are the ones the periodic rebase exercises: a resync wins even
when
Ndrops from 47 to 5, and a higherNon an older base never wins.Files
scripts/apply-local-version-suffix.sh(new): resolves the changelog viadebian/debian.env, computesN, and rewrites the topmost entry..github/workflows/build-kernel.yml: checks out the CI scripts, installsthe host tools, and runs the script between the devel-PR merge and the build,
then surfaces the resulting version in the run summary. The script runs on the
runner host rather than inside the build container, so the workflow installs
gh,jqanddpkg-dev, and the script preflights all three and names anythat are missing in a single warning.
premerge-pr.ymlcalls this workflow withsuite: resolute-qcom-devel, so PRcheck builds are suffixed too, with
Nand the sha taken from the merge ref.Verification
Every link from the changelog rewrite to the package version was traced in the
source before the validation build:
debian/debian.envsetsDEBIAN=debian.qcomdocker run -v "${GITHUB_WORKSPACE}:${GITHUB_WORKSPACE}"cleanpropagates itdebian/rules:143,cp $(DEBIAN)/changelog debian/changelogrules.d/0-common-vars.mk:18,DEB_VERSION := $(shell dpkg-parsechangelog -l$(DEBIAN)/changelog -S version)ABI naming is unaffected, which was the main risk.
abinum := $(firstword $(subst .,$(space),$(DEB_REVISION)))splits1013.16+qcom47.e2d8ff83c07finto1013 16+qcom47 ge2d8ff83c07f, soabinumstays
1013andabi_releasestays7.0.0-1013. Package names are unchanged;only the version carries the suffix.
Validated
Focused qcom build 34644977835 completed successfully against
resolute-qcom-develcommit673305dcc96b722530c34576e6ff6df636cca637withdbgsymdisabled. The validation ref contained the PR head plus one isolated workflow-only override soCheckout CI scriptscould load the unmerged script; that override is not part of this PR.The raw job log confirms the script selected the nearest Canonical sync tag and applied the new format:
The suffix reached the actual kernel build, not only the changelog or package metadata:
All six expected packages were produced with the new version, while their package names and ABI remained unchanged:
All six packages uploaded successfully under
pkg/temp/ubuntu-qcom-kernel/34644977835-1/. The log contains no GitHub Actions warning or error annotations and no output using the previousg-prefixed rendering.The values reach the expected runtime surfaces:
uname -r7.0.0-1013-qcomuname -v#16+qcom88.673305dcc96b SMP PREEMPT_DYNAMIC <build date>/proc/versionLinux version 7.0.0-1013-qcom ... #16+qcom88.673305dcc96b SMP PREEMPT_DYNAMIC .../proc/version_signatureUbuntu 7.0.0-1013.16+qcom88.673305dcc96b-qcom 7.0.14uname -rremains unchanged because it isabi_releaseplus flavour, preserving the module search path and package layout.KBUILD_BUILD_VERSIONfeedsUTS_VERSION, which is reported byuname -vand/proc/version;CONFIG_VERSION_SIGNATUREis exposed directly through/proc/version_signature.A previous downstream server and desktop image build validated consumption of the suffixed Canonical package set. This format adjustment changes only the commit rendering inside the package version; package names, ABI release, and downstream package structure remain unchanged.
Non-goals
Nothing is ever committed back to git. No changes to Canonical packaging under
debian/. Mirror andkernel_version-pinned builds are untouched.Builds using the
devel_prsinput keep the Canonical version, since the mergedtree exists only on the runner.