Skip to content

ci: harden the release workflow and keep native caches warm - #302

Merged
devopvoid merged 4 commits into
mainfrom
ci/pipeline-hardening
Sep 29, 2026
Merged

devopvoid merged 4 commits into
mainfrom
ci/pipeline-hardening

Conversation

@devopvoid

@devopvoid devopvoid commented Sep 28, 2026 •

Copy link
Copy Markdown
Owner

Release workflow

  • Release notes no longer run as shell code. publish-release expanded the git-cliff notes (commit subjects) straight into its script, so a quote or backtick in a commit subject broke the step and ran as shell code. The notes, and developmentVersion elsewhere, now go through env:, and the notes reach gh release create as a file.
  • The GitHub release uses the same version as the Maven tag. Before, git-cliff computed the version with --bump from the commit types, while release:prepare tags the pom's version. A breaking-change commit would have made them differ, and gh release create would have created an unbuilt tag on main. git-cliff now gets --tag v<pom version>, and --verify-tag makes any mismatch fail.
  • Nothing reaches main until the release is prepared. The changelog job is merged into prepare-release, which commits CHANGELOG.md locally before release:prepare. The changelog commit, the release commit, the tag and the development commit then go out in one git push --atomic. Before, the changelog was pushed first, so a failed release left main with a changelog entry for a release that never happened.
  • The input is checked first. developmentVersion must look like 1.2.3-SNAPSHOT and differ from the release version, the tag must not already exist, and release:prepare must choose the version the changelog was written for.
  • The tag is passed to the build and publish jobs. Four jobs used to guess the newest tag in the repository with git rev-list --tags. They now check out the tag prepare-release pushed, taken from its job output.
  • release:prepare runs clean validate instead of the default clean verify. That compiled WebRTC and FFmpeg on an uncached runner (about 35 of the 52 minutes of the last release), and the platform jobs build the tagged tree again anyway.
  • Token permissions: read-only by default. Only prepare-release and publish-release get contents: write. A release concurrency group stops two releases running at once.

Build workflow

  • More pushes to main build. The push trigger used to match only .cpp, .h and .java. Now it runs for everything except docs, *.md, issue templates and pages.yml. PRs can only restore caches from their own ref and from main. After build: sync the WebRTC checkout without git history #298 changed the WebRTC CMakeLists and main didn't rebuild, fix: repair the release pipeline and publish the media module #299 compiled WebRTC again under the same key: 24–52 min per platform instead of 4–6. Pom, CMake and submodule changes also deploy a snapshot now.
  • Caches are saved right after the build. The WebRTC and FFmpeg caches are now restored with actions/cache/restore and saved with actions/cache/save right after the Build step, in all four build/release actions. Before, actions/cache saved only when the whole job succeeded, so one failing test threw away a finished native build.
  • Superseded PR builds are cancelled. A new push to a PR cancels that PR's running build. Runs on main are never cancelled.
  • Docs-only PRs skip the native build. pages.yml now builds the site, without deploying, on PRs that touch docs/.
  • Timeouts: every test run gets a 30-minute Surefire timeout (-Dsurefire.timeout=1800), and test-natives gets a 30-minute job timeout. Composite action steps can't set timeout-minutes, and macOS has no timeout command.
  • Test reports are uploaded when a job fails.
  • Setup:
    • The Chromium Clang package is cached, and tar no longer lists every extracted file.
    • Homebrew's auto-update is off.
    • Two disabled Windows steps are deleted.

Behavior changes

  • The "next development iteration" push from each release now starts a normal SNAPSHOT build on main.
  • The CHANGELOG commit is authored by whoever dispatches the release, not github-actions[bot].
  • prepare-release uses JDK 17, like the platform builds, instead of 21.

Testing

  • The workflows pass actionlint 1.7.12.
  • I ran the version checks locally. The literal default X.Y.Z-SNAPSHOT, 0.21.0 and 0.20.0-SNAPSHOT are rejected, 0.21.0-SNAPSHOT passes, and an existing tag is detected.
  • Build run 36482716826 passed. On every platform the new cache steps restored from main and saved the WebRTC and FFmpeg builds right after compiling.
  • A dry-run release dispatched from this branch tests the new release flow.

Pass the release notes and the development version to the scripts through
the environment instead of expanding them into the script text. The notes
are commit subjects, so a quote or a backtick in one broke the release step
and ran as shell code.

Take the release version from the pom, which is what release:prepare tags,
instead of letting git-cliff bump it from the commit types. The GitHub
release now always matches the Maven tag, and --verify-tag makes a mismatch
fail instead of creating an unbuilt tag on main.

Grant write access only to the jobs that push, and keep two releases from
running at once.
Build on every push to main that can change a build, not only on source
changes. A PR can restore caches only from its own ref and from main, so
when main skipped a build change, every following PR compiled WebRTC from
scratch under a key that already existed on another PR's ref. Pom, CMake
and submodule changes now also deploy a snapshot.

Save the WebRTC and FFmpeg caches right after the build instead of in the
post step of actions/cache, which saves only when the whole job succeeds;
a failing test threw the finished native build away.

Cancel a PR's running build when a newer push supersedes it, and give the
workflow a read-only token.
Fold the changelog job into prepare-release and commit CHANGELOG.md
locally, before release:prepare. The changelog commit, the release commit,
the tag and the development commit now go out in one atomic push at the
end, so a failure on the way leaves main untouched instead of carrying a
changelog for a release that never happened.

Check the input before anything else: developmentVersion must be a
SNAPSHOT that differs from the release version, the release tag must not
exist yet, and release:prepare must choose the version the changelog was
written for.

Hand the pushed tag to the platform and publish jobs as an output instead
of having each of them guess the newest tag in the repository.

Run release:prepare with "clean validate" as its preparation goals. The
default "clean verify" compiled WebRTC and FFmpeg on an uncached runner,
about 35 of the 52 minutes of the last release, only for the platform jobs
to build the tagged tree again.
Leave docs-only PRs out of the native build and build the docs site on
PRs that touch it instead, without deploying, so a broken VitePress build
shows up before the merge.

Cache the Chromium Clang package and stop listing every file it
extracts, keep Homebrew from updating itself before each install, and drop
the two disabled steps of the Windows preparation.

Give every test run a 30 minute Surefire timeout, and test-natives a job
timeout, so a hung test no longer holds a runner for six hours. Composite
action steps have no timeout-minutes, and macOS has no timeout command.

Upload the Surefire reports when a job fails.
@devopvoid
devopvoid added this pull request to stack #305 September 28, 2026 21:49
@devopvoid
devopvoid merged commit 7b8e4e9 into main Sep 29, 2026
29 checks passed
@devopvoid
devopvoid deleted the ci/pipeline-hardening branch September 29, 2026 15:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant