Skip to content

Modernize testing: GitHub Actions CI, broader specs, dependabot - #286

Open
tas50 wants to merge 7 commits into
chef:mainfrom
tas50:modernize-testing
Open

tas50 wants to merge 7 commits into
chef:mainfrom
tas50:modernize-testing

Conversation

@tas50

@tas50 tas50 commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Moves testing to GitHub Actions and fills the gaps in the spec suite that could let a regression through unnoticed. The new specs found three real bugs; each is fixed in its own commit.

Bug fix: FakeShellOut raised NameError

The helper returns FakeShellOut when commands run over a train transport (chef target mode). It used OpenStruct but never required ostruct, so it only worked if something else had loaded it first. ostruct is also no longer a default gem as of Ruby 3.5. It's now a small Struct. Without the fix, 8 of the new helper specs fail with exactly that NameError.

Bug fix: login: true kept the parent's primary group

With user: and login: true but no group:, #gid resolves the user's primary group, but set_group only ran when group was set. The child switched uid and cleared supplementary groups, yet kept the parent's primary gid. Run from root, a command "as nobody" still had gid 0. Found by the new root-only CI job.

Bug fix: timeouts raised LoadError on Windows with Ruby 4.0

wmi-lite requires win32ole, which is a bundled rather than default gem as of Ruby 4.0. mixlib-shellout loads wmi-lite on timeout to kill the process tree, so every timeout raised LoadError: cannot load such file -- win32ole instead of CommandTimeout, and the process tree kept running. win32ole is now a dependency of the ucrt gem and in the Gemfile on Windows.

CI (.github/workflows/ci.yml)

  • Matrix: Linux, macOS and Windows × Ruby 3.1, 3.2, 3.3, 3.4 and 4.0, plus ruby-head as an allowed failure.
  • Root on Linux: the user/group switching, login simulation and cgroup specs previously only ran because the Buildkite container happened to run as root. They now have a dedicated job.
  • Frozen string literals: one run with --enable-frozen-string-literal forced on for everything.
  • Packaging: builds both gems with --strict, installs the .gem into an isolated GEM_HOME, and smoke tests it outside the source tree.

Specs (147 → 198 examples locally)

  • Helper (Chef's and Ohai's entry point): the shell_out_compacted argument contract that Chef's own specs stub against, default locale/PATH injection and overrides, default_env: false, Chef provider timeout injection, live streaming, and the full train transport path.
  • ShellOut:
    • array commands never go through a shell
    • env reaches the child and doesn't leak into the parent
    • umask is applied
    • signal-killed children have a nil exitstatus and count as errors
    • sensitive: keeps output out of exception messages
    • logger, log_tag and log_level
    • elevated: is rejected on Unix
  • Root-only: uid/gid switching, and login simulation dropping root's supplementary groups.
  • Packaging/load:
  • Infrastructure:
    • modern RSpec config: no monkey patching, random order, verified partial doubles, --only-failures
    • any Ruby warning from lib/ fails the run
    • cgroup specs skip explicitly instead of passing without asserting anything

Actions and dependabot

  • Every action is pinned to a commit SHA with a version comment and updated to its latest release. actionshub/dco and get-pr-commits were tracking @main, and linelint was tracking @master.
  • .github/dependabot.yml updates actions and bundler weekly, grouped, with a 7-day cooldown.
  • DCO is skipped on dependabot-authored PRs, since dependabot can't sign off.

README

  • CI badge now points at GitHub Actions, gem version badge moved to shields.io, license badge added.
  • Links fixed, and a Development section added covering how to run the specs, including the root-only ones.
  • The CONTRIBUTING link no longer points at master.

Needs a maintainer

  • Buildkite removal: this PR deletes .expeditor/verify.pipeline.yml, its runner scripts, and the verify pipeline entry. Expeditor's version bump and publish steps are untouched. If branch protection requires the Buildkite verify check, it needs to be swapped for the new CI checks.
  • Org-managed file: ci-main-pull-request-stub-1.0.7.yml is managed centrally and left as is. It still uses the deprecated ::set-output.

FakeShellOut, which shell_out returns when running over a train transport
connection (chef target mode), built its status with OpenStruct but never
required ostruct. It only worked when something else in the process had
already loaded it; otherwise it raised NameError. ostruct is also no longer
a default gem as of Ruby 3.5, so requiring it would just move the failure to
consumers whose Gemfile doesn't list it.

Replace it with a small Struct that answers success?.

Also teach cspell the words helper.rb already uses, since the spellcheck
job only scans files a PR touches.

Signed-off-by: Tim Smith <tsmith84@proton.me>
Spec infrastructure:
- Modern RSpec defaults: RSpec.describe without monkey patching, random
  ordering, verified partial doubles, --only-failures support.
- Fail the run if lib/ emits a Ruby warning.
- Drop the DependencyProc ruby-version filter; the gemspec requires 3.1+.
- Tag the cgroup specs :requires_root and skip explicitly when cgroup v2 is
  missing, instead of passing without asserting anything.

New coverage:
- Helper: the shell_out_compacted argument contract chef's own specs stub
  against, default locale/PATH injection and overrides, default_env: false,
  chef provider timeouts, live streaming, and the train transport path
  (command joining, cwd/input wrapping, FakeShellOut results).
- ShellOut: array commands bypass the shell, env reaches the child without
  leaking into the parent, umask, signal-killed children, sensitive output
  kept out of exceptions, logger/log_tag/log_level, elevated rejected on unix.
- Root-only: uid/gid switching and login simulation, including dropping
  root's supplementary groups.
- Packaging: VERSION matches version.rb, both gemspecs are valid and ship
  the right files and dependencies, the library loads cleanly with frozen
  string literals and warnings on, and requiring it does not pull in
  tmpdir/fileutils (chef#282).

Signed-off-by: Tim Smith <tsmith84@proton.me>
Replace the Buildkite verify pipeline with a GitHub Actions workflow:
- Linux, macOS and Windows on every supported Ruby (3.1 through 4.0), plus
  ruby-head as an allowed failure.
- The full suite as root on Linux, so user/group switching, login simulation
  and cgroup specs actually run.
- A run with --enable-frozen-string-literal forced on for everything.
- Build both gems with --strict, install the result in an isolated GEM_HOME
  and smoke test it outside the source tree.

Pin every action to a commit SHA with a version comment and add dependabot
(with a cooldown) for actions and bundler. Update actions that had fallen
behind or tracked a branch. Skip the DCO check on dependabot PRs, since
dependabot cannot sign off.

Give the gemspec a real description so gem build --strict passes, and drop
Gemfile branches for Ruby 3.0, which is no longer supported.

Signed-off-by: Tim Smith <tsmith84@proton.me>
Swap the Buildkite badge for GitHub Actions, use shields.io for the gem
version and add a license badge. Link docs instead of bare URLs, point the
"see also" section at Process.spawn/Open3, use https for the license URL,
and document how to run the specs, including the root-only ones.

Fix the CONTRIBUTING link that still pointed at master, and update the
copilot instructions for the new CI layout.

Signed-off-by: Tim Smith <tsmith84@proton.me>
@tas50
tas50 requested review from a team and jaymzh as code owners September 26, 2026 07:07
With login: true and a user but no explicit group, #gid resolves the user's
primary group, but set_group only acted when group was set. The child
switched uid and cleared supplementary groups but kept the parent's primary
group, so a command run as "nobody" from a root process still had gid 0.

Check #gid instead. Without login and without a group #gid is still nil,
so that case is unchanged.

Signed-off-by: Tim Smith <tsmith84@proton.me>
wmi-lite requires win32ole, which is a bundled rather than default gem as
of Ruby 4.0, so it can no longer be required under bundler unless it is
declared. mixlib-shellout loads wmi-lite when a command times out, to kill
the process tree, so on Ruby 4.0 every timeout raised LoadError instead of
CommandTimeout and left the process tree running.

Add it to the Windows gem's dependencies and the Gemfile.

Signed-off-by: Tim Smith <tsmith84@proton.me>
- Run the killed-by-signal spec without a shell. dash, /bin/sh on
  Debian/Ubuntu, forks instead of exec'ing, so it survived and reported
  137 rather than being the killed process.
- Use real doubles in the kill_process_tree specs; stubbing methods that
  bare Objects don't have fails with verified partial doubles.
- Skip -w in the load spec on Windows, where core_ext.rb deliberately
  redefines win32-process's Process.create.

Signed-off-by: Tim Smith <tsmith84@proton.me>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant