Skip to content

fix(test): remove specs that cannot fail, and fix one whose name lied - #30

Merged
damacus merged 1 commit into
mainfrom
test/remove-specs-that-cannot-fail
Aug 25, 2026
Merged

damacus merged 1 commit into
mainfrom
test/remove-specs-that-cannot-fail

Conversation

@tas50

@tas50 tas50 commented Aug 24, 2026

Copy link
Copy Markdown
Member

Seven tests removed, one rewritten. Every removal was verified by mutation — break the thing the test claims to guard, confirm the suite still catches it.

353 runs / 1155 assertions, down from 360 / 1179.

Method

For each candidate, mutate lib/ and record which spec files fail:

Mutation Before After
add a global url= setter to Client regression regression
reparent NotFound outside the hierarchy 5 files 5 files
container_create drops platform from the query 3 files 3 files
drop platform from the signature 3 files 3 files
npipe URLs route to Tcp regression + transport transport
add a method to ::Docker regression + version regression
break the Windows pipe constant regression regression
default_url ignores Windows entirely nothing regression
break the unix socket constant nothing regression

Two mutations that nothing caught before are caught now.

Cannot fail at all

session_spec — "constructs without going through Net::HTTP.new's proxy handling"

_(session).must_be_kind_of Net::HTTP   # true by `class Session < Net::HTTP`
_(session.proxy?).must_equal false

proxy? is false in every configuration — including @proxy_from_env forced true with http_proxy set — because the address is localhost. Verified directly. The regression it names is caught by the two tests above it, which could not construct a session at all.

fake_spec — "is a real socket, so it satisfies what Net::BufferedIO asks of one" asserts a UNIXSocket responds to read_nonblock, write, to_io. That is Ruby's stdlib. The three tests above it already drive a real HTTP exchange through the fake.

regression — "puts everything under Docker::API" — Client.name restated from its own declaration.

regression — "resolves registry credentials per call" — respond_to? :resolve on a module_function declared three lines away in the source. auth_spec exercises resolve behaviourally nine times.

Strictly duplicated

  • version_spec "defines no methods on ::Docker" — same mutation, same catch, as the regression spec's "adds nothing to ::Docker".
  • regression "builds a named-pipe transport from a npipe URL" — transport_spec covers npipe routing, which is where transport routing belongs.
  • regression "declares every query parameter the specification defines" — reflection over the signature. Dropping platform from the query hash fails the wire assertion directly above it while this one passes; dropping it from the signature fails both. A strict subset.

Superseded

regression — "loads no HTTP gem at all" greps $LOADED_FEATURES for four named gems — which only populates if the gem is actually installed:

excon            allowlist catches: true  | LOADED_FEATURES grep could catch: false
some-random-gem  allowlist catches: true  | LOADED_FEATURES grep could catch: false

zero_dependency_spec holds every require in lib/ to an allowlist of default gems. It catches all four plus the ones nobody thought to name.

Rewritten, not removed

regression — "defaults to the named pipe on Windows" was:

_(Docker::API::Config::DEFAULT_WINDOWS_PIPE).must_include "pipe/docker_engine"

A constant containing a substring of itself. The name promised routing; the body checked spelling. Deleting the Windows branch from default_url was caught by nothing in the suite.

It now stubs windows? and asserts default_url both ways — against literals rather than the constants, since comparing default_url to DEFAULT_WINDOWS_PIPE only proves the two agree with each other. Those literals are an external contract: the pipe Docker Desktop actually publishes.

What I checked and kept

Three I expected to cut turned out to earn their place, and the mutations say so:

  • "offers no way to change a daemon out from under a live client" — hand-lists five method names, which read as weak. Adding Client.url= is caught by this test and nothing else.
  • "defaults to the named pipe on Windows" (the constant check) did catch a broken constant — it just missed the routing. Hence rewrite rather than delete.
  • errors_spec "keeps every error under the one root" restates the class hierarchy, but it is the test that names the gem's central promise and gives the clearest failure message.

Verification

bundle exec rake — 353 runs, 0 failures. steep check clean. Chefstyle clean. Full mutation table above re-run against the trimmed suite.

Seven tests removed and one rewritten. Each removal was checked by
mutating lib/ and confirming the same defect is still caught -- by a spec
that exercises behaviour rather than restating the source.

Cannot fail at all:

  session_spec "constructs without going through Net::HTTP.new's proxy
  handling" -- `must_be_kind_of Net::HTTP` is true by the class
  declaration, and `proxy?` returns false in every configuration,
  including @proxy_from_env forced true with http_proxy set, because the
  address is localhost. Forcing Session through Net::HTTP.new is caught by
  the two tests above it, which would fail to construct at all.

  fake_spec "is a real socket" -- asserts a UNIXSocket responds to
  read_nonblock, write and to_io. That is Ruby's stdlib. The three tests
  above it already drive a real exchange through the fake.

  regression "puts everything under Docker::API" -- Client.name restated
  from its own declaration.

  regression "resolves registry credentials per call" -- respond_to?
  :resolve on a module_function declared three lines away. auth_spec
  exercises resolve nine times.

Strictly duplicated:

  version_spec "defines no methods on ::Docker" -- same mutation, same
  catch, as the regression spec's "adds nothing to ::Docker".

  regression "builds a named-pipe transport from a npipe URL" --
  transport_spec covers npipe routing, which is where transport routing
  lives.

  regression "declares every query parameter the specification defines" --
  reflection over the method signature. Dropping platform from the query
  hash fails the wire assertion directly above it while this one passes;
  dropping it from the signature fails both. A strict subset.

Superseded:

  regression "loads no HTTP gem at all" -- greps $LOADED_FEATURES for four
  named gems, which only populates if the gem is installed.
  zero_dependency_spec holds every require in lib/ to an allowlist of
  default gems, catching any of them plus the ones nobody thought to name.

Rewritten:

  regression "defaults to the named pipe on Windows" asserted that
  DEFAULT_WINDOWS_PIPE contains a substring of itself. Removing the
  Windows branch from default_url entirely was caught by nothing in the
  suite -- the name promised routing and the body checked spelling. It now
  stubs windows? and asserts default_url both ways, against literals
  rather than against the constants, so a typo in a constant fails too.
  That is three mutations caught where there was one.

353 runs, 1155 assertions.

Signed-off-by: Tim Smith <tim@mondoo.com>
@damacus damacus changed the title test: remove specs that cannot fail, and fix one whose name lied fix(test): remove specs that cannot fail, and fix one whose name lied Aug 25, 2026
@damacus
damacus merged commit fc0118d into main Aug 25, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants