Skip to content

fix(testing): keep killed CLI children out of tarpaulin's coverage merge - #2748

Merged
chaliy merged 1 commit into
mainfrom
eve/clever-mccarthy-snhng4
Oct 18, 2026
Merged

chaliy merged 1 commit into
mainfrom
eve/clever-mccarthy-snhng4

Conversation

@chaliy

@chaliy chaliy commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

What changed

Every test that spawns the instrumented bashkit binary now points the child's LLVM_PROFILE_FILE outside the directory cargo tarpaulin merges, via a new bashkit::testing::child_profile_path().

Applied at every spawn site: run_cli_argv_bounded and run_raw_core in threat_model_tests (the latter takes the program as a parameter, so it was a latent trap), and cli() plus run_bounded_child in cli_oneshot — whose one direct Command::new(BIN) now goes through cli() so it cannot drift again.

run_input_shape_bounded keeps its own path: it asserts on that profraw as a bounded-artifact receipt. The rule is only that a child never writes into the merge set, not which mechanism gets it there.

Why

The Coverage job has been failing on main intermittently — runs 4588 and 4600, two days apart on unrelated commits — each time after all 2,441 tests had already passed:

Merging coverage reports
thread 'main' panicked at llvm_profparser-0.14.0/src/instrumentation_profile/mod.rs:58:35:
internal error: entered unreachable code: llvm_profparsers works on complete data

A spawned child inherits the parent's LLVM_PROFILE_FILE template, so the instrumented CLI writes a profraw named after the parent test binary into target/tarpaulin/profraws/ — exactly the directory tarpaulin merges. The failing log shows this directly: seven profraws for one integration-* binary, each with a different child PID.

The supervised-child harnesses kill their children by design (bounded_helper_kills_hung_child and friends), and a SIGKILLed process never flushes its profraw, so the merge step hits a truncated file. Whether the victim flushed in time is a race, which is why it is intermittent — and why required CI never sees it: required CI is not instrumented.

Redirecting also fixes attribution. Without it, the CLI's counters land in a file tarpaulin credits to the parent test binary.

Before / After

Before — run 4600 on main, all tests green then a merge panic:

test result: ok. 2441 passed; 0 failed; 4 ignored; 0 measured; 0 filtered out
...
cargo_tarpaulin::statemachine::instrumented: Merging coverage reports
thread 'main' panicked at llvm_profparser-0.14.0/.../mod.rs:58:35:
internal error: entered unreachable code: llvm_profparsers works on complete data
##[error]Process completed with exit code 101.

That the cause is a race, not the tree, is settled by re-running the same commit: attempt 2 of run 4600 passed with no code change. The same tree also passed on the PR head it was merged from. So this PR removes the race rather than a wrong result.

After — the two guards, plus the helper's own unit test:

test testing::tests::child_profile_paths_are_unique_and_outside_the_merge_set ... ok
test bounded_child_launcher_scan_tests::cli_spawning_tests_redirect_the_child_coverage_profile ... ok
test threat_model_tests::resource_exhaustion::bounded_cli_children_get_a_profile_path_outside_the_merge_set ... ok

Validation on this branch:

Check Result
cargo test --workspace --lib --bins --tests --features http_client,ssh,sqlite all 27 test binaries pass
cargo clippy --workspace --all-targets --features http_client,ssh,sqlite -- -D warnings clean
cargo fmt --all --check clean
just check-okf OKF v0.2 conformant

The one local failure is ssh_supabase_tests::ssh_supabase_connects, which dials the live public supabase.sh endpoint on port 22 — unreachable from this HTTPS-only sandbox, unrelated to this change.

How it is guarded

Two layers, because neither alone is sufficient:

  • cli_spawning_tests_redirect_the_child_coverage_profile scans every crate and fails any file that reaches the binary (bashkit_cli_path(), CARGO_BIN_EXE_bashkit) without setting a profile path. It fails closed when the scan finds no spawners at all, so a scan that silently matches nothing cannot pass.
  • The scan can only see that a token is present, not what value is used, so bounded_cli_children_get_a_profile_path_outside_the_merge_set asserts on the built Command: the path ends in .profraw, is not under tarpaulin, and differs per child.

That second test asserts on the Command rather than from inside the child on purpose. The sandbox deliberately withholds host env from scripts (TM-INF-013), so a bounded run cannot echo back what its own process received — the first draft of this test tried exactly that and read back an empty string.

Risk

  • Low. Test-harness only; no production code path changes. The change adds one environment variable to spawned children and touches no assertion, bound, limit or receipt.
  • Cleanup of the redirect directory is deliberately unguarded (one dir per process, removal left to the OS). A killed child's foreign descendant can still be writing there, so a per-call remove could fail with ENOTEMPTY and sink a security test for a reason unrelated to it — profile hygiene is not one of this suite's receipts.
  • Not verifiable end-to-end from this sandbox: cargo tarpaulin is not run locally, so the proof that the merge no longer sees truncated files is the mechanism plus the guards above. The Coverage job on this PR exercises the real path.

Checklist

  • Tests added or updated
  • Backward compatibility considered — new pub fn on the #[doc(hidden)] testing module; no existing signature changes.

Generated by Claude Code

The Coverage job failed on main runs 4588 and 4600, two days apart on
unrelated commits, each time after all 2,441 tests had passed:

    Merging coverage reports
    thread 'main' panicked at llvm_profparser-0.14.0/.../mod.rs:58:35:
    internal error: entered unreachable code: llvm_profparsers works on
    complete data

A spawned child inherits the *parent's* `LLVM_PROFILE_FILE` template, so the
instrumented `bashkit` binary writes a profraw named after the parent test
binary into `target/tarpaulin/profraws/` — the directory tarpaulin merges.
The supervised-child harnesses kill their children by design, and a SIGKILLed
process never flushes its profraw, so the merge step hits a truncated file.
It is a race on whether the victim flushed, hence intermittent, and invisible
to required CI, which is not instrumented. Re-running attempt 2 of run 4600
on the same commit went green, which is the race.

`testing::child_profile_path()` hands each child its own path outside the
merge set: one directory per process, a unique file per call, removal left to
the OS. Cleanup is deliberately unguarded — a killed child's foreign
descendant can still be writing there, and a per-call remove that failed with
ENOTEMPTY would sink a security test for a reason unrelated to it.

Applied at every spawn of the instrumented binary: `run_cli_argv_bounded` and
`run_raw_core` in `threat_model_tests` (the latter takes the program as a
parameter, so it is a latent trap), and `cli()` plus `run_bounded_child` in
`cli_oneshot`, whose direct `Command::new(BIN)` now goes through `cli()` so it
cannot drift again. `run_input_shape_bounded` keeps its own path: it asserts
on that profraw as a bounded-artifact receipt, and the rule is only that the
child never writes into the merge set.

Redirecting also fixes attribution: without it the CLI's counters land in a
file tarpaulin credits to the parent test binary.

Two guards. `cli_spawning_tests_redirect_the_child_coverage_profile` scans
every crate and fails any file that reaches the binary without setting a
profile path, failing closed when it finds no spawners at all. The scan cannot
pin the value, so `bounded_cli_children_get_a_profile_path_outside_the_merge_set`
asserts on the built `Command` — not from inside the child, because the
sandbox withholds host env from the script (TM-INF-013), so a bounded run
cannot echo back what its own process received.

All 27 test binaries pass, clippy -D warnings and fmt clean, check-okf
conformant. The one local failure is `ssh_supabase_connects`, which dials a
live port-22 endpoint unreachable from this sandbox.
@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
bashkit 3656c7e Commit Preview URL

Branch Preview URL
Oct 18 2026, 10:47 AM

@chaliy
chaliy merged commit cd34bf5 into main Oct 18, 2026
48 checks passed
@chaliy
chaliy deleted the eve/clever-mccarthy-snhng4 branch October 18, 2026 11:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant