Repository navigation
Streamline the test suite: fewer, behavior-focused tests #497
Description
The suite has about 3,200 tests and takes about 5 minutes on a laptop with xdist. A small behavior change in the orchestrator (resuming a session that stops with nothing staged, #496) first broke 53 tests. Most of them did not test that behavior. They pinned an exact sequence of harness calls along their way to testing something else.
Goal: fewer tests, each testing one behavior, so a change breaks only the tests about it.
Suggested pass:
- Find tests that assert full harness call sequences or exact prompt text when their subject is something else. Replace those assertions with the one property under test.
- Merge parametrized cases that exercise the same code path (for example the many
test_scope_refusal_*variants andtest_review_submit_parks_and_delivers_verdictwith 12 parameter combinations). - Move shared scenario setup into a few fixtures, so a behavior change is fixed in one place.
- Delete tests that duplicate a covered path with a different fixture.
- Track the count and the runtime before and after. Coverage of
src/should not drop.
This should run as its own PR series, with no behavior changes mixed in.
Activity
Merge pull request #507 from outerloop-science/tests/slim-orchestrator
Merge pull request #508 from outerloop-science/tests/slim-attempt-review
Merge pull request #509 from outerloop-science/tests/slim-attempt
Merge pull request #510 from outerloop-science/tests/slim-tick
Merge pull request #511 from outerloop-science/tests/slim-inbox
Merge pull request #518 from outerloop-science/tests/slim-syscall
Merge pull request #519 from outerloop-science/tests/slim-climbboard
Merge pull request #567 from outerloop-science/maint/tests-orchestrator
Merge pull request #576 from outerloop-science/maint/tests-attempt-re…
Merge pull request #582 from outerloop-science/maint/tests-tick-2
Merge pull request #584 from outerloop-science/maint/tests-init-start
Merge pull request #585 from outerloop-science/maint/tests-attempt
Merge pull request #587 from outerloop-science/maint/tests-syscall-gi…
Merge pull request #588 from outerloop-science/maint/tests-harness-local
Merge pull request #589 from outerloop-science/maint/tests-climbboard…
renmengye commentedon Oct 24, 2026
Slices merged 2026-10-24, all tests-only:
About 140 tests removed. Each slice was checked with missed-line sets before and after, and no line that was covered before is uncovered now.
Lesson: line coverage alone let boundary cases slip. Reviewers caught dropped permission-bit, cadence, ending-kind and per-kind fencing cases, and all were restored. Future slices should diff the removed parametrize entries against the branches the test body takes.