[wilke]

Why

No Episode test runs in CI. tests.yml requests ubuntu-latest-96-core (and the e2e jobs -32-core). Those are larger runners this repo doesn't have, so the jobs sit in the queue for 24 h and are cancelled; every Python PR ends red or cancelled. Branch protection is off, so nothing blocks a merge.

Meanwhile, API changes from Oct 2–8 broke 94 tests in 43 of 129 Episode test files, all unnoticed.

What

• episode-tests.yml: a standalone workflow on ubuntu-latest. It sets up the locked test environment through setup-pm (core, plus the dev and test groups) and runs on PRs, on pushes to main, and manually.
• scripts/run_episode_suite.py:
  • stdlib only; runs one pytest process per file, in parallel, as scripts/run_tests.sh does;
  • deselects known failures and exits 1 on any other failure;
  • --check-known lists known-failure entries that pass again.
• tests/episode_suite.txt: the suite. It covers the core packages (method_loop, numeric_control_library, function_library, episode_builder, episode_runtime, iterative_episode_refiner, the inspector) plus the OpenChia test files in tests/agent, tests/openchia_cli, tests/pm and tests/scripts.
• tests/episode_suite_known_failures.txt: the remaining failures, grouped by the change that broke them. Tracked in Episode suite: 75 tests broken by Oct 2–8 refactors (known-failures list) #87.
• 19 tests fixed:
  • the shared refinement fixture no longer imports the removed measure_groups module except in its grouped variant;
  • the test_reports assignments include materialization_targets;
  • the method_loop recursion test supplies the ReportContract and synthesize_report that child invocations now require;
  • the emitter turn-error test uses its own workspace, because the repo conftest puts HERMES_HOME under tmp_path.

Local result (macOS)

110 files run and 1 fails: tests/pm/test_prepared_project.py needs the installed uv, which setup-pm provides in CI. 13 files and 42 tests are deselected as known failures. The slowest file takes 16 s.

Not in this PR

• Moving the large-runner, release and canary workflows to manual dispatch. This changes shared CI, so it needs agreement first.
• Making this check required on main. That needs an admin.
• The installer gap (ModuleNotFoundError: episode_runtime in the fresh-install E2E).

🤖 Generated with Claude Code