[ethan-scitix]

Type

• refactor — code restructuring, no behavior change (except the removals below, which are BREAKING)

Summary

A cleanup PR in two commits: speed up the unit suite, and delete the APIs whose deprecation window has closed.

1. Unit + integration wall time: 64s → 37s (-n 16; serial 245s). Almost nothing was redundant; the time was idle waiting.

• tests/conftest.py defaults USE_TORCH=0. datasets imports torch whenever it is installed, for a formatter nothing here uses: import sieval drops from 2.1s to 1.1s, paid once per xdist worker, offload-pool worker and subprocess test (~25s of the saving).
• test_across_releases.py::test_a_changed_prompt_restarts_every_sample (~11s → 0.4s, ×3 releases). It patched preprocess from the test module, so the preprocess identity walk AST-parsed the CLI and everything it imports; it now patches from an on-disk package, as the report test already did. The restarted samples' retries against the dead endpoint now skip the SDK's backoff, not the retries.
• spider2_lite missing-credentials test (~12s → ms): it only deleted the env var, so google.auth.default() probed the GCE metadata server until timeout. It now uses no_bigquery_credentials (moved to tests/unit/tasks/conftest.py), which also stops it depending on the host having no ADC.
• Deleted test_no_model_type_restriction_accepts_both, a byte-identical twin of test_no_requires_accepts_any_model. A sweep of tests/unit + tests/integration found no other duplicate or orphaned tests; "legacy" tests all exercise live code (_legacy_binding, schema-1/2 identity reads).

2. BREAKING — remove what was deprecated in 0.10 and 0.11. Migration is in the CHANGELOG ### Removed entry.

• sieval.cli.leaderboard.session (deprecated 0.11.0, "kept for one release", refactor(cli)!: scope SiEval to an evaluation engine #199) → import from sieval.cli.session. Its FORBIDDEN_SUBPACKAGE rule and tests go with it.
• Resume aliases (deprecated 0.10.0): --resume-from is now an unknown option; runner_config.auto_resume / .resume_from are refused with the resume form to use rather than dropped, so a resume can never silently turn into a fresh run. The same-named Python args and fold_resume_aliases are gone.
• sieval.core.models.Transport, OpenAIChatTransport, OpenAICompletionsTransport (one compatibility cycle since feat(models): rebuild the provider-neutral model binding plane (RFC #25) #45). SglangTransport stays: it is the live sglang_legacy executor.
• sieval.infer.InferConfig, unused since DeploymentPlan.

Deliberately not removed — marked deprecated but still load-bearing: ChatModel / GenModel (13 in-tree task modules), the Capability enum + Model.capabilities (52 uses across dialects), the dialects' direct arun() (SglangGenModel and the dialect tests call it), and the class-name-only deserialization fallback (reads old on-disk records — a reproducibility contract).

Test Plan

Automated

• Lint/format clean (ruff check && ruff format --check)
• Type check clean (ty check, whole repo incl. tests/)
• Unit tests pass — pytest tests/unit tests/integration tests/acceptance -m "not stress and not benchmark" -n 16: 9577 passed, 2 skipped in 50s
• scripts/check_preflight.py all PASS, scripts/sync_package_stubs.py --check, scripts/check_layer_imports.py

Manual

• sieval eval cfg.yaml --resume-from /x → exit 2, No such option: --resume-from (Possible options: --result-dir, --resume)
• A config with runner_config.resume_from → exit 1, runner_config.resume_from was removed in 0.12; use runner_config.resume: <dir>, no result dir created

Note: one full run hit test_runner.py::TestE2EStageConcurrency::test_stage_concurrency_limit_allows_higher (5 <= 4) once; 40 isolated repeats (with and without torch) all passed. Pre-existing flake, not touched here.

Checklist

Required (all PRs)

• PR title follows conventional format (type(scope): description)
• No internal paths, credentials, or personal info in committed files
• AI-generated code has AI-Generated Code - <model> (<provider>) in module docstring
• No new upper-layer dependencies added to core/
• Deleted code verified — no remaining call sites depend on it

If: Breaking Change

• Described what breaks and migration path in Summary
• Existing tests updated to reflect new behavior

🤖 Generated with Claude Code