[ethan-scitix]

Type

• feature — new benchmark, task, or capability

Summary

Connects the dry-run from #217 to the per-stage plan from #211. Before this, the dry-run reported outcome counts without naming the stage that changed.

• RunPlan.stages: plan_resume's comparison of the recorded identities with the runner's own (data, inference, postprocess, feedback, report), with rerun_from set to what the runner decides. preprocess is left out: it is decided per sample, so it would always read unknown, and a data change already shows in resume_plan as restarted. report names a re-report. A directory with no identities, which the runner trusts, gets rerun_from: null, not the bare comparison's postprocess. refused is always null here (the bare comparison says inference for a directory with no identities), because a refusal raises before a plan is returned and the task entry holds error instead.
• Each dry-run task carries stages and its own needs_inference, using _infers, the same test _models_to_launch uses to decide launches.
• Text: task echo: resume (3 regraded), from feedback, 0 to infer, 3 to grade.

A report-only change reads start: reuse, resume_plan {kept: N, rereported: N, ...}, rerun_from: report, to_infer: 0, needs_inference: false.

Identity impact. None. Nothing new is recorded.

Related Issues

Closes #219. Follows #211 and #217.

Test Plan

Automated

• Lint/format clean (ruff check && ruff format --check)
• Type check clean (ty check)
• Unit tests pass (pdm run pytest)

pytest tests/unit tests/integration tests/acceptance -m "not stress and not benchmark" -n auto --cov passes on macOS, at 97.12% coverage. The one failure is test_exec_quotebench_anchor.py, which also fails on main on macOS. ty check, sync_package_stubs.py --check, check_preflight.py and pre-commit run -a are clean.

• test_stage_resume.py::TestPlanRunNamesTheStage calls plan_run on a plugin task, one test per outcome: kept (nothing changed), rereported (metrics.py edit), regraded (grade.py), repostprocessed (postprocess code), restarted (changed sample, which also checks preprocess is absent), dropped (removed sample), and a directory with no identities (which also checks refused is null). Each test checks stages.rerun_from, the stage comparisons, to_infer and to_grade. Each also checks that planning writes nothing and calls no model, judge or grading stage.
• test_lazy_launch.py::TestDryRunStagePlan runs the real CLI:
  • Report-only: the dry-run shows rerun_from: report and needs_inference: false. sieval run --resume then re-reports on a node with no GPU, with no engine started and no request sent, and records the same runs[].resume.
  • For kept, re-report, regrade, new sample and dropped sample, the dry-run's top-level and per-task needs_inference match what sieval run --resume then launches and requests. to_infer matches the requests, and resume_plan matches the recorded runs[].resume.
• Removing the rerun_from or refused override fails the no-identities test; keeping preprocess fails the changed-sample test. Making _infers ignore to_infer fails 12 tests.