Skip to content

Streamline orchestrator tests (#497, slice 1) - #567

Merged
renmengye merged 2 commits into
mainfrom
maint/tests-orchestrator
Oct 23, 2026
Merged

renmengye merged 2 commits into
mainfrom
maint/tests-orchestrator

Conversation

@renmengye

Copy link
Copy Markdown
Member

First slice of #497 ("fewer tests, each testing one behavior, so a change breaks only the tests about it"): tests/test_orchestrator.py only, test-only, no behavior changes.

  • Incidental assertions replaced by the property under test (exact brief text and incidental call counts where the test's subject was something else).
  • Same-path parametrized cases merged; the scope-refusal family keeps one case per distinct behavior.
  • Duplicate tests folded into the test that owns the behavior (e.g. the candidate SHA into the end-to-end improvement test, the pre-snapshot scope rejection into the scope-violation test).
  • Shared helpers run_dir and refusal_receipts replace repeated setup.
main this PR
tests in the file 182 171
orchestrator.py statements missed (this file alone) 127 / 869 127 / 869
orchestrator.py partial branches 31 / 324 31 / 324

Equal coverage was not enough: the first pass (154 tests) kept coverage but dropped behaviors (per-prefix refusal handling, GPU-type mismatch, the sibling wake dependency, the implicit-stop path, exact regression boundaries including infinities, the brief's ledger baseline, violating-path truncation, workspace export of refusals, the single-node budget boundary). Two codex rounds found them, and they are restored, mostly as main's original assertions.

Full gate: ruff, format, mypy, pytest (3158 passed, 7 skipped).

Built by DeepSeek-V4.1-Flash; reviewed by codex (two rounds) and Claude.

@renmengye renmengye added the outerloop:review re-request the advisory review label Oct 23, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Round 1 — reviewed head 12176257 — reviewer hermes/gpt-5.6-terra.

terra
Advisory findings from outerloop — the code owner decides. Reply to disagree; the outerloop:no-review label opts this PR out.

Verdict: no defects found.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Round 2 (re-run on the same head) — reviewed head 12176257 — reviewer summarizer:hermes/gpt-5.6-terra over coverage+credentials+deployment+general+lifecycle+prose.

terra
Advisory findings from outerloop — the code owner decides. Reply to disagree; the outerloop:no-review label opts this PR out.

Verdict: nothing blocking — 2 advisory notes.

2 findings attached to the lines below.

Verdict: 2 advisory findings, no blocking findings. The general lens found a missing default explicit-stop refusal-path test. The prose lens supplied a clearer replacement comment. Rejected: none.

Comment thread tests/test_orchestrator.py Resolved
Comment thread tests/test_orchestrator.py Resolved
@renmengye renmengye added outerloop:review re-request the advisory review and removed outerloop:review re-request the advisory review labels Oct 23, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Round 3 — reviewed head ab1b0678 — reviewer hermes/gpt-5.6-terra.

terra
Advisory findings from outerloop — the code owner decides. Reply to disagree; the outerloop:no-review label opts this PR out.

Verdict: no defects found.

@renmengye
renmengye merged commit b29ab74 into main Oct 23, 2026
5 checks passed
@renmengye
renmengye deleted the maint/tests-orchestrator branch October 23, 2026 05:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

outerloop:review re-request the advisory review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant