Skip to content

feat(pr-management): with-reviewers: runs model CLIs as adversarial reviewers - #1373

Merged
potiuk merged 3 commits into
feat/adversarial-review-pre-prfrom
feat/adversarial-review-code-review
Sep 24, 2026
Merged

potiuk merged 3 commits into
feat/adversarial-review-pre-prfrom
feat/adversarial-review-code-review

Conversation

@potiuk

@potiuk potiuk commented Sep 24, 2026

Copy link
Copy Markdown
Member

PR 4 of 4, stacked on #1372. This PR's base is feat/adversarial-review-pre-pr, so the diff shows only PR 4. Merge #1371 and #1372 first; this PR then retargets to main. It is the last of the series.

Summary

  • pr-management-code-review gains a tool path. with-reviewers:codex,copilot has the agent run the adversarial-review tool over each PR at Step 5. So does a configured adversarial-review.md whose mode is not off. The findings are folded in and attributed per reviewer. with-reviewer:<slash command> and "Review preferences" keep today's propose-and-type flow.
  • What the reviewers see. The tool reviews --target pr:<N> from an empty temporary directory, since the skill has no checkout of the PR's head and the maintainer's own checkout must not be readable by other models. A private repository's PR asks before the first run. The session announcement names the reviewers, so the maintainer knows where diffs go.
  • Resolution order. It is defined once, in prerequisites.md §2, and SKILL.md, selectors.md and adversarial.md point at it. Prefetched PRs get Step 5 from the parent.
  • Plan folded in. The implementation plan becomes the design's As built section and is deleted, per docs/designs/README.md. The design's status is now "Built".

Type of change

  • Skill change (.claude/skills/<name>/) — eval fixtures updated below
  • Documentation (docs/, README.md, CONTRIBUTING.md)

Test plan

  • prek run --all-files passes
  • For Python packages touched: tools/adversarial-review passes pytest (144 tests). A test now pins code-review's copy of the command, --repo-dir included, to the sandbox exclusion.
  • For skill behaviour changes: new eval suite pr-management-code-review/step-2-reviewer-resolution (8 cases). It covers:
    • with-reviewers:, with-reviewer:, and config on-demand / off;
    • no-adversarial;
    • a missing plugin, both with an explicit with-reviewer: and with a config-selected tool path.

RFC-AI-0004 compliance

  • HITL: the reviews are advisory, and the maintainer still picks the disposition and confirms every posted review.
  • Privacy LLM: reviewers see the public PR's diff, title and body, from an empty directory. A private repository asks first.
  • Vendor neutrality: any of the four model CLIs, or any slash-command reviewer.

Linked issues

Stacked on #1372 and #1371; follows #1368.

🤖 Generated with Claude Code

@potiuk potiuk added family:pr-management pr-management-* skills capability:review Deep per-item code review or contributor mentoring labels Sep 24, 2026
@potiuk
potiuk added this pull request to stack #1374 September 24, 2026 16:52
@potiuk
potiuk force-pushed the feat/adversarial-review-code-review branch 2 times, most recently from 0f31415 to e15a229 Compare September 24, 2026 17:14
…eviewers

pr-management-code-review gains a tool path next to the slash path:
`with-reviewers:codex,copilot` (or a configured adversarial-review.md whose
mode is not off) has the agent run the adversarial-review tool over the PR at
Step 5 and fold the findings in, attributed per reviewer. `with-reviewer:`
and Review preferences keep the propose-and-type flow. A private
repository's PR asks before the first tool run.

Eval suite pr-management-code-review/step-2-reviewer-resolution (6 cases).
The implementation plan is folded into the design's As built section and
deleted, per docs/designs/README.md.

Generated-by: Claude Opus 5
- the tool path reviews --target pr:<N> from an empty temporary directory
  created once per session: the skill has no checkout of the PR's head,
  and the maintainer's own checkout must not be readable by other models
  (a tracker checkout is refused outright); exit 2 skips the tool path
  for the session
- the announcement says where diffs go, rather than promising a prompt
  that auto-approving harness modes skip
- the plugin-missing fallthrough keeps an explicit with-reviewer:
- the resolution order is now the same in SKILL.md, selectors.md and
  adversarial.md as in prerequisites.md §2
- prefetched PRs get Step 5 from the parent (subagents have no shell)
- a test pins code-review's copy of the command to the sandbox exclusion;
  two more resolution eval cases

Generated-by: Claude Opus 5
@potiuk
potiuk force-pushed the feat/adversarial-review-code-review branch from e15a229 to 6f5abc4 Compare September 24, 2026 21:44
@potiuk
potiuk merged commit f3f90da into main Sep 24, 2026
10 checks passed
@potiuk
potiuk deleted the feat/adversarial-review-code-review branch September 24, 2026 21:50
@potiuk potiuk added substrate:framework-dev Tool substrate: build / validate / eval the framework itself substrate:review Tool substrate: second-model read of a change before publishing (read-only CLIs, merged findings) labels Sep 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

capability:review Deep per-item code review or contributor mentoring family:pr-management pr-management-* skills substrate:framework-dev Tool substrate: build / validate / eval the framework itself substrate:review Tool substrate: second-model read of a change before publishing (read-only CLIs, merged findings)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant