Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 8 additions & 7 deletions .github/workflows/advisory-review-agent.yml
Original file line number Diff line number Diff line change
Expand Up @@ -62,18 +62,19 @@ on:
required: false
default: false
backend:
description: Review backend (claude | hermes | codex).
description: Review backend (codex | claude | hermes).
type: string
required: false
default: claude
default: codex
Comment thread
renmengye marked this conversation as resolved.
Resolved
model:
description: >-
Model for the session. Empty uses the OUTERLOOP_CLAUDE_MODEL
Actions variable for claude (no built-in default; the backend's
own default otherwise).
Model for the session. Empty uses the kernel's pinned model for
codex (harnesses.toml, moved with releases), the
OUTERLOOP_CLAUDE_MODEL Actions variable for claude (no built-in
default), and hermes's own default.
For hermes pass an OpenRouter id — except with
hermes_provider: openai, which REQUIRES the provider-native id
(gpt-5.6-terra, no openai/ prefix).
(gpt-x.y, no openai/ prefix).
type: string
required: false
default: ''
Expand All @@ -82,7 +83,7 @@ on:
Credential provider for the hermes backend: openrouter (default) or
openai — openai-direct uses OPENAI_REVIEWER_KEY, avoids the
OpenRouter platform fee, and takes provider-NATIVE model ids
(gpt-5.6-terra, no openai/ prefix).
(gpt-x.y, no openai/ prefix).
type: string
required: false
default: openrouter
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/advisory-review-summarize.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,10 @@ on:
required: false
default: ''
backend:
description: Backend for the summarizer session (claude | hermes | codex).
description: Backend for the summarizer session (codex | claude | hermes).
type: string
required: false
default: claude
default: codex
model:
description: Model for the summarizer session (same rules as the reviewer).
type: string
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/maintenance-agent.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,9 @@ on:
workflow_call:
inputs:
backend:
description: Session backend — claude, codex or hermes
description: Session backend — codex, claude or hermes
type: string
default: claude
default: codex
model:
description: >-
Model id for the backend (hermes+openrouter needs the provider prefix,
Expand Down
3 changes: 0 additions & 3 deletions .github/workflows/maintenance.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,6 @@ jobs:
digest:
uses: outerloop-science/outerloop/.github/workflows/maintenance-agent.yml@main
with:
backend: hermes
hermes_provider: openai
model: gpt-5.6-terra
opinion_label: terra
secrets:
openai_reviewer_key: ${{ secrets.OPENAI_REVIEWER_KEY }}
8 changes: 4 additions & 4 deletions .github/workflows/review-agent.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,15 +15,15 @@ on:
backend:
description: Which agent backend drives the review
type: choice
options: [claude, codex, hermes]
default: claude
options: [codex, claude, hermes]
default: codex
model:
description: >-
Model for the session. The claude backend requires one: this input,
else OUTERLOOP_CLAUDE_MODEL in the environment (no built-in default).
Empty -> codex configured, hermes default. For hermes pass the
Empty -> the kernel's pinned codex model, hermes default. For hermes pass the
PROVIDER'S native id (openrouter: deepseek/deepseek-chat; openai:
gpt-5.6-terra).
gpt-x.y).
type: string
required: false
default: ''
Expand Down
12 changes: 1 addition & 11 deletions .github/workflows/review.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,11 +31,7 @@ jobs:
with:
bot_login: 'outerloop-science[bot]'
bot_aliases: 'outerloop-autoresearch[bot], agentic-learning-bot'
backend: hermes
# openai-direct: same token rate, no OpenRouter platform fee (the org's
# OpenAI account is tax-exempt); model id is provider-native
hermes_provider: openai
model: gpt-5.6-terra
# backend and model: the kernel's defaults (codex on its pinned model)
# `general` is a real lens name meaning the full rubric (no empty-string
# ternary — its GitHub Actions form silently falls through to the else)
lens: ${{ matrix.lens }}
Expand All @@ -51,9 +47,6 @@ jobs:
with:
bot_login: 'outerloop-science[bot]'
bot_aliases: 'outerloop-autoresearch[bot], agentic-learning-bot'
backend: hermes
hermes_provider: openai
model: gpt-5.6-terra
opinion_label: terra
# keep in lockstep with the lens matrix above
expected_lenses: general credentials deployment lifecycle coverage prose
Expand All @@ -67,9 +60,6 @@ jobs:
with:
bot_login: 'outerloop-science[bot]'
bot_aliases: 'outerloop-autoresearch[bot], agentic-learning-bot'
backend: hermes
hermes_provider: openai
model: gpt-5.6-terra
opinion_label: terra
# a maintainer's review label approves a fork PR; the fork's head is
# never checked out, but its diff reaches the model
Expand Down
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,13 @@ Versions follow [SemVer](https://semver.org).

## [Unreleased]

### Upgrading

- The reusable review, summary and maintenance workflows now default to the codex backend. A caller that relied on the old `claude` default must now set `backend: claude`; otherwise its reviews post a could-not-run stub until an `openai_reviewer_key` secret is passed.

### Changed

- The reviewer, summarizer and maintenance digest default to Codex, on a model the kernel pins next to the Codex CLI version (`model` in the `[codex]` table of `harnesses.toml`). A caller that leaves `model` unset follows the pin, so moving it in a release moves every such caller. A `model` input still wins.
- `merge: auto` merges a blessed head even while its author is still at work. Before, the merge waited until the run had no jobs in flight and no unread messages, so an author that kept revising on each panel round's advisory findings held its own cleared pull request open for days. A human's comment still holds the merge until the author has read it. Merging ends the run and cancels its jobs, as before; further work comes back as a new pull request.

## [0.6.0] - 2026-10-26
Expand Down
23 changes: 12 additions & 11 deletions docs/reviewer.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,21 +54,21 @@ jobs:
reviewer_ref: v0 # keep on the same ref as the line above
bot_login: my-bot # PRs by this login are never reviewed
# personal: true # personal mode: skip only this login's marked PRs
model: claude-sonnet-5-5 # required: the claude backend has no default
secrets:
anthropic_reviewer_key: ${{ secrets.ANTHROPIC_REVIEWER_KEY }}
openai_reviewer_key: ${{ secrets.OPENAI_REVIEWER_KEY }}
```

3. Add the key as a repository secret. Under
`Settings → Secrets and variables → Actions`, create
`ANTHROPIC_REVIEWER_KEY` with your Anthropic key. A dedicated, spend-capped
`OPENAI_REVIEWER_KEY` with your OpenAI key. A dedicated, spend-capped
key is strongly recommended.

Open a pull request and a review appears within a minute or two. The same
file ships as [`examples/review.yml`](https://github.com/outerloop-science/outerloop/blob/main/examples/review.yml).
The claude backend has no built-in model, so a review workflow needs `model`
or a repository variable `OUTERLOOP_CLAUDE_MODEL` it inherits; without one,
every PR gets a could-not-run stub.
The reviewer runs on Codex with the model Outerloop pins for it. A release
that moves the pin moves every workflow on `@v0` that leaves `model` unset,
so you get model upgrades with kernel releases. Set `model` to stay on a
model of your own choosing.

## Re-review and opt-out

Expand All @@ -79,16 +79,18 @@ label opts a PR out entirely.

## Backends and second opinions

`backend: claude` is the default. Two other backends work with the same
`backend: codex` is the default. Two other backends work with the same
workflow and their own key:

- `backend: claude` uses the `ANTHROPIC_REVIEWER_KEY` secret
(`anthropic_reviewer_key`). It has no built-in model: set `model`, or a
repository variable `OUTERLOOP_CLAUDE_MODEL` the workflow inherits;
without one, every PR gets a could-not-run stub.
- `backend: hermes` uses `hermes_provider: openrouter` by default with the
`OPENROUTER_API_KEY` secret and an OpenRouter model id such as
`openai/gpt-5.6-terra`. Add `hermes_provider: openai` to run directly
against the OpenAI API with the `OPENAI_REVIEWER_KEY` secret and a
provider-native id such as `gpt-5.6-terra`.
- `backend: codex` uses the `OPENAI_REVIEWER_KEY` secret and a Codex model
id.

Every input of the reusable workflow is documented in
[`advisory-review-agent.yml`](https://github.com/outerloop-science/outerloop/blob/main/.github/workflows/advisory-review-agent.yml).
Expand Down Expand Up @@ -162,9 +164,8 @@ jobs:
uses: outerloop-science/outerloop/.github/workflows/maintenance-agent.yml@v0
with:
reviewer_ref: v0
model: claude-sonnet-5-5 # required: the claude backend has no default
secrets:
anthropic_reviewer_key: ${{ secrets.ANTHROPIC_REVIEWER_KEY }}
openai_reviewer_key: ${{ secrets.OPENAI_REVIEWER_KEY }}
```

The `backend`, `model` and `hermes_provider` inputs select the model as they
Expand Down
8 changes: 4 additions & 4 deletions examples/review.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
# are skipped. The model key lives only in the read-only session job.
#
# Setup (see docs/reviewer.md): (1) have a bot account whose PRs are skipped,
# (2) add this file, (3) set the ANTHROPIC_REVIEWER_KEY repo secret.
# (2) add this file, (3) set the OPENAI_REVIEWER_KEY repo secret.
name: outerloop-review
on:
pull_request_target:
Expand All @@ -36,7 +36,7 @@ jobs:
bot_login: your-bot-login
# set it when the kernel runs with your own token (personal mode)
# personal: true
backend: claude # claude | hermes | codex
model: claude-sonnet-5-5 # required for claude: it has no default model
# backend: codex (default) | claude | hermes. Codex runs on the model
# this release pins unless you set `model`; claude needs `model`.
secrets:
anthropic_reviewer_key: ${{ secrets.ANTHROPIC_REVIEWER_KEY }}
openai_reviewer_key: ${{ secrets.OPENAI_REVIEWER_KEY }}
4 changes: 2 additions & 2 deletions src/outerloop/attempt.py
Original file line number Diff line number Diff line change
Expand Up @@ -312,7 +312,7 @@ def author_config_error(
if not model or model.startswith("claude"):
return (
"author-backend codex needs a codex/openai model in OUTERLOOP_AUTHOR_MODEL "
f"(e.g. gpt-5.6-terra), not a claude model or none (got {model!r})"
f"(e.g. gpt-x.y), not a claude model or none (got {model!r})"
)
return ""

Expand Down Expand Up @@ -5774,7 +5774,7 @@ def _run_id(value: str) -> str:
help="agent backend for the author/editor role (config-driven: default "
"from OUTERLOOP_AUTHOR_BACKEND). codex and hermes run contained (apptainer + "
"--sandbox danger-full-access) and REQUIRES --image and a codex/openai "
"--model (e.g. gpt-5.6-terra).",
"--model (e.g. gpt-x.y).",
)
parser.add_argument(
"--codex-config",
Expand Down
2 changes: 2 additions & 0 deletions src/outerloop/harnesses.toml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ linux-arm64 = "24fac77749bed3d91365d6b6915aa4b824e14318ecb6bc17adbc192f01c9173d"
version = "0.160.0"
sha256 = "306865417d4ee7a927785852910a527f41e1e159add390ac5ae3accb67d44a13"
code_mode_host_sha256 = "ac6cd6288f0e39f46a33eba1cfd37711651f14f33dbc73d6a1b82eb45e038cac"
# review sessions on codex use this model when the caller names none
model = "gpt-5.6-terra"

[hermes]
ref = "v2026.9.24"
Expand Down
7 changes: 5 additions & 2 deletions src/outerloop/review_agent_cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
from outerloop.endpoints import resolve_endpoint
from outerloop.github import EnvTokenProvider, GitHubClient
from outerloop.harness import ClaudeModelUnset, Harness
from outerloop.harness_pins import pins
from outerloop.review_agent import (
emit_envelope,
run_agent_review,
Expand Down Expand Up @@ -73,6 +74,8 @@ def resolve_reviewer_harness(spec: RoleSpec) -> tuple[Harness | None, str, str]:
return harness, "", backend
except ValueError as exc:
return None, str(exc), backend
if not review_model and backend == "codex":
review_model = pins("codex")["model"]
hermes_provider = os.environ.get("REVIEW_HERMES_PROVIDER", "").lower() or "openrouter"
key_var = {
"claude": "ANTHROPIC_REVIEWER_KEY",
Expand Down Expand Up @@ -108,14 +111,14 @@ def resolve_reviewer_harness(spec: RoleSpec) -> tuple[Harness | None, str, str]:
return (
None,
"hermes+openrouter requires an OpenRouter-shaped REVIEW_MODEL "
"(openai/gpt-5.6-terra, with the provider prefix)",
"(openai/gpt-x.y, with the provider prefix)",
backend,
)
if provider == "openai" and (not review_model or "/" in review_model):
return (
None,
"hermes+openai requires a provider-native REVIEW_MODEL "
"(gpt-5.6-terra, no openai/ prefix)",
"(gpt-x.y, no openai/ prefix)",
backend,
)
try:
Expand Down
2 changes: 2 additions & 0 deletions tests/test_review_agent_cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,8 @@ def test_codex_backend_reads_openai_key(monkeypatch: Any) -> None:
# the codex backend AND its openai key are forwarded to the harness builder
assert calls["build_kwargs"]["backend"] == "codex"
assert calls["build_key"] == "sk-openai"
# no REVIEW_MODEL: the kernel's pinned codex model, not codex's own default
assert calls["build_kwargs"]["model"] == cli.pins("codex")["model"]


def test_hermes_backend_reads_openrouter_key(monkeypatch: Any, tmp_path: Path) -> None:
Expand Down
Loading