From 7fbdcf50fb80f83164dc2f911ba262b368ee29be Mon Sep 17 00:00:00 2001 From: Mengye Ren Date: Fri, 23 Oct 2026 09:52:24 -0400 Subject: [PATCH 1/4] README and getting-started point to the quickstart repository (#563) --- CHANGELOG.md | 1 + README.md | 13 ++++++++++++- docs/getting-started.md | 2 ++ 3 files changed, 15 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 93d63595..165ae137 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,7 @@ Versions follow [SemVer](https://semver.org). ## [Unreleased] +- The README and the getting-started guide point to the quickstart repository, two CPU benchmarks that run with the local forge and local compute: no GitHub account and no cluster (#563). - When check runs cannot be read on a token deployment, the warning now says the token cannot read them (fine-grained tokens have no checks permission) instead of asking for an App permission. ## [0.5.0] - 2026-10-23 diff --git a/README.md b/README.md index 3416c779..3cc29ee5 100644 --- a/README.md +++ b/README.md @@ -29,7 +29,18 @@ where it co-develops our research codebases. ## Get started -Three commands and one file. You need a repo with a benchmark command, an +To see a run first, the [quickstart](https://github.com/outerloop-science/quickstart) +climbs two CPU benchmarks on your own machine, with no GitHub account and no +cluster: + +```bash +pip install outerloop-science +outerloop init --forge local --compute local --target me/quickstart \ + --forge-source https://github.com/outerloop-science/quickstart +outerloop start +``` + +For your own repository: three commands and one file. You need a repo with a benchmark command, an API key for the model that will write the code, and a Slurm cluster or one machine with a GPU. diff --git a/docs/getting-started.md b/docs/getting-started.md index 5c903f70..0d4eb014 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -4,6 +4,8 @@ This page walks you through one Outerloop run end to end: install the package, d The tutorial uses the simplest supported setup: everything runs on your machine. [Install](install.md) covers Slurm clusters, upgrades and the other deployment shapes. +To watch a run before you write a contract, the [quickstart repository](https://github.com/outerloop-science/quickstart) has two CPU benchmarks and runs without GitHub or a cluster; its README has the three commands. + ## Before you start - Python 3.12 or later on a Linux machine, or any machine you can leave a terminal open on. From 7ee434ac0b070c1efe59f252da7f489f00237213 Mon Sep 17 00:00:00 2001 From: Mengye Ren Date: Fri, 23 Oct 2026 09:59:21 -0400 Subject: [PATCH 2/4] README quickstart command: --no-image --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 3cc29ee5..614cb743 100644 --- a/README.md +++ b/README.md @@ -35,7 +35,7 @@ cluster: ```bash pip install outerloop-science -outerloop init --forge local --compute local --target me/quickstart \ +outerloop init --forge local --compute local --no-image --target me/quickstart \ --forge-source https://github.com/outerloop-science/quickstart outerloop start ``` From a07074e84033d4d3426cdca544bf3f886afbc814 Mon Sep 17 00:00:00 2001 From: Mengye Ren Date: Fri, 23 Oct 2026 10:31:52 -0400 Subject: [PATCH 3/4] Docs quickstart pointer: name uv, Git and the key; note the uncontained session and a spend-capped key; plainer wording --- CHANGELOG.md | 2 +- README.md | 8 ++++++-- 2 files changed, 7 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 165ae137..4aed70e3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,7 +6,7 @@ Versions follow [SemVer](https://semver.org). ## [Unreleased] -- The README and the getting-started guide point to the quickstart repository, two CPU benchmarks that run with the local forge and local compute: no GitHub account and no cluster (#563). +- The README and the getting-started guide link to the quickstart repository. It runs two CPU benchmarks with the local forge and local compute, and needs no GitHub account or cluster (#563). - When check runs cannot be read on a token deployment, the warning now says the token cannot read them (fine-grained tokens have no checks permission) instead of asking for an App permission. ## [0.5.0] - 2026-10-23 diff --git a/README.md b/README.md index 614cb743..c3cd73e1 100644 --- a/README.md +++ b/README.md @@ -30,8 +30,9 @@ where it co-develops our research codebases. ## Get started To see a run first, the [quickstart](https://github.com/outerloop-science/quickstart) -climbs two CPU benchmarks on your own machine, with no GitHub account and no -cluster: +runs two CPU benchmarks on your own machine, with no GitHub account and no +cluster. You need [uv](https://docs.astral.sh/uv/), Git 2.38 or later and an +Anthropic API key: ```bash pip install outerloop-science @@ -40,6 +41,9 @@ outerloop init --forge local --compute local --no-image --target me/quickstart \ outerloop start ``` +Without a container the author's session runs as your user under Claude Code's +own sandbox, so give it a spend-capped key. + For your own repository: three commands and one file. You need a repo with a benchmark command, an API key for the model that will write the code, and a Slurm cluster or one machine with a GPU. From bc4d5a2fcd48939d66872befa2f3cd429829b194 Mon Sep 17 00:00:00 2001 From: Mengye Ren Date: Fri, 23 Oct 2026 12:32:46 -0400 Subject: [PATCH 4/4] Uncontained sessions: say what they get (scrubbed environment, own home, account-readable files), not a sandbox; quickstart names Python 3.12 --- CHANGELOG.md | 2 +- README.md | 8 ++++---- docs/install.md | 4 ++-- docs/reference/environment.md | 4 ++-- src/outerloop/tick.py | 8 ++++---- 5 files changed, 13 insertions(+), 13 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index eed1d79c..97fa46d5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,7 +6,7 @@ Versions follow [SemVer](https://semver.org). ## [Unreleased] -- The README and the getting-started guide link to the quickstart repository. It runs two CPU benchmarks with the local forge and local compute, and needs no GitHub account or cluster (#563). +- The README and the getting-started guide link to the quickstart repository. It runs two CPU benchmarks with the local forge and local compute, and needs no GitHub account or cluster (#563). The docs and the startup warning no longer say an uncontained session runs in a sandbox: it gets a scrubbed environment and its own home, but can read any file the account can. - The advisory review covers exactly the head its triggering event named (`REVIEW_HEAD_SHA`, #462). If the PR has moved off that head, nothing is reviewed or posted. Otherwise the diff is read between fixed commits, every envelope records the reviewed `head_sha`, and the review is attached to that commit, with no fallback comment once the head has moved. The summarizer skips clean when its lenses read different heads. For a fork PR (`REVIEW_CHECKOUT_BASE=1`) the brief says the tree is the base branch and the change is only in the diff. Upgrading: envelopes without `head_sha` (from an older reviewer ref) still post; callers that do not pass REVIEW_HEAD_SHA behave as before. - When check runs cannot be read on a token deployment, the warning now says the token cannot read them (fine-grained tokens have no checks permission) instead of asking for an App permission. diff --git a/README.md b/README.md index c3cd73e1..fe6ca928 100644 --- a/README.md +++ b/README.md @@ -31,8 +31,8 @@ where it co-develops our research codebases. To see a run first, the [quickstart](https://github.com/outerloop-science/quickstart) runs two CPU benchmarks on your own machine, with no GitHub account and no -cluster. You need [uv](https://docs.astral.sh/uv/), Git 2.38 or later and an -Anthropic API key: +cluster. You need Python 3.12 or later, [uv](https://docs.astral.sh/uv/), Git +2.38 or later and an Anthropic API key: ```bash pip install outerloop-science @@ -41,8 +41,8 @@ outerloop init --forge local --compute local --no-image --target me/quickstart \ outerloop start ``` -Without a container the author's session runs as your user under Claude Code's -own sandbox, so give it a spend-capped key. +Without a container the author's session runs as your user and can read any file +your account can, so give it a spend-capped key. For your own repository: three commands and one file. You need a repo with a benchmark command, an API key for the model that will write the code, and a Slurm cluster or one diff --git a/docs/install.md b/docs/install.md index 08b59ea8..6497846a 100644 --- a/docs/install.md +++ b/docs/install.md @@ -46,7 +46,7 @@ With `OUTERLOOP_COMPUTE=local`, or with no `sbatch` on PATH, `outerloop start` r Local compute has no lanes, so GPU benchmarks need no GPU partition: evaluations and the author's launches run on the machine's own GPUs, shared first come first served. Launch arrays run their tasks in parallel as GPUs become available, and `OUTERLOOP_LOCAL_GPUS` overrides automatic detection through `nvidia-smi` (`0` disables GPU allocation). The pool also follows the loop's own `CUDA_VISIBLE_DEVICES`: start each deployment with its own list (for example `0,1` and `2,3`) to give it its own GPUs. The author's session itself starts with `CUDA_VISIBLE_DEVICES` empty, so an experiment that needs a GPU goes through the author's `launch` syscall, where it is recorded and metered against `gpu_hours_per_run`. Uncontained, that is a default the session could reset; containment is what keeps the GPU out of the session, which is one more reason to run with the image. -Without a container image, sessions run under the harness's own sandbox and evaluations run bare on your machine with your keys, and the loop says so once at start. The verification panel is off in this mode unless you set `OUTERLOOP_PANEL_UNCONTAINED=1`, because an uncontained judge holds a shell next to its own key file. A pull request opened without a panel says so. A Codex or Hermes author needs the image in every mode. For priorities, walltime accounting, or several machines, use Slurm. +Without a container image, sessions and evaluations run as your account on your machine, and the loop says so once at start. A session gets a scrubbed environment and its own home directory, but it can read any file your account can, keys included, so use a spend-capped key. The verification panel is off in this mode unless you set `OUTERLOOP_PANEL_UNCONTAINED=1`, because an uncontained judge holds a shell next to its own key file. A pull request opened without a panel says so. A Codex or Hermes author needs the image in every mode. For priorities, walltime accounting, or several machines, use Slurm. ### One machine as its own Slurm cluster @@ -214,7 +214,7 @@ Only the local loop falls back to running uncontained on its own. On Slurm the t ### Running a cluster without a container -A Slurm cluster with no container runtime sets `OUTERLOOP_UNCONTAINED=1` in `~/.config/outerloop/.env`. Sessions and evaluations then run as local mode does without an image, but as Slurm jobs on your account. Sessions run under the harness's own sandbox. Evaluations run bare, in a throwaway tree with an allowlisted environment. The image is ignored even when one is on disk. +A Slurm cluster with no container runtime sets `OUTERLOOP_UNCONTAINED=1` in `~/.config/outerloop/.env`. Sessions and evaluations then run as local mode does without an image, but as Slurm jobs on your account. Sessions get a scrubbed environment and their own home directory but can read any file the account can. Evaluations run bare, in a throwaway tree with an allowlisted environment. The image is ignored even when one is on disk. The local-mode rules carry over. The panel is off unless you set `OUTERLOOP_PANEL_UNCONTAINED=1`, and `start` refuses a Codex or Hermes author. A session can read any file your account can: the bot's credential, the model key files, the evaluator's tree and the kernel's state. Key secrecy and measurement integrity both weaken: the benchmark command can write its own eval directory, and on a multi-node benchmark any node can write every node's result files, so a measurement is only as trustworthy as the author whose code ran. Use it where the site already confines processes, or where that exposure is acceptable to you. diff --git a/docs/reference/environment.md b/docs/reference/environment.md index 2b601b91..a88a069b 100644 --- a/docs/reference/environment.md +++ b/docs/reference/environment.md @@ -214,8 +214,8 @@ and Codex and Hermes authors need one in every mode. ### `OUTERLOOP_UNCONTAINED` -`1` runs a Slurm deployment without a container: sessions under the -harness's own sandbox, evaluations bare, as jobs on your account. The image +`1` runs a Slurm deployment without a container: sessions with a scrubbed +environment and a per-run home, evaluations bare, as jobs on your account. The image is ignored. A session can then read any file the account can, keys included. See [running a cluster without a container](../install.md#running-a-cluster-without-a-container). diff --git a/src/outerloop/tick.py b/src/outerloop/tick.py index a16d22bb..65983b93 100644 --- a/src/outerloop/tick.py +++ b/src/outerloop/tick.py @@ -3732,8 +3732,8 @@ def _service_spec_from_env( panel = os.environ.get("OUTERLOOP_PANEL", "verify,review") uncontained = uncontained_opt_in() if uncontained or (not image_ok and local_mode()): - # No container: sessions run under the harness's own sandbox and - # evaluations run bare, with the operator's own account and keys. The + # No container: sessions run as the operator's account with a scrubbed + # environment and a per-run home, and evaluations run bare. The # local loop falls back to this when it has no image; any other # deployment needs OUTERLOOP_UNCONTAINED=1, so a missing image on a # cluster never silently drops containment. The panel is off unless @@ -3744,8 +3744,8 @@ def _service_spec_from_env( if not _UNCONTAINED_WARNED: _UNCONTAINED_WARNED = True log.warning( - "%s; sessions run under the harness sandbox and evaluations run bare " - "as this account; the panel is %s; a codex or hermes author needs the " + "%s; sessions and evaluations run as this account and can read its " + "files; the panel is %s; a codex or hermes author needs the " "image (docs/install.md, running without a container)", "OUTERLOOP_UNCONTAINED=1: no container" if uncontained