Skip to content

Commit 8d4ebb4

Browse files
committed
docs: Ask When Risky after review — whole commands only, all three ratings, every run reviewed, typed goals only
The design note now says what the review fixes changed: a command over the reviewer's budget goes to the card unreviewed; an answer without an injection rating is unreadable; a pass is never reused; only a goal from the chat's input box is the person's word; a Settings change takes effect at once. What was run is re-counted on develop e528846.
1 parent 46a790b commit 8d4ebb4

1 file changed

Lines changed: 28 additions & 16 deletions

File tree

‎docs/ASK-WHEN-RISKY.md‎

Lines changed: 28 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -63,8 +63,11 @@ produce ratings, and `high` asks however the person was quoted.
6363

6464
### Every failure asks
6565

66-
No answer within 30 seconds, an error from the provider, an answer that is not the JSON above: the
67-
command goes to the person, and the card says the reviewer could not decide. A broken reviewer makes
66+
No answer within 30 seconds, an error from the provider, an answer that is not the JSON above (all three
67+
ratings, `injection` included — an answer that leaves it out has not said there was none): the
68+
command goes to the person, and the card says the reviewer could not decide. So does a command longer than
69+
the reviewer's 4,000-character budget, without a model call: the reviewer must see the whole line it vouches
70+
for, and a cut in the middle is exactly where something could hide. A broken reviewer makes
6871
the mode behave like Default Approvals, never like Autopilot. Stopping the run during a review ends
6972
it without asking and without running.
7073

@@ -86,9 +89,11 @@ Command line:
8689
<command>
8790
```
8891

89-
- **Only the PERSON section speaks for the person.** The host records the goals as they are typed
90-
(`humanGoals` in `extension.js`) and `agent.js` records ask_user choices; nothing is inferred from
91-
the transcript, which also holds the loop's own nudges and compaction summaries.
92+
- **Only the PERSON section speaks for the person.** The host records a goal only when it arrives from
93+
the chat's input box (`agentFlow(…, { typed: true })`, set by `handleSend` alone): a diagram retry,
94+
Retry after a resume and Continue pass words the person did not type here, and stay activity.
95+
`agent.js` records ask_user choices. Nothing is inferred from the transcript, which also holds the
96+
loop's own nudges and compaction summaries.
9297
- **Everything else is activity**, inside a fence with a per-request random id, so text in a file
9398
that looks like `PERSON` or `END ACTIVITY` stays inside it. Long entries are cut in the middle with
9499
`[... N characters left out ...]`, and the reviewer is told that what it cannot see is not safe by
@@ -132,7 +137,8 @@ Both new settings are **application-scoped**, and `readApprovalMode()` reads onl
132137
value: a folder's `.vscode/settings.json` is written by whoever wrote the repository and must not be
133138
able to loosen approvals. When `agent.approvals` has never been set, the older `agent.autopilot`
134139
boolean still means Autopilot, and choosing a mode keeps that boolean equal to "mode is Autopilot".
135-
(The old boolean itself is not application-scoped on the desktop; that is tracked separately.)
140+
A change in Settings, or in another window, takes effect on the next command, and the pill follows it
141+
(`onConfigChanged`).
136142

137143
## Where the idea came from
138144

@@ -168,22 +174,28 @@ What LevelCode does differently, on purpose:
168174
The first three rows follow from the first one: without a sandbox, a wrong "allow" runs on the
169175
person's real machine, so the model never gets the last word on anything that cannot be undone.
170176

171-
## What was run, and what was not (2026-10-19, macOS 27.0.1, Node 24)
177+
## What was run, and what was not (2026-10-19, macOS 27.0.1, Node 24, on develop e528846)
172178

173179
Run:
174-
- `scripts/test-extensions.sh`: all 129 suites, including `test/agentReview.test.js` (25: policy,
175-
parsing, the request's shape and bounds, every failure path) and `test/agentReviewGate.test.js` (16:
176-
the real `runAgent` → `run_command` chain in all three modes, with the provider and `spawn` stood in).
177-
- Ten single-defect mutations (the danger set reaching the reviewer, `high` running, injection
178-
ignored, a failed review running, the reviewer's cost dropped, the cache ignoring edits, a goal
179-
repeated as activity, ask_user choices not recorded, no timeout race, `delete_file` ungated): each
180-
turned its test red. The timeout mutation first passed, because a promise that never settles lets
181-
Node exit 0 midway; both suites now fail unless they reach their last test.
180+
- `scripts/test-extensions.sh`: all 131 suites, including `test/agentReview.test.js` (27: policy,
181+
parsing, the request's shape and bounds, every failure path), `test/agentReviewGate.test.js` (15: the
182+
real `runAgent` → `run_command` chain in all three modes, with the provider and `spawn` stood in),
183+
`test/agentReviewAnthropic.test.js` (the real dispatcher and native Anthropic adapter, only `fetch`
184+
stood in), and the host halves lifted into `sessionExpiredHost` and `diagramHost` (typed goals, the
185+
settings reload).
186+
- Sixteen single-defect mutations (the danger set reaching the reviewer, `high` running, injection
187+
ignored, a failed review running, the reviewer's cost dropped, a goal repeated as activity, ask_user
188+
choices not recorded, no timeout race, `delete_file` ungated, the Anthropic callback, a missing
189+
injection rating accepted, a long command reviewed, an untyped goal recorded, the send path not
190+
marking its goal, a diagram retry marked typed, a settings change ignored): each turned a test red.
191+
The timeout mutation first passed, because a promise that never settles lets Node exit 0 midway; the
192+
suites now fail unless they reach their last test.
182193
- `scripts/approvals-editor-check.js` in the dev build with a stand-in provider: 12 checks (the menu,
183194
the saved setting, a passed `ls -la` with no card, the `.env` upload stopped with the reviewer's
184195
reason, `rm -rf build` asking without a review, the README's planted instruction inside the fence,
185196
the count at the end, no page errors).
186-
- `tsc --checkJs` over `extension.js`, `agent.js` and `agentReview.js`: no new undefined names.
197+
- `tsc --checkJs` over `extension.js`, `agent.js`, `agentReview.js` and `providers/anthropic.js`: no new
198+
undefined names.
187199

188200
Not run:
189201
- **No real model has been the reviewer.** The stand-in answers with fixed ratings. How often a real

0 commit comments

Comments
 (0)