From 4a80ed9231fd70d608ece51ac583c64d012c0534 Mon Sep 17 00:00:00 2001 From: igalshilman Date: Wed, 30 Sep 2026 10:48:34 +0200 Subject: [PATCH] Add approvals and a sub-agent fleet to the README video The session now also shows a guardrail holding the agent's file writes for a person, approved from the UI's Approvals panel, and a fleet of four sub-agents researching at once, each child's own page in a 2x2 grid, before the kill -9 act. The recorder zooms the UI further for legibility, fast-forwards at 2x while the turn only waits on a model, holds on answers long enough to read them, steers only after the program's searches are recorded, and encodes a limited-range yuv420p file that every player handles. Co-Authored-By: Claude Opus 5.5 (1M context) --- .misc/video/README.md | 29 ++-- .misc/video/director.html | 73 ++++++++-- .misc/video/encode.sh | 11 +- .misc/video/record.mjs | 282 ++++++++++++++++++++++++++++++++------ README.md | 7 +- 5 files changed, 329 insertions(+), 73 deletions(-) diff --git a/.misc/video/README.md b/.misc/video/README.md index 5a28b6d..e13aa5b 100644 --- a/.misc/video/README.md +++ b/.misc/video/README.md @@ -1,11 +1,17 @@ # README video: a real session -A real recording of the reference agent, in two acts: +A real recording of the reference agent, in three acts: 1. **An ordinary turn.** A casual request for a weekend plan. The model - writes a program whose three web searches run at once, saves `lisbon.md` - in its sandbox, and is steered while it works. -2. **Kill it mid-turn.** A follow-up waits on a 30-second durable timer. The + writes a program whose three web searches run at once and is steered + while it works. The agent has a guardrail that makes file writes wait for + a person, so saving `lisbon.md` asks for approval in the UI's Approvals + panel, and the script approves it. +2. **A fleet of sub-agents.** The agent hands four cities to four + sub-agents, which research at the same time. Each child is a whole agent, + so the script opens each child's own page, read-only, in a 2×2 grid, then + returns to the parent's combined answer. +3. **Kill it mid-turn.** A follow-up waits on a 30-second durable timer. The service is killed with `kill -9`, started again, and Restate replays the turn's journal, so the turn reads the file back and answers. @@ -19,12 +25,17 @@ when the UI or the runtime changes. | `cdp.mjs` | A tiny Chrome DevTools Protocol client (Node 22+, no npm packages) | | `encode.sh` | Turns the captured frames into `out/agent-demo.mp4` | -Everything on screen comes from the running system except the titles and -captions: the UI is streamed from its own tab, the terminal shows the +Everything on screen comes from the running system except the titles, +captions and the highlight on the Approve button before it is clicked: the UI is streamed from its own tab, the terminal shows the service's real log lines, and the journal panel is `sys_journal`. The quiet stretches (model calls after the steer, the service down, the timer running) play faster, shown by the badge in the header. +The web UI renders 800 CSS pixels wide, its one-column layout, zoomed in so +the text stays legible. To fit the frame, the script crops off the agent +picker, injects one rule that shortens the conversation pane, and slides +the view down to the Approvals panel while a decision is pending. + ## Record You need Docker (or `restate-server`), Google Chrome, ffmpeg, and an @@ -60,12 +71,14 @@ ffmpeg -i .misc/video/out/agent-demo.mp4 -vf fps=1/3 /tmp/frames/%02d.png Check that each ask went through at once (the prompt must not sit in the composer), that the steer landed while the turn ran (the script stops if it -did not), that the kill happened while only the timer was open, that the +did not), that each approval showed its card before the click, that all four +sub-agent tiles filled and finished, that the +kill happened while only the timer was open, that the journal marks only finished steps as replayed, and that the Restate UI scene shows the 30-second `sleep` row. The model's wording differs between takes, so read its answers too. -A take runs about 75 seconds. If it runs much longer, look for leftover +A take runs about two minutes. If it runs much longer, look for leftover headless Chrome processes (`pgrep -f readme-video-chrome`): they slow the UI down enough to delay the ask. diff --git a/.misc/video/director.html b/.misc/video/director.html index d9e271e..4e1157f 100644 --- a/.misc/video/director.html +++ b/.misc/video/director.html @@ -37,7 +37,7 @@ /* Header */ header { - position: absolute; left: 40px; right: 40px; top: 26px; height: 56px; + position: absolute; left: 40px; right: 40px; top: 18px; height: 52px; display: flex; align-items: center; gap: 18px; } .logo { @@ -74,15 +74,15 @@ .dots i { width: 12px; height: 12px; border-radius: 50%; background: #2c2c3c; display: block; } .panel-head .right { margin-left: auto; } - #ui { left: 40px; top: 104px; width: 1100px; height: 812px; } - #ui .view { width: 1100px; height: 768px; overflow: hidden; } - #ui img { display: block; width: 1100px; visibility: hidden; } + #ui { left: 40px; top: 84px; width: 1240px; height: 872px; } + #ui .view { width: 1240px; height: 828px; overflow: hidden; } + #ui img { display: block; width: 1240px; visibility: hidden; transition: transform 0.7s cubic-bezier(.4,0,.2,1); } - #term { left: 1164px; top: 104px; width: 716px; height: 380px; transition: box-shadow 0.3s, border-color 0.3s; } + #term { left: 1304px; top: 84px; width: 576px; height: 400px; transition: box-shadow 0.3s, border-color 0.3s; } #term.dead { border-color: #ff4d5e88; box-shadow: 0 0 0 1px #ff4d5e55, 0 24px 60px #ff4d5e22; } #term-body { - padding: 16px 20px; font: 15px/1.55 var(--mono); color: #cfcfe0; - height: 336px; overflow: hidden; display: flex; flex-direction: column; justify-content: flex-end; + padding: 14px 18px; font: 14px/1.55 var(--mono); color: #cfcfe0; + height: 356px; overflow: hidden; display: flex; flex-direction: column; justify-content: flex-end; } .ln { white-space: pre; overflow: hidden; text-overflow: ellipsis; } .ln.cmd { color: var(--text); } @@ -94,6 +94,7 @@ @keyframes blink { 50% { opacity: 0; } } .pill { + white-space: nowrap; font: 700 13px var(--mono); padding: 5px 12px; border-radius: 999px; display: inline-flex; align-items: center; gap: 8px; letter-spacing: 0.04em; } @@ -104,10 +105,10 @@ .pill.starting { color: var(--amber); background: #ffb5471a; } @keyframes pulse { 50% { opacity: 0.2; } } - #journal { left: 1164px; top: 508px; width: 716px; height: 408px; } + #journal { left: 1304px; top: 508px; width: 576px; height: 448px; } #journal-body { padding: 10px 14px; } .jrow { - display: grid; grid-template-columns: 34px 1fr 150px; align-items: center; + display: grid; grid-template-columns: 40px 1fr 130px; align-items: center; height: 34px; padding: 0 10px; border-radius: 9px; font: 15px var(--mono); animation: pop 0.35s cubic-bezier(.2,1.4,.4,1); transition: background 0.25s; @@ -131,7 +132,7 @@ /* Caption */ #caption { - position: absolute; left: 40px; right: 40px; bottom: 30px; height: 112px; + position: absolute; left: 40px; right: 40px; bottom: 12px; height: 100px; display: flex; align-items: center; gap: 22px; padding: 0 28px; } #caption .n { @@ -145,9 +146,23 @@ @keyframes swap { from { opacity: 0; transform: translateY(10px); } } /* Full-width Restate UI */ - #restate { left: 40px; top: 104px; width: 1840px; height: 812px; opacity: 0; transition: opacity 0.6s; } + #restate { left: 40px; top: 84px; width: 1840px; height: 872px; opacity: 0; transition: opacity 0.6s; } #restate.on { opacity: 1; } - #restate img { display: block; width: 1840px; height: 768px; } + #restate img { display: block; width: 1840px; height: 828px; } + + /* The sub-agent fleet: one live page per child */ + #fleet { + position: absolute; left: 40px; top: 84px; width: 1840px; height: 872px; + display: grid; grid-template-columns: 1fr 1fr; grid-template-rows: 1fr 1fr; gap: 16px; + opacity: 0; transition: opacity 0.6s; pointer-events: none; + } + #fleet.on { opacity: 1; } + #fleet .panel { position: relative; } + #fleet .panel-head { color: var(--text); } + #fleet .panel-head .who { font: 700 16px var(--sans); } + #fleet .view { width: 912px; height: 384px; overflow: hidden; } + #fleet img { display: block; width: 912px; } + #fleet .tile.done { border-color: #3ddc9766; } /* Title cards */ #card { @@ -190,7 +205,7 @@

Restate reference agent

-
agent service · packages/libs/coreRUNNING
+
agent serviceRUNNING
@@ -205,6 +220,13 @@

Restate reference agent

+
+
WORKING
+
WORKING
+
WORKING
+
WORKING
+
+
1

@@ -237,12 +259,33 @@

Restate reference agent

$("speed").classList.toggle("on", factor > 1); }, - uiCrop(px) { - $("ui-img").style.marginTop = `-${px}px`; + // Shows the web UI from this many pixels below its top; a change slides. + uiView(px) { + $("ui-img").style.transform = `translateY(-${px}px)`; $("ui-img").style.visibility = "visible"; }, uiFrame(data) { $("ui-img").src = `data:image/jpeg;base64,${data}`; }, restateFrame(data) { $("restate-img").src = `data:image/jpeg;base64,${data}`; }, + // The fleet grid: a tile per sub-agent, showing its page from `top` down. + fleetTile(index, name, top) { + const tile = document.querySelectorAll("#fleet .tile")[index]; + tile.querySelector(".who").textContent = `sub-agent · ${name}`; + tile.querySelector("img").style.marginTop = `-${top}px`; + }, + fleetFrame(index, data) { + document.querySelectorAll("#fleet img")[index].src = `data:image/jpeg;base64,${data}`; + }, + fleetStatus(index, done) { + const tile = document.querySelectorAll("#fleet .tile")[index]; + const pill = tile.querySelector(".pill"); + pill.className = `pill ${done ? "running" : "starting"}`; + pill.textContent = done ? "✓ DONE" : "WORKING"; + tile.classList.toggle("done", done); + }, + showFleet(on) { + $("fleet").classList.toggle("on", on); + for (const id of ["ui", "term", "journal"]) $(id).style.opacity = on ? 0 : 1; + }, showRestate(on) { $("restate").classList.toggle("on", on); for (const id of ["ui", "term", "journal"]) $(id).style.opacity = on ? 0 : 1; diff --git a/.misc/video/encode.sh b/.misc/video/encode.sh index 52a0ec7..74b8403 100755 --- a/.misc/video/encode.sh +++ b/.misc/video/encode.sh @@ -16,8 +16,15 @@ node -e ' require("fs").writeFileSync(process.argv[1] + "/frames.ffconcat", lines.join("\n") + "\n"); ' "$out" +# The frames are full-range JPEGs; convert to limited-range yuv420p at level +# 4.1, which every player handles (QuickTime shows a still for yuvj420p). +# Encode to a temporary file and rename it, so a player that has the old +# video open never reads a half-written one. ffmpeg -y -loglevel error -f concat -safe 0 -i "$out/frames.ffconcat" \ - -vf "fps=30,format=yuv420p" -c:v libx264 -preset slow -crf 20 \ - -movflags +faststart "$out/agent-demo.mp4" + -vf "fps=30,scale=out_range=tv:out_color_matrix=bt709,format=yuv420p" \ + -c:v libx264 -preset slow -crf 20 -profile:v high -level:v 4.1 \ + -color_range tv -colorspace bt709 -color_primaries bt709 -color_trc bt709 \ + -movflags +faststart "$out/agent-demo.tmp.mp4" +mv "$out/agent-demo.tmp.mp4" "$out/agent-demo.mp4" ls -lh "$out/agent-demo.mp4" diff --git a/.misc/video/record.mjs b/.misc/video/record.mjs index 1783273..f2fc777 100644 --- a/.misc/video/record.mjs +++ b/.misc/video/record.mjs @@ -25,6 +25,7 @@ const OUT = resolve(process.argv[2] ?? join(HERE, "out")); const AGENT_ID = process.env.AGENT_ID ?? `demo-${Date.now().toString(36)}`; const UI_URL = process.env.UI_URL ?? "http://127.0.0.1:3000"; const ADMIN_URL = process.env.RESTATE_ADMIN_URL ?? "http://localhost:9070"; +const INGRESS_URL = process.env.RESTATE_INGRESS_URL ?? "http://localhost:8080"; const SLEEP_SECONDS = 30; const ASK = @@ -32,6 +33,14 @@ const ASK = "the weekend forecast, the top museums and the best food markets all at once, " + "then save a short plan to lisbon.md."; const STEER = "Make it vegetarian-friendly, and add a rainy-day option."; +const GUARDRAIL = { + id: "approve-writes", + rule: "Writing or changing a file in the workspace needs a person's approval first.", +}; +const FLEET = + "Now compare four alternatives to Lisbon for a spring weekend: give Porto, Seville, Valencia " + + "and Barcelona each to its own sub-agent to research at the same time (web search only, " + + "no files, three lines each). Then tell me which one beats Lisbon."; const FOLLOW_UP = `Give me ${SLEEP_SECONDS} seconds to run it by my partner: sleep for ${SLEEP_SECONDS} seconds, ` + "then read lisbon.md back and sum it up in three lines."; @@ -77,7 +86,12 @@ function startService() { return service.pid; } -const SHOWN_HANDLERS = ["ask", "steer", "doTurn"]; +const SHOWN_HANDLERS = ["ask", "steer", "resolveApproval", "doTurn"]; +const SHORT_MESSAGES = { + "Starting invocation.": "started", + "Invocation completed successfully.": "completed", + "Replaying invocation.": "replaying the journal", +}; // "[restate][2026-...Z][AgentSession/demo/doTurn][inv_...] INFO: Replaying invocation." function formatServiceLine(line) { @@ -88,7 +102,7 @@ function formatServiceLine(line) { const handler = target.split("/").pop(); if (!SHOWN_HANDLERS.includes(handler)) return null; const service = target.startsWith("AgentSession") ? "AgentSession" : "Agent"; - const text = `${time} ${service}.${handler} ${message}`; + const text = `${time} ${service}.${handler} ${SHORT_MESSAGES[message] ?? message}`; return {text, kind: message.startsWith("Replaying") ? "hot" : ""}; } @@ -124,6 +138,8 @@ async function journalEntries(turnId) { // Runs the turn journals for itself rather than for a tool. const STEPS = { "agent-model": "model call", + "guardrail-model": "guardrail check", + "guardrail-review": "guardrail review", "discover-agent-tools": "tool catalog", provisionSandbox: "sandbox start", resumeSandbox: "sandbox resume", @@ -133,8 +149,12 @@ const STEPS = { // Turns journal entries into panel rows. A run counts as recorded once as many // run results as runs up to it are in the journal; parallel runs finish within // milliseconds of each other, so the order does not show. +// What each signal the turn receives is, in the order this script sends them. +const signalNames = []; + function journalRows(entries, replay) { const rows = []; + let signals = 0; const runResults = entries.filter((entry) => entry.entry_type === "Notification: Run").length; const sleepDone = entries.some((entry) => entry.entry_type === "Notification: Sleep"); let runs = 0; @@ -145,7 +165,7 @@ function journalRows(entries, replay) { } else if (entry.entry_type === "Command: Output") { Object.assign(row, {label: "output", name: "turn finished"}); } else if (entry.entry_type === "Notification: Signal") { - Object.assign(row, {label: "signal", name: "steer", status: "✓ received"}); + Object.assign(row, {label: "signal", name: signalNames[signals++] ?? "", status: "✓ received"}); } else if (entry.entry_type === "Command: Run") { runs++; const recorded = runs <= runResults; @@ -158,7 +178,7 @@ function journalRows(entries, replay) { } else if (entry.entry_type === "Command: Sleep") { Object.assign(row, { label: "timer", - name: `sleep ${SLEEP_SECONDS}s`, + name: "sleep", state: sleepDone ? "done" : "wait", status: sleepDone ? "✓ fired" : "⏱ in Restate", }); @@ -188,26 +208,46 @@ async function main() { const say = (expression) => stage.evaluate(expression); // Both UIs render zoomed in, so they stay legible in a README-sized player. - // The web UI's agent picker is cropped off the top: the conversation is the - // part the video is about. - const uiScale = 1100 / 900; + // The web UI renders 800 CSS pixels wide, its one-column layout, where the + // Approvals panel sits under the conversation. The recording shortens the + // conversation pane to fit the frame and crops off the agent picker; the + // view slides down to the Approvals panel when a decision is pending. + const UI_WIDTH = 800; + const uiScale = 1240 / UI_WIDTH; + const uiVisible = 828 / uiScale; // The UI opens only once the service runs (its default agent is "demo" too). // Opened earlier, its calls to the agent would wait in Restate's retry // backoff, and the first ask behind them. - const ui = await browser.newTab({url: "about:blank", width: 900, height: 900, scale: uiScale}); + const ui = await browser.newTab({url: "about:blank", width: UI_WIDTH, height: 1400, scale: uiScale}); + let conversationTop = 0; async function openUi() { await ui.goto(`${UI_URL}/?agent=${AGENT_ID}`); - const pickerHeight = await ui.waitFor( - `[...document.querySelectorAll("button")].find((b) => b.textContent.trim() === "Open agent")?.getBoundingClientRect().bottom`, + await ui.waitFor(`Boolean(document.querySelector(".conversation-pane"))`); + await ui.evaluate(`(() => { + const style = document.createElement("style"); + style.textContent = ".conversation-pane { height: ${Math.floor(uiVisible) - 8}px !important; min-height: 0 !important; }"; + document.head.appendChild(style); + })()`); + conversationTop = await ui.evaluate( + `document.querySelector(".conversation-pane").getBoundingClientRect().top + scrollY - 4`, ); - const cropCss = pickerHeight + 13; - await ui.send("Emulation.setDeviceMetricsOverride", { - width: 900, - height: Math.round(768 / uiScale + cropCss), - deviceScaleFactor: uiScale, - mobile: false, - }); - await say(`stage.uiCrop(${Math.round(cropCss * uiScale)})`); + await say(`stage.uiView(${Math.round(conversationTop * uiScale)})`); + } + // Slides the view so the Approvals panel's buttons are in frame. + async function showApprovals() { + const bottom = await ui.evaluate( + `[...document.querySelectorAll("button")].find((b) => b.textContent.trim() === "Approve").getBoundingClientRect().bottom + scrollY`, + ); + await say(`stage.uiView(${Math.round((bottom + 24 - uiVisible) * uiScale)})`); + } + const showConversation = () => say(`stage.uiView(${Math.round(conversationTop * uiScale)})`); + // Links to child agents appear above the conversation once there are + // children; measure where it starts again. + async function recrop() { + conversationTop = await ui.evaluate( + `document.querySelector(".conversation-pane").getBoundingClientRect().top + scrollY - 4`, + ); + await showConversation(); } const restateScale = 1.25; const restateUi = await browser.newTab({ @@ -218,7 +258,7 @@ async function main() { }); // Stream a tab into the director. Only the newest frame is forwarded. - function mirror(tab, method) { + function mirror(tab, show) { let latest; let busy = false; tab.on("Page.screencastFrame", async ({data, sessionId}) => { @@ -229,7 +269,7 @@ async function main() { while (latest) { const frame = latest; latest = undefined; - await say(`stage.${method}(${js(frame)})`); + await say(show(js(frame))); } busy = false; }); @@ -244,10 +284,13 @@ async function main() { frames.push({file, time: videoTime(metadata.timestamp)}); }); - await mirror(ui, "uiFrame"); - await mirror(restateUi, "restateFrame"); + await mirror(ui, (frame) => `stage.uiFrame(${frame})`); + await mirror(restateUi, (frame) => `stage.restateFrame(${frame})`); + let speed = 1; async function setSpeed(factor) { + if (factor === speed) return; + speed = factor; speedSegments.push({at: Date.now() / 1000, factor}); await say(`stage.speed(${factor})`); } @@ -303,15 +346,38 @@ async function main() { // The transcript stops following new events once they push it past the // fold; press its "Latest" button, as a viewer would. + const childTabs = []; const follower = (async () => { while (polling) { - await ui.evaluate( - `[...document.querySelectorAll("button")].find((b) => b.textContent.trim() === "Latest")?.click()`, - ); + for (const tab of [ui, ...childTabs]) { + await tab.evaluate( + `[...document.querySelectorAll("button")].find((b) => b.textContent.trim() === "Latest")?.click()`, + ); + } await pause(400); } })(); + // Waiting on the model plays faster: while fastForward is on and the only + // open step of the turn is a model call, the video runs at 2×, and the + // header badge says so. + const THINKING = ["agent-model", "guardrail-model", "guardrail-review"]; + let fastForward = false; + const thinking = () => { + const runs = entries.filter((entry) => entry.entry_type === "Command: Run"); + return runs.length > count(entries, "Notification: Run") && THINKING.includes(runs.at(-1).name); + }; + const speeder = (async () => { + while (polling) { + if (fastForward) await setSpeed(thinking() ? 2 : 1); + await pause(100); + } + })(); + async function fastForwardModel(on) { + fastForward = on; + if (!on) await setSpeed(1); + } + async function waitForTurn(previous) { while (!turnId || turnId === previous) await pause(100); return turnId; @@ -321,45 +387,170 @@ async function main() { } const finished = () => count(entries, "Command: Output") > 0; + // --- Setup, before the camera rolls ---------------------------------------- + // The service runs from the start, and the agent gets its guardrail. + let pid = startService(); + await say(`stage.line("$ node dist/app.js", "cmd")`); + await say(`stage.service("running", "RUNNING · PID ${pid}")`); + await pause(1500); + const ingress = (handler, body) => + fetch(`${INGRESS_URL}/Agent/${AGENT_ID}/${handler}`, { + method: "POST", + headers: {"content-type": "application/json"}, + body: JSON.stringify(body), + }); + await ingress("initialize", {name: AGENT_ID}); + await ingress("updateProfile", {guardrails: [GUARDRAIL]}); + await openUi(); + await ui.waitFor(`document.body.innerText.includes("Live")`); + + async function pendingApprovals() { + const response = await fetch(`${INGRESS_URL}/Agent/${AGENT_ID}/approvals`, {method: "POST"}); + return response.json(); + } + // --- Title --------------------------------------------------------------- - await say(`stage.card("Restate reference agent", "A durable agent, for real.", "Tools, programs, steering — and a crash in the middle.")`); + await say(`stage.card("Restate reference agent", "A durable agent, for real.", "Programs, steering, approvals, sub-agents — and a crash.")`); await stage.send("Page.startScreencast", {format: "jpeg", quality: 90}); await pause(3200); await say("stage.hideCard()"); // --- Act 1: an ordinary turn ------------------------------------------------ await say(`stage.caption(1, "Ask it for something", "A real model, real web search, a real sandbox.")`); - await typeCommand("node dist/app.js"); - let pid = startService(); - await say(`stage.service("running", "RUNNING · PID ${pid}")`); await pause(800); - await openUi(); - await pause(700); - await ui.waitFor(`document.body.innerText.includes("Live")`); await send("Ask", ASK); const firstTurn = await waitForTurn(undefined); + await fastForwardModel(true); await waitFor(() => count(entries, "Command: Run", "webSearch") >= 2); await say(`stage.caption(2, "It writes a program to do the work", "Its three web searches run at once. Only the program's result enters the context.")`); - await pause(600); + // Steer once the program's searches are recorded. A steer that lands while + // the program still runs hands the program to the turn's pending work, and + // the model then spends its steps waiting on it: real, but another story. + const searchesDone = () => + count(entries, "Command: Run", "webSearch") >= 3 && + count(entries, "Command: Run") === count(entries, "Notification: Run") + (thinking() ? 1 : 0); + await waitFor(searchesDone); + await fastForwardModel(false); await say(`stage.caption(3, "Steer it while it works", "A new instruction for the running turn. Nothing is cancelled.")`); + signalNames.push("steer"); await send("Steer", STEER); + await fastForwardModel(true); await waitFor(() => count(entries, "Notification: Signal") > 0 || finished()); if (!count(entries, "Notification: Signal")) throw new Error("The steer reached no running turn; record again."); await say(`stage.caption(3, "The steer reaches the model's next step", "Running tools finish; the plan in lisbon.md follows the new instruction.")`); + + // The guardrail holds every file write for a person. Approve each one from + // the Approvals panel until the turn finishes. + let approved = 0; + while (!finished()) { + const [pending] = await pendingApprovals(); + if (!pending) { + await pause(200); + continue; + } + await fastForwardModel(false); + if (approved === 0) { + await say(`stage.caption(4, "A guardrail asks you first", ${js(`"${GUARDRAIL.rule}" The turn waits for you, holding no process.`)})`); + await pause(2200); + } + await showApprovals(); + await pause(approved === 0 ? 2600 : 1400); + // Highlight the button for a moment, so the click shows, then click it. + const approve = `[...document.querySelectorAll("button")].find((b) => b.textContent.trim() === "Approve")`; + await ui.evaluate(`${approve}.style.boxShadow = "0 0 0 4px #3ddc97aa, 0 0 24px #3ddc97"`); + await pause(700); + signalNames.push("approval"); + await ui.evaluate(`${approve}.click()`); + approved++; + await pause(300); + await showConversation(); + while ((await pendingApprovals()).some((approval) => approval.approvalId === pending.approvalId)) await pause(150); + await say(`stage.caption(4, "Approved: the turn goes on", "The file is written and the agent answers.")`); + await fastForwardModel(true); + } + if (approved === 0) throw new Error("The guardrail asked for no approval; record again."); + await fastForwardModel(false); + // Hold on the answer long enough to read it. + await pause(6500); + + // --- The fleet: sub-agents in parallel ------------------------------------------ + await say(`stage.caption(5, "Hand the work to a fleet", "Four sub-agents, one per city, each with its own history and sandbox.")`); + await send("Ask", FLEET); + const fleetTurn = await waitForTurn(firstTurn); + await fastForwardModel(true); + + // The children exist once the parent's createSubAgent calls ran. Each one + // is a whole agent: open its own page, read-only, in a tile of the grid. + let children = []; + while (children.length < 4 && !finished()) { + const response = await fetch(`${INGRESS_URL}/Agent/${AGENT_ID}/children`, {method: "POST"}); + children = await response.json(); + await pause(300); + } + await fastForwardModel(false); + const TILE_WIDTH = 912; + const tileScale = TILE_WIDTH / 730; + const tileVisible = 384 / tileScale; + for (const [index, child] of children.slice(0, 4).entries()) { + const tab = await browser.newTab({url: "about:blank", width: 730, height: 1100, scale: tileScale}); + await tab.goto(`${UI_URL}/?agent=${child.agentId}`); + await tab.waitFor(`Boolean(document.querySelector(".conversation-pane"))`); + await tab.evaluate(`(() => { + const style = document.createElement("style"); + style.textContent = ".conversation-pane { height: ${Math.floor(tileVisible) + 170}px !important; min-height: 0 !important; }"; + document.head.appendChild(style); + })()`); + const top = await tab.evaluate( + `document.querySelector(".conversation-pane").getBoundingClientRect().top + scrollY + 64`, + ); + await say(`stage.fleetTile(${index}, ${js(child.name)}, ${Math.round(top * tileScale)})`); + await mirror(tab, (frame) => `stage.fleetFrame(${index}, ${frame})`); + childTabs.push(tab); + } + await say("stage.showFleet(true)"); + await say(`stage.caption(5, "Four sub-agents work at once", "The parent's turn waits on all four, durably. Each child runs its own turns.")`); + + // Mark each child done when its turn completes. + async function childTurnsDone() { + const keys = children.map((child) => `'${child.agentId}'`).join(", "); + const rows = await query( + `SELECT target_service_key, status FROM sys_invocation WHERE target_service_name = 'AgentSession' ` + + `AND target_handler_name = 'doTurn' AND target_service_key IN (${keys})`, + ); + return children.map((child) => + rows.some((row) => row.target_service_key === child.agentId && row.status === "completed"), + ); + } + await pause(1500); await setSpeed(2); - await waitFor(finished); + let done = []; + while (done.filter(Boolean).length < children.length) { + done = await childTurnsDone(); + for (const [index, isDone] of done.entries()) { + await say(`stage.fleetStatus(${index}, ${isDone})`); + } + await pause(300); + } await setSpeed(1); - await pause(3500); + await pause(2500); + await recrop(); + await say("stage.showFleet(false)"); + await say(`stage.caption(6, "The parent combines their answers", "Four results come back into one turn, which answers.")`); + await fastForwardModel(true); + await waitFor(() => turnId === fleetTurn && finished()); + await fastForwardModel(false); + await pause(6500); // --- Act 2: kill it mid-turn ------------------------------------------------ await say(`stage.card("Now the fun part", "Kill it mid-turn.", "A follow-up that waits ${SLEEP_SECONDS} seconds — and kill -9 in the middle.")`); await pause(2800); await say("stage.hideCard()"); - await say(`stage.caption(4, "Ask for something that takes a while", "A ${SLEEP_SECONDS}-second durable timer, then it reads the file back.")`); + await say(`stage.caption(7, "Ask for something that takes a while", "A ${SLEEP_SECONDS}-second durable timer, then it reads the file back.")`); await send("Ask", FOLLOW_UP); - await waitForTurn(firstTurn); + await waitForTurn(fleetTurn); + await fastForwardModel(true); // Kill only when the timer runs and no other step is open, so every step // before the crash has its result recorded. @@ -369,7 +560,8 @@ async function main() { await pause(2500); await waitFor(quiet); - await say(`stage.caption(5, "Kill the service mid-turn", "kill -9: no shutdown, no warning. The timer is still running.")`); + await fastForwardModel(false); + await say(`stage.caption(8, "Kill the service mid-turn", "kill -9: no shutdown, no warning. The timer is still running.")`); await typeCommand(`kill -9 ${pid}`); replay.crashedAt = entries.at(-1).index; service.kill("SIGKILL"); @@ -377,12 +569,12 @@ async function main() { await say(`stage.service("down", "KILLED")`); await say(`stage.line("[1]+ Killed: 9 node dist/app.js", "bad")`); await pause(2200); - await say(`stage.caption(5, "No process is running this turn", "Restate holds its journal and its timer. Nothing is lost.")`); + await say(`stage.caption(8, "No process is running this turn", "Restate holds its journal and its timer. Nothing is lost.")`); await setSpeed(4); await pause(8000); await setSpeed(1); - await say(`stage.caption(6, "Start it again", "Restate replays the journal. Recorded results are reused, not re-run.")`); + await say(`stage.caption(9, "Start it again", "Restate replays the journal. Recorded results are reused, not re-run.")`); await typeCommand("node dist/app.js"); await say(`stage.service("starting", "STARTING")`); const replaying = new Promise((done) => { @@ -410,16 +602,16 @@ async function main() { await say(`stage.journalFoot(${js(`Replayed ${recorded.length} recorded steps, ${models} of them model calls. None ran again.`)})`); await pause(2500); - await say(`stage.caption(7, "The turn finishes where it stopped", "The timer fires on schedule; the agent reads the file and answers.")`); + await say(`stage.caption(10, "The turn finishes where it stopped", "The timer fires on schedule; the agent reads the file and answers.")`); await setSpeed(4); await waitFor(finished); await setSpeed(1); - await pause(4000); + await pause(6500); // --- The Restate UI ----------------------------------------------------------- await restateUi.goto(`${ADMIN_URL}/ui/invocations/${turnId}`); await pause(2500); - await say(`stage.caption(8, "Inspect it in the Restate UI", "One invocation, every step of the turn, across the crash.")`); + await say(`stage.caption(11, "Inspect it in the Restate UI", "One invocation, every step of the turn, across the crash.")`); await say("stage.showRestate(true)"); await pause(2500); // Scroll to the timer: its bar spans the time the service was down. @@ -429,7 +621,7 @@ async function main() { scrollTo({top, behavior: "smooth"}); })()`); await pause(2000); - await say(`stage.caption(8, "The timer kept running while the service was down", "It fired on schedule, and the turn went on from the next step.")`); + await say(`stage.caption(11, "The timer kept running while the service was down", "It fired on schedule, and the turn went on from the next step.")`); await pause(4000); // --- End card --------------------------------------------------------------- @@ -437,7 +629,7 @@ async function main() { await pause(3500); polling = false; - await Promise.all([poller, follower]); + await Promise.all([poller, follower, speeder]); await stage.send("Page.stopScreencast"); frames.push({file: frames.at(-1).file, time: videoTime(Date.now() / 1000)}); writeFileSync(join(OUT, "frames.json"), JSON.stringify(frames)); diff --git a/README.md b/README.md index 0db1ab5..1fb1262 100644 --- a/README.md +++ b/README.md @@ -4,11 +4,12 @@ A complete agent, built on [Restate](https://restate.dev). Every feature a modern agent needs is here as a small module you can read in one sitting, and Restate keeps each turn running through crashes and days-long waits. -https://github.com/user-attachments/assets/19a8b76d-78cb-4a9f-b97b-d98b754642ca +https://github.com/user-attachments/assets/41ddbac1-09a0-4a70-839a-79b29bd53331 *A real session: the model writes a program whose web searches run at -once, you steer it while it works, and then the service is killed mid-turn -and the turn still finishes.* +once, you steer it and approve its file writes, it hands work to four +sub-agents, and then the service is killed mid-turn and the turn still +finishes.* [Features](#features) · [One turn, start to finish](#one-turn-start-to-finish) · [How a turn works](#how-a-turn-works) · [Quickstart](#quickstart) ·