Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 21 additions & 8 deletions .misc/video/README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,17 @@
# README video: a real session

A real recording of the reference agent, in two acts:
A real recording of the reference agent, in three acts:

1. **An ordinary turn.** A casual request for a weekend plan. The model
writes a program whose three web searches run at once, saves `lisbon.md`
in its sandbox, and is steered while it works.
2. **Kill it mid-turn.** A follow-up waits on a 30-second durable timer. The
writes a program whose three web searches run at once and is steered
while it works. The agent has a guardrail that makes file writes wait for
a person, so saving `lisbon.md` asks for approval in the UI's Approvals
panel, and the script approves it.
2. **A fleet of sub-agents.** The agent hands four cities to four
sub-agents, which research at the same time. Each child is a whole agent,
so the script opens each child's own page, read-only, in a 2×2 grid, then
returns to the parent's combined answer.
3. **Kill it mid-turn.** A follow-up waits on a 30-second durable timer. The
service is killed with `kill -9`, started again, and Restate replays the
turn's journal, so the turn reads the file back and answers.

Expand All @@ -19,12 +25,17 @@ when the UI or the runtime changes.
| `cdp.mjs` | A tiny Chrome DevTools Protocol client (Node 22+, no npm packages) |
| `encode.sh` | Turns the captured frames into `out/agent-demo.mp4` |

Everything on screen comes from the running system except the titles and
captions: the UI is streamed from its own tab, the terminal shows the
Everything on screen comes from the running system except the titles,
captions and the highlight on the Approve button before it is clicked: the UI is streamed from its own tab, the terminal shows the
service's real log lines, and the journal panel is `sys_journal`. The quiet
stretches (model calls after the steer, the service down, the timer
running) play faster, shown by the badge in the header.

The web UI renders 800 CSS pixels wide, its one-column layout, zoomed in so
the text stays legible. To fit the frame, the script crops off the agent
picker, injects one rule that shortens the conversation pane, and slides
the view down to the Approvals panel while a decision is pending.

## Record

You need Docker (or `restate-server`), Google Chrome, ffmpeg, and an
Expand Down Expand Up @@ -60,12 +71,14 @@ ffmpeg -i .misc/video/out/agent-demo.mp4 -vf fps=1/3 /tmp/frames/%02d.png

Check that each ask went through at once (the prompt must not sit in the
composer), that the steer landed while the turn ran (the script stops if it
did not), that the kill happened while only the timer was open, that the
did not), that each approval showed its card before the click, that all four
sub-agent tiles filled and finished, that the
kill happened while only the timer was open, that the
journal marks only finished steps as replayed, and that the Restate UI scene
shows the 30-second `sleep` row. The model's wording differs between takes,
so read its answers too.

A take runs about 75 seconds. If it runs much longer, look for leftover
A take runs about two minutes. If it runs much longer, look for leftover
headless Chrome processes (`pgrep -f readme-video-chrome`): they slow the UI
down enough to delay the ask.

Expand Down
73 changes: 58 additions & 15 deletions .misc/video/director.html
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@

/* Header */
header {
position: absolute; left: 40px; right: 40px; top: 26px; height: 56px;
position: absolute; left: 40px; right: 40px; top: 18px; height: 52px;
display: flex; align-items: center; gap: 18px;
}
.logo {
Expand Down Expand Up @@ -74,15 +74,15 @@
.dots i { width: 12px; height: 12px; border-radius: 50%; background: #2c2c3c; display: block; }
.panel-head .right { margin-left: auto; }

#ui { left: 40px; top: 104px; width: 1100px; height: 812px; }
#ui .view { width: 1100px; height: 768px; overflow: hidden; }
#ui img { display: block; width: 1100px; visibility: hidden; }
#ui { left: 40px; top: 84px; width: 1240px; height: 872px; }
#ui .view { width: 1240px; height: 828px; overflow: hidden; }
#ui img { display: block; width: 1240px; visibility: hidden; transition: transform 0.7s cubic-bezier(.4,0,.2,1); }

#term { left: 1164px; top: 104px; width: 716px; height: 380px; transition: box-shadow 0.3s, border-color 0.3s; }
#term { left: 1304px; top: 84px; width: 576px; height: 400px; transition: box-shadow 0.3s, border-color 0.3s; }
#term.dead { border-color: #ff4d5e88; box-shadow: 0 0 0 1px #ff4d5e55, 0 24px 60px #ff4d5e22; }
#term-body {
padding: 16px 20px; font: 15px/1.55 var(--mono); color: #cfcfe0;
height: 336px; overflow: hidden; display: flex; flex-direction: column; justify-content: flex-end;
padding: 14px 18px; font: 14px/1.55 var(--mono); color: #cfcfe0;
height: 356px; overflow: hidden; display: flex; flex-direction: column; justify-content: flex-end;
}
.ln { white-space: pre; overflow: hidden; text-overflow: ellipsis; }
.ln.cmd { color: var(--text); }
Expand All @@ -94,6 +94,7 @@
@keyframes blink { 50% { opacity: 0; } }

.pill {
white-space: nowrap;
font: 700 13px var(--mono); padding: 5px 12px; border-radius: 999px;
display: inline-flex; align-items: center; gap: 8px; letter-spacing: 0.04em;
}
Expand All @@ -104,10 +105,10 @@
.pill.starting { color: var(--amber); background: #ffb5471a; }
@keyframes pulse { 50% { opacity: 0.2; } }

#journal { left: 1164px; top: 508px; width: 716px; height: 408px; }
#journal { left: 1304px; top: 508px; width: 576px; height: 448px; }
#journal-body { padding: 10px 14px; }
.jrow {
display: grid; grid-template-columns: 34px 1fr 150px; align-items: center;
display: grid; grid-template-columns: 40px 1fr 130px; align-items: center;
height: 34px; padding: 0 10px; border-radius: 9px; font: 15px var(--mono);
animation: pop 0.35s cubic-bezier(.2,1.4,.4,1);
transition: background 0.25s;
Expand All @@ -131,7 +132,7 @@

/* Caption */
#caption {
position: absolute; left: 40px; right: 40px; bottom: 30px; height: 112px;
position: absolute; left: 40px; right: 40px; bottom: 12px; height: 100px;
display: flex; align-items: center; gap: 22px; padding: 0 28px;
}
#caption .n {
Expand All @@ -145,9 +146,23 @@
@keyframes swap { from { opacity: 0; transform: translateY(10px); } }

/* Full-width Restate UI */
#restate { left: 40px; top: 104px; width: 1840px; height: 812px; opacity: 0; transition: opacity 0.6s; }
#restate { left: 40px; top: 84px; width: 1840px; height: 872px; opacity: 0; transition: opacity 0.6s; }
#restate.on { opacity: 1; }
#restate img { display: block; width: 1840px; height: 768px; }
#restate img { display: block; width: 1840px; height: 828px; }

/* The sub-agent fleet: one live page per child */
#fleet {
position: absolute; left: 40px; top: 84px; width: 1840px; height: 872px;
display: grid; grid-template-columns: 1fr 1fr; grid-template-rows: 1fr 1fr; gap: 16px;
opacity: 0; transition: opacity 0.6s; pointer-events: none;
}
#fleet.on { opacity: 1; }
#fleet .panel { position: relative; }
#fleet .panel-head { color: var(--text); }
#fleet .panel-head .who { font: 700 16px var(--sans); }
#fleet .view { width: 912px; height: 384px; overflow: hidden; }
#fleet img { display: block; width: 912px; }
#fleet .tile.done { border-color: #3ddc9766; }

/* Title cards */
#card {
Expand Down Expand Up @@ -190,7 +205,7 @@ <h1>Restate reference agent</h1>
</section>

<section class="panel" id="term">
<div class="panel-head"><span class="dots"><i></i><i></i><i></i></span>agent service · packages/libs/core<span class="right"><span class="pill running" id="svc">RUNNING</span></span></div>
<div class="panel-head"><span class="dots"><i></i><i></i><i></i></span>agent service<span class="right"><span class="pill running" id="svc">RUNNING</span></span></div>
<div id="term-body"></div>
</section>

Expand All @@ -205,6 +220,13 @@ <h1>Restate reference agent</h1>
<img id="restate-img" alt="">
</section>

<section id="fleet">
<div class="panel tile"><div class="panel-head"><span class="dots"><i></i><i></i><i></i></span><span class="who"></span><span class="right"><span class="pill starting">WORKING</span></span></div><div class="view"><img alt=""></div></div>
<div class="panel tile"><div class="panel-head"><span class="dots"><i></i><i></i><i></i></span><span class="who"></span><span class="right"><span class="pill starting">WORKING</span></span></div><div class="view"><img alt=""></div></div>
<div class="panel tile"><div class="panel-head"><span class="dots"><i></i><i></i><i></i></span><span class="who"></span><span class="right"><span class="pill starting">WORKING</span></span></div><div class="view"><img alt=""></div></div>
<div class="panel tile"><div class="panel-head"><span class="dots"><i></i><i></i><i></i></span><span class="who"></span><span class="right"><span class="pill starting">WORKING</span></span></div><div class="view"><img alt=""></div></div>
</section>

<div id="caption"><div class="n">1</div><div><div class="t"></div><div class="d"></div></div></div>

<div id="card"><div><div class="k" id="card-k"></div><h2 id="card-t"></h2><p id="card-p"></p></div></div>
Expand Down Expand Up @@ -237,12 +259,33 @@ <h1>Restate reference agent</h1>
$("speed").classList.toggle("on", factor > 1);
},

uiCrop(px) {
$("ui-img").style.marginTop = `-${px}px`;
// Shows the web UI from this many pixels below its top; a change slides.
uiView(px) {
$("ui-img").style.transform = `translateY(-${px}px)`;
$("ui-img").style.visibility = "visible";
},
uiFrame(data) { $("ui-img").src = `data:image/jpeg;base64,${data}`; },
restateFrame(data) { $("restate-img").src = `data:image/jpeg;base64,${data}`; },
// The fleet grid: a tile per sub-agent, showing its page from `top` down.
fleetTile(index, name, top) {
const tile = document.querySelectorAll("#fleet .tile")[index];
tile.querySelector(".who").textContent = `sub-agent · ${name}`;
tile.querySelector("img").style.marginTop = `-${top}px`;
},
fleetFrame(index, data) {
document.querySelectorAll("#fleet img")[index].src = `data:image/jpeg;base64,${data}`;
},
fleetStatus(index, done) {
const tile = document.querySelectorAll("#fleet .tile")[index];
const pill = tile.querySelector(".pill");
pill.className = `pill ${done ? "running" : "starting"}`;
pill.textContent = done ? "✓ DONE" : "WORKING";
tile.classList.toggle("done", done);
},
showFleet(on) {
$("fleet").classList.toggle("on", on);
for (const id of ["ui", "term", "journal"]) $(id).style.opacity = on ? 0 : 1;
},
showRestate(on) {
$("restate").classList.toggle("on", on);
for (const id of ["ui", "term", "journal"]) $(id).style.opacity = on ? 0 : 1;
Expand Down
11 changes: 9 additions & 2 deletions .misc/video/encode.sh
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,15 @@ node -e '
require("fs").writeFileSync(process.argv[1] + "/frames.ffconcat", lines.join("\n") + "\n");
' "$out"

# The frames are full-range JPEGs; convert to limited-range yuv420p at level
# 4.1, which every player handles (QuickTime shows a still for yuvj420p).
# Encode to a temporary file and rename it, so a player that has the old
# video open never reads a half-written one.
ffmpeg -y -loglevel error -f concat -safe 0 -i "$out/frames.ffconcat" \
-vf "fps=30,format=yuv420p" -c:v libx264 -preset slow -crf 20 \
-movflags +faststart "$out/agent-demo.mp4"
-vf "fps=30,scale=out_range=tv:out_color_matrix=bt709,format=yuv420p" \
-c:v libx264 -preset slow -crf 20 -profile:v high -level:v 4.1 \
-color_range tv -colorspace bt709 -color_primaries bt709 -color_trc bt709 \
-movflags +faststart "$out/agent-demo.tmp.mp4"
mv "$out/agent-demo.tmp.mp4" "$out/agent-demo.mp4"

ls -lh "$out/agent-demo.mp4"
Loading
Loading