-
-Parallel tool calls
Every call in a model response runs at once; a failure goes back to the model. |
-Background operations
Timers and approvals run on while the model works. It can wait or cancel. |
-
-
-Steering and interrupts
Redirect or stop a turn while it runs, without losing finished work. |
-Sub-agents
Delegate tasks to agents with their own history, sandbox and memory. |
-
-
-Guardrails
Plain-language rules. A policy model allows, blocks or asks a person. |
-Human approval
A turn can wait days for a decision, holding no process. |
-
-
-Programmatic tool calls
The model writes a small program; only its result enters the context. |
-Compaction
Long conversations and long turns are summarized, recent steps kept verbatim. |
-
-
-Schedules
Messages to the agent later, once or on a recurrence. No cron. |
-Memory
A searchable index, so context does not grow with the memories. |
-
-
-Tool search
MCP and discovered tools load on demand, keeping large catalogs out of context. |
-Sandboxes
A local directory or Modal sandbox for files and commands. |
-
-
-Extensible tools
One module per tool, plus MCP servers and Restate handlers. |
-Output recovery
A truncated response gets one retry with a bigger output budget. |
-
-
-
-## One turn, start to finish
-
-Here is one turn, from the first message to the answer, and what Restate
-does for it along the way.
-
-### 1. It calls tools in parallel
-
-The model asks for three tools, the guardrails check them once, and they
-run together. Each result is recorded as it lands, and a call that fails
-goes back to the model as an error.
-
-
-
-### 2. You steer it while it works
-
-A new message does not have to wait for the turn to end. A steer reaches the
-next model step without cancelling tools in flight, and an interrupt stops
-the turn with a summary of what it did.
-
-
-
-### 3. It waits a day for your approval
-
-When a guardrail needs a person, the turn suspends: no process, only stored
-state. It can wait a day, through new versions of the service, and resumes
-where it stopped.
-
-
-
-### 4. It survives a crash
-
-Every model call and tool result is in the turn's journal. If the process
-dies, Restate replays the journal: nothing is asked or run twice, and the
-turn finishes.
-
-
-
-## Why Restate
-
-Every feature above gets the same guarantees, because they come from Restate
-rather than from each feature:
-
-- **A crash resumes the turn.** Restate replays the turn's journal. Recorded
- model responses and tool results are reused, so the model is never asked
- to repeat a decision and completed tool calls are not re-run.
-- **New versions don't break running turns.** Restate keeps each turn on
- the service version it started on; new turns use the new version.
-- **Waiting is free.** A turn can wait hours or days for an approval, a timer
- or a sub-agent. While it waits it holds no process, only stored state.
-- **Always responsive.** `ask`, `steer` and `interrupt` return right away,
- even while a turn runs for minutes.
-- **Nothing else to operate.** State, messages, timers and notifications all
- live in Restate: no database, queue or scheduler of the agent's own. Each
- agent's state has a single owner, so there are no locks. The service is
- stateless: it scales out, or builds into a single bundle for serverless.
-
-**What replay does not do:** make external side effects exactly-once. A crash
-between an MCP call completing and its result being recorded can repeat that
-call.
-
-## How a turn works
-
-Each agent is two Restate Virtual Objects with the same key:
-
-- **`Agent`**, the controller. It decides what happens to each incoming
- message and owns the profile: instructions, guardrails, memories, tool
- grants, approvals, schedules and child agents. Its handlers are short, so
- it always answers.
-- **`AgentSession`**, the turn. One `doTurn` invocation is one turn. It owns
- the append-only conversation log and runs the model/tool loop.
-
-```mermaid
-sequenceDiagram
- autonumber
- participant C as Client
- participant A as Agent (controller)
- participant S as AgentSession (turn)
- participant M as Model, guardrails, tools
-
- C->>A: ask("Weather in Berlin?")
- A-)S: doTurn(profile snapshot), one way
- A-->>C: started, turnId
- S->>S: append to log, build context
-
- S->>M: model step
- M-->>S: proposal: call 3 tools
- S->>M: guardrail check
- par tool calls run concurrently
- S->>M: getWeather
- and
- S->>M: webSearch
- and
- S->>M: runCommand
- end
-
- C->>A: steer("Use Fahrenheit")
- A-)S: steering signal, addressed to turnId
- Note over S: A crash anywhere here is harmless. Restate replays the journal and reuses every recorded result.
-
- S->>M: model step, with tool results and steering
- M-->>S: final answer
- S-)A: onTurnEnd(outcome)
- Note over A: Retire the turn. Start the next one if messages were queued.
-
- C->>A: watch(afterRevision), long poll
- A-->>C: history changed
- C->>S: history(fromSequence)
- S-->>C: new entries
-```
+- **[`Agent`](packages/libs/core/src/agent/service.ts)** handles incoming messages
+ and tracks the active turn, queued input, profile, memories, approvals, and schedules.
+- **[`AgentSession`](packages/libs/core/src/session/service.ts)** stores the
+ conversation and runs the model/tool loop. Each `doTurn` invocation handles
+ one turn, which may incorporate several queued messages or steering updates.
+
+
+
+This architecture makes the following advanced features possible:
+
+### Steer and interrupt ongoing agent executions
+
+When an `Agent` starts a turn, it gets the `turnId` back, with which it can:
+- **Steer the turn**: adds an instruction to the context for the next LLM call.
+ Tools already running can finish.
+- **Interrupt the turn**: stops unfinished work, including subagent tasks, and asks
+ the model to summarize what it completed.
+
+Restate provides durable signal delivery. The reference implements the routing,
+turn state machine, and cleanup rules on top of it.
+
+
+
+See [steering](docs/turn-runtime.md#steering) and
+[interruption](docs/turn-runtime.md#interruption-and-stopping).
+
+### Fine-grained recovery within the agent loop
+
+Each step in an agent loop is recorded in Restate's journal: LLM calls, guardrail
+checks, tool calls, state updates, approvals,...
+After a failure or a long wait, the agent process can recover to the exact step
+where it left off, by replaying the journal.
+
+Restate only adds a few milliseconds of overhead to persist a journal entry,
+making fine-grained recovery feasible.
+
+
+
+### Resilient parallel work within an execution
+
+Tools run concurrently in the same process and share resources such as sandbox
+connections. Each tool's durable steps are recorded independently, so recovery
+can reuse completed work across the batch. The runtime handles deterministic
+replay during recovery.
+
+
+
+See [parallel tools](docs/turn-runtime.md#agent-loop-iterations)
+and [background operations](docs/tools.md#pending-tools).
+
+### Scaling thousands of stateful agents
+
+Agent IDs run in parallel across service instances, while each session runs one
+turn at a time. Restate coordinates state access, routes calls, and recovers
+work after process failures, without the need for locks or coordination.
+
+See [state ownership](docs/architecture.md#state-ownership).
+
+### Consistent per-session memory and background compaction
+
+This architecture uses Restate's embedded KV store for both conversation state
+and context. Restate gives each agent session its own isolated store, and
+ensures that only a single process can write to it at a time.
+
+This architecture also implements:
+- **Selective loading:** history is stored in chunks, and memories are retrieved
+ on demand. Clients read only the history they need to update the UI,
+ and the model sees only relevant memories.
+- **Compaction background jobs:** Older messages are compacted in the background,
+ to avoid exceeding the LLM's context limit. The model sees the latest messages
+ verbatim, incl. the last summary. The full chat transcript is retained so the
+ UI client can retrieve it, when needed.
+
+See [history and context](docs/architecture.md#control-and-history).
+
+### Waiting durably for approval
+
+When a guardrail needs a person, the turn suspends: no process, only stored state. It can wait weeks, through new versions of the service, and resumes where it stopped.
+
+
+
+See [approvals](docs/protocol.md#context-and-approvals).
+
+### Subscribing to session updates and reconnecting later
-The controller keeps track of the currently executing turn. `steer`,
-`interrupt` and approval decisions reach the turn as durable signals
-addressed to its invocation ID, the `turnId`, so they never land in the
-wrong turn.
+The reference UI uses a [typed client](packages/libs/client/src/index.ts) to read session data and follow
+changes to history, approvals, configuration, and schedules. The UI
+uses HTTP long-polling and revision tags to fetch only what changed, while
+transcript sequence numbers let it catch up after disconnects.
-`doTurn` runs the model calls and tools itself, as in-process function calls.
-Each result is appended to the turn's journal over one open, low-latency
-stream to Restate. That append is the only persistence a step needs.
+See [session updates](docs/protocol.md#history-and-notifications).
## Further reading
diff --git a/docs/README.md b/docs/README.md
index 1346e8e..3a41f57 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -6,7 +6,9 @@ request path begins with an agent ID, without an account or login prerequisite.
## Reading path
1. [Run a conversation](../README.md).
-2. [Architecture](architecture.md): controller, turn, state owners, notifications.
+2. [Architecture](architecture.md): state owners and an
+ [end-to-end communication diagram](architecture.md#one-request) covering
+ the controller, turn, tools, steering and client updates.
3. [Protocol](protocol.md): ask, steer, interrupt, history, approvals and clients.
4. [Turn runtime](turn-runtime.md): steps, policy, pending work and recovery.
5. [Tools](tools.md): built-ins, PTC, dynamic Restate handlers and MCP.
diff --git a/docs/architecture.md b/docs/architecture.md
index c988c38..376608e 100644
--- a/docs/architecture.md
+++ b/docs/architecture.md
@@ -34,20 +34,82 @@ its own session waits for that child's outcome without holding the parent lock.
```mermaid
sequenceDiagram
+ autonumber
participant C as Client
- participant A as Agent
- participant S as AgentSession
- participant M as Model and tools
- C->>A: ask(message)
- A->>S: send doTurn(snapshot)
- A-->>C: started + turnId
- S->>M: model / policy / tool steps
- C->>A: steer or interrupt
- A-->>S: durable control signal
+ participant A as Agent (controller)
+ participant S as AgentSession (turn)
+ participant M as Model and guardrails
+ participant T as Tools
+
+ C->>A: ask("Compare weather in Berlin, Paris and Rome")
+ A-)S: doTurn(profile snapshot and input), one way
+ A-->>C: started, turnId
+ S->>S: append input to history, build context
+
+ S->>M: model call
+ M-->>S: proposal: three tool calls
+ opt guardrails configured
+ S->>M: evaluate proposed tool batch
+ M-->>S: allow
+ end
+ Note over S,T: Allowed foreground calls run concurrently inside the turn
+ par Berlin
+ S->>T: getWeather(Berlin)
+ T-->>S: result
+ and Paris
+ S->>T: getWeather(Paris)
+ T-->>S: result
+ and Rome
+ S->>T: getWeather(Rome)
+ T-->>S: result
+ and new input while tools run
+ C->>A: steer("Use Fahrenheit")
+ A-)S: durable steering signal addressed to turnId
+ A-->>C: accepted
+ end
+ Note over S,T: Restate journals durable operations; recovery reuses recorded results
+
+ S->>S: retain tool results, consume steering
+ S->>M: next model call with results and steering
+ M-->>S: proposed final answer
+ opt guardrails configured
+ S->>M: evaluate proposed answer
+ M-->>S: allow
+ end
S->>A: onTurnEnd(outcome)
+ Note over A: Reconcile late input, retire matching turn, dispatch queued work
A-->>S: reconciled outcome
+ S->>S: append final outcome to history
+ S-)A: publish(history)
+
+ C->>A: watch(afterRevision), long poll
+ A-->>C: changed topic revisions
+ C->>S: history(fromSequence)
+ S-->>C: new entries and next sequence
```
+The diagram shows a successful tool batch and final answer. Model/tool steps
+can repeat, and a guardrail can instead block a proposal or wait for approval.
+`getWeather` is a synthetic demo tool. The client can watch throughout the
+turn; history appends publish changes as work progresses, not just at the end.
+
+These arrows describe logical calls through Restate. `doTurn` is dispatched
+one way so the controller can return immediately. Steering, interruption and
+approval decisions use durable signals addressed to that invocation's
+`turnId`. Normal completion uses a request/response call to `onTurnEnd` so the
+session can record the controller's reconciled outcome. A queued successor
+cannot execute until the current exclusive `doTurn` finishes.
+
+The model and tool lanes represent work owned by the turn, not additional
+Virtual Objects. Built-in tools run as in-process functions; model requests,
+MCP calls and sandbox operations use journaled effects. Discovered Restate
+tools use durable RPCs. Recovery reuses recorded results, but an external
+effect that completed before its result was recorded may run again; remote
+side effects still need provider-level idempotency.
+
+For exact message shapes, see the [protocol](protocol.md). For the loop's
+approval, pending-work and cancellation paths, see the [turn runtime](turn-runtime.md).
+
## Control and history
diff --git a/docs/images/agent-architecture.png b/docs/images/agent-architecture.png
new file mode 100644
index 0000000..91fb9db
Binary files /dev/null and b/docs/images/agent-architecture.png differ
diff --git a/docs/images/agent-architecture.svg b/docs/images/agent-architecture.svg
new file mode 100644
index 0000000..15c2979
--- /dev/null
+++ b/docs/images/agent-architecture.svg
@@ -0,0 +1,130 @@
+