Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased] - 2026-09-28

### Added
- Fork and replay (LOU-D44): `AgentExecutor.fork({ sessionId, fromStep, newSessionId?, checkpointStore, patch? })` takes the newest entry of `fromStep` in the session's checkpoint history, applies `patch` (`messages(messages)`, `businessState`, `toolResult: { toolCallId, result }` replaced in place so the transcript stays provider-valid, `appendInput` queued as a user message) and saves it as the `'in-progress'` checkpoint of `newSessionId` (default `<sessionId>.fork-<n>`), leaving the original session untouched; it returns `{ sessionId, step, checkpoint }`, and `execute({ sessionId, checkpointStore, input: [] })` continues the fork. A missing step throws the new `LOUSHY_CHECKPOINT_NOT_FOUND` code. `createAgent()` agents gain `agent.fork(sessionId, { fromStep, newSessionId?, patch? })` over `store.checkpoints` (resume with `agent.resume()`). `compareTrajectories(a, b)` compares two checkpoints or transcripts step by step: each run's steps (text, tool calls and results), the first diverging step and the `loushy eval --drift` diff. New types `ForkOptions`, `ForkPatch`, `ForkResult`, `TrajectoryComparison`, `TrajectoryStep` and `DriftEntry`. See docs/durable-execution.md#fork-and-replay.
- Multimodal message parts (LOU-V11): `Message.content` is now `string | ContentPart[]`, where a `ContentPart` is `{ type: 'text', text }`, `{ type: 'image', image, mimeType? }` (an `http(s)` URL, a `data:` URL or a `Uint8Array`) or `{ type: 'file', data, mimeType, filename? }` (new types `ContentPart`, `TextContentPart`, `ImageContentPart`, `FileContentPart`). The built-in providers map the parts of `user` messages to the `ai` v4 user-content parts: images go to the model with all four (the `ai` SDK downloads an image URL first for Anthropic and Ollama); file parts become a `[file <name> (<mimeType>) not sent]` text note with a one-time `console.warn`, since the pinned peers (`@ai-sdk/*` 0.0.x, `ollama-ai-provider`) cannot send files (a subclass sets `acceptsFileParts = true` to send them). `system`, `assistant` and `tool` messages are sent as their text. The new `textOf(message)` returns a message's text (the string, or its text parts joined), and is what `estimateTokens()` (each image or file part counts 1,000 tokens), compaction, `recordReplay()` fingerprints (cassette messages gain an optional `attachments` list: each image or file as its URL or a digest of its bytes), `mockModel()`, `MockLLMProvider`, OpenTelemetry content attributes and the React reducer (`ui.send` takes parts too) now read. `FileSessionStore` saves `Uint8Array`s as `{ "$bytes": "<base64>" }` and loads them back. Pass multimodal messages as `AgentExecutor.execute({ input: Message[] })`; `agent.send()` / `session.send()` taking parts is LOU-V12. See docs/providers.md#multimodal-input.
- Declarative permission policies (LOU-X2): `permissions?: PermissionRule[]` on `createAgent()`, `ExecuteOptions` and `resumeAfterApproval()` options. A rule is `{ tool: string | string[] | RegExp | '*', when?: (args, { toolName, toolCallId, sessionId }) => boolean | Promise<boolean>, action: 'allow' | 'deny' | 'ask', reason? }`; rules are checked in order for each tool call (after argument validation and `preToolCall` hooks, before `needsApproval`) and the first match decides: `allow` runs the call without approval even when `needsApproval` would ask, `deny` gives the model a tool error with the new `kind: 'denied'` (`ToolDeniedError`, plus `reason`) and streams `tool.error`, `ask` pauses for approval (or asks `approve`). No match keeps today's behaviour. New helpers `allow(tools)`, `deny(tools, reason?)`, `ask(tools)` and types `PermissionRule`, `PermissionAction`, `PermissionToolMatcher`, `PermissionContext`, `PermissionDecision`, `PermissionDecisionEntry`, `PermissionOptions`. Audit log: `onPermissionDecision(entry)` gets `{ toolName, toolCallId, decision: 'allow' | 'deny' | 'ask' | 'default', rule?: { index, reason? }, args?, at }` per call (`args` left out under `redactContent`), and streams get the same entry as the new typed `permission.decision` event (`PermissionDecisionEvent`; additive, the event schema version stays 1). Neither is produced unless `permissions` or `onPermissionDecision` is set. Sub-agents inherit the lead's rules (checked before their own) and its `onPermissionDecision`. See docs/approvals.md#permission-policies.
- Compaction events and the `compaction` option (LOU-W3.2): `agent.stream()`, `session.stream()` and `AgentExecutor.stream()` emit two new typed events whenever the compaction hook compacts a request: `compaction.start` (`strategy`, `tokensBefore`, `contextWindow`, `thresholdTokens`) before the work and `compaction.done` (`strategy`, `tokensBefore`, `tokensAfter`, `prunedToolCallIds`, `summary?`, `error?: { message }`) after it, inside the step and before the model call; `error` is set when the strategy failed and the hook fell back. Additive: the event schema version stays 1. `GenerateHookContext` gains the optional `emit(event)` (set in streamed runs only) that the hook uses. `createAgent({ compaction })` takes `true` (the prune hook with its defaults) or `{ strategy?, thresholdPercent?, contextWindow?, protectedTokens?, summarizer? }`, where `summarizer` (a `'provider/model'` string or an `LLMProvider`) selects `twoPhaseStrategy()` with that model; `summarizer` together with `strategy` is a `ConfigurationError`. `createAgent({ hooks })` (new) takes any `AgentHook[]`, run before the compaction hook. See docs/compaction.md.
Expand Down
69 changes: 69 additions & 0 deletions docs/durable-execution.md
Original file line number Diff line number Diff line change
Expand Up @@ -226,6 +226,75 @@ await checkpoints.delete('chat-42', { keepHistory: true });
an existing database file is opened; `prune()` also removes history entries
older than its cutoff.

## Fork and replay

`AgentExecutor.fork()` starts a new session from a step of another session's
history: "what if the tool had returned something else at step 1?" It takes the
newest history entry of `fromStep`, applies `patch`, saves the result as the
`'in-progress'` checkpoint of the new session and returns
`{ sessionId, step, checkpoint }`. Resume the fork like any unfinished run:

```ts
import { AgentExecutor, compareTrajectories, memoryStore } from '@loushy/build-ai-agent';

const { checkpoints } = memoryStore();
await AgentExecutor.execute({ agent, provider, toolRegistry, input: 'Plan my trip', sessionId: 'trip', checkpointStore: checkpoints });

const fork = await AgentExecutor.fork({
sessionId: 'trip',
fromStep: 1,
checkpointStore: checkpoints,
patch: { toolResult: { toolCallId: 'call_weather', result: { forecast: 'rain' } } },
});
// fork.sessionId is 'trip.fork-1'; the 'trip' checkpoint and history are unchanged.
await AgentExecutor.execute({ agent, provider, toolRegistry, input: [], sessionId: fork.sessionId, checkpointStore: checkpoints });

const original = await checkpoints.load('trip');
const replayed = await checkpoints.load(fork.sessionId);
if (original && replayed) {
const { divergedAt, a, b, drift } = compareTrajectories(original, replayed);
console.log(divergedAt, a.length, b.length, drift); // 1, then each run's steps and the tool-call diff
}
```

With `createAgent({ store })`, `agent.fork(sessionId, { fromStep, patch? })`
does the same over `store.checkpoints`, and `agent.resume(fork.sessionId)`
continues the fork:

```ts
import { createAgent, memoryStore } from '@loushy/build-ai-agent';

const assistant = createAgent({ provider, store: memoryStore() });
await assistant.send('Plan my trip', { sessionId: 'trip' });
const fork = await assistant.fork('trip', { fromStep: 1, patch: { appendInput: 'It will rain, plan for that.' } });
const result = await assistant.resume(fork.sessionId);
```

- `patch` is applied in this order: `messages(messages)` rewrites the
transcript; `businessState` replaces it; `toolResult: { toolCallId, result }`
replaces that call's result in place (same position, call id and tool name,
so the transcript stays valid for every provider), or records it when the
call has no result yet; `appendInput` queues a user message, which is sent
after any tool calls still pending.
- The fork keeps the source checkpoint's step count and usage, so the rest of
the `maxSteps` budget applies to it. Recorded tool results are not re-run;
tool calls without a result run first, as on any resume (so a fork of an
`'awaiting-approval'` entry goes through `needsApproval` again; the old
`approvalId` is not carried over).
- `newSessionId` names the fork; the default is `<sessionId>.fork-<n>`, the
first `n` with no checkpoint. A `newSessionId` that already has a checkpoint
is refused.
- A step the history does not have (the session is unknown, the step never
ran, or it was dropped past `historyLimit`) throws an `SDKError` with code
[`LOUSHY_CHECKPOINT_NOT_FOUND`](./errors.md#loushy_checkpoint_not_found),
whose message lists the steps that are kept. A store without `history()`
cannot fork (`ConfigurationError`).
- `compareTrajectories(a, b)` takes two checkpoints or two transcripts and
returns each run's steps (one per assistant turn: its text and tool calls
with their results), `divergedAt` (the first step that differs, tool call
ids aside) and `drift`, the tool-order, argument, step-count and
finish-reason differences in the same shape as `loushy eval --drift`.

## Compatibility with stored data

- Checkpoints written before `status` existed are treated as
Expand Down
12 changes: 12 additions & 0 deletions docs/errors.md
Original file line number Diff line number Diff line change
Expand Up @@ -290,6 +290,18 @@ streaming runner.

**Example:** `new AgentSession(run).stream('hi')`.

### LOUSHY_CHECKPOINT_NOT_FOUND

**Means:** `AgentExecutor.fork()` or `agent.fork()` was asked for a step the
session's checkpoint history does not have: the session is unknown, the step
was never reached, or its entries were dropped past the store's `historyLimit`.
The message lists the steps that are kept.

**Fix:** fork at one of the listed steps, or raise `historyLimit` on the store.
See [Durable execution](./durable-execution.md#fork-and-replay).

**Example:** `agent.fork('job-1', { fromStep: 9 })` after a 3-step run.

### LOUSHY_RUN_ALREADY_ITERATED

**Means:** an `AgentRun` from `session.stream()` was iterated a second time.
Expand Down
81 changes: 81 additions & 0 deletions llms-full.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3180,6 +3180,75 @@ await checkpoints.delete('chat-42', { keepHistory: true });
an existing database file is opened; `prune()` also removes history entries
older than its cutoff.

## Fork and replay

`AgentExecutor.fork()` starts a new session from a step of another session's
history: "what if the tool had returned something else at step 1?" It takes the
newest history entry of `fromStep`, applies `patch`, saves the result as the
`'in-progress'` checkpoint of the new session and returns
`{ sessionId, step, checkpoint }`. Resume the fork like any unfinished run:

```ts
import { AgentExecutor, compareTrajectories, memoryStore } from '@loushy/build-ai-agent';

const { checkpoints } = memoryStore();
await AgentExecutor.execute({ agent, provider, toolRegistry, input: 'Plan my trip', sessionId: 'trip', checkpointStore: checkpoints });

const fork = await AgentExecutor.fork({
sessionId: 'trip',
fromStep: 1,
checkpointStore: checkpoints,
patch: { toolResult: { toolCallId: 'call_weather', result: { forecast: 'rain' } } },
});
// fork.sessionId is 'trip.fork-1'; the 'trip' checkpoint and history are unchanged.
await AgentExecutor.execute({ agent, provider, toolRegistry, input: [], sessionId: fork.sessionId, checkpointStore: checkpoints });

const original = await checkpoints.load('trip');
const replayed = await checkpoints.load(fork.sessionId);
if (original && replayed) {
const { divergedAt, a, b, drift } = compareTrajectories(original, replayed);
console.log(divergedAt, a.length, b.length, drift); // 1, then each run's steps and the tool-call diff
}
```

With `createAgent({ store })`, `agent.fork(sessionId, { fromStep, patch? })`
does the same over `store.checkpoints`, and `agent.resume(fork.sessionId)`
continues the fork:

```ts
import { createAgent, memoryStore } from '@loushy/build-ai-agent';

const assistant = createAgent({ provider, store: memoryStore() });
await assistant.send('Plan my trip', { sessionId: 'trip' });
const fork = await assistant.fork('trip', { fromStep: 1, patch: { appendInput: 'It will rain, plan for that.' } });
const result = await assistant.resume(fork.sessionId);
```

- `patch` is applied in this order: `messages(messages)` rewrites the
transcript; `businessState` replaces it; `toolResult: { toolCallId, result }`
replaces that call's result in place (same position, call id and tool name,
so the transcript stays valid for every provider), or records it when the
call has no result yet; `appendInput` queues a user message, which is sent
after any tool calls still pending.
- The fork keeps the source checkpoint's step count and usage, so the rest of
the `maxSteps` budget applies to it. Recorded tool results are not re-run;
tool calls without a result run first, as on any resume (so a fork of an
`'awaiting-approval'` entry goes through `needsApproval` again; the old
`approvalId` is not carried over).
- `newSessionId` names the fork; the default is `<sessionId>.fork-<n>`, the
first `n` with no checkpoint. A `newSessionId` that already has a checkpoint
is refused.
- A step the history does not have (the session is unknown, the step never
ran, or it was dropped past `historyLimit`) throws an `SDKError` with code
[`LOUSHY_CHECKPOINT_NOT_FOUND`](https://github.com/LinuxDevil/agent-sdk/blob/main/docs/errors.md#loushy_checkpoint_not_found),
whose message lists the steps that are kept. A store without `history()`
cannot fork (`ConfigurationError`).
- `compareTrajectories(a, b)` takes two checkpoints or two transcripts and
returns each run's steps (one per assistant turn: its text and tool calls
with their results), `divergedAt` (the first step that differs, tool call
ids aside) and `drift`, the tool-order, argument, step-count and
finish-reason differences in the same shape as `loushy eval --drift`.

## Compatibility with stored data

- Checkpoints written before `status` existed are treated as
Expand Down Expand Up @@ -3496,6 +3565,18 @@ streaming runner.

**Example:** `new AgentSession(run).stream('hi')`.

### LOUSHY_CHECKPOINT_NOT_FOUND

**Means:** `AgentExecutor.fork()` or `agent.fork()` was asked for a step the
session's checkpoint history does not have: the session is unknown, the step
was never reached, or its entries were dropped past the store's `historyLimit`.
The message lists the steps that are kept.

**Fix:** fork at one of the listed steps, or raise `historyLimit` on the store.
See [Durable execution](https://github.com/LinuxDevil/agent-sdk/blob/main/docs/durable-execution.md#fork-and-replay).

**Example:** `agent.fork('job-1', { fromStep: 9 })` after a 3-step run.

### LOUSHY_RUN_ALREADY_ITERATED

**Means:** an `AgentRun` from `session.stream()` was iterated a second time.
Expand Down
15 changes: 15 additions & 0 deletions src/createAgent.ts
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ import type { SubagentSpec } from './execution/delegation';
import type { ApprovalStore } from './execution/ApprovalGate';
import { InMemoryApprovalStore } from './execution/InMemoryApprovalStore';
import { resumeAfterApproval } from './execution/resume';
import type { ForkOptions, ForkResult } from './execution/checkpoint';
import { ConfigurationError } from './execution/errors';
import { createAgentApprovals, type AgentApprovals, type ApproveToolCall } from './createAgentApprovals';
import type { PermissionOptions } from './execution/permissions';
Expand Down Expand Up @@ -392,6 +393,18 @@ export interface SimpleAgent<TObject = unknown> {
* ```
*/
resume: (sessionId: string, options?: { signal?: AbortSignal }) => Promise<ExecutionResult<TObject> | null>;
/**
* Forks the `sessionId` run at `fromStep` (LOU-D44): `AgentExecutor.fork()`
* over the agent's `store.checkpoints`, which must keep a history. Continue
* the fork with `agent.resume(fork.sessionId)`; the original is unchanged.
*
* @example
* ```ts
* const fork = await agent.fork('job-1', { fromStep: 1, patch: { appendInput: 'Use Celsius.' } });
* const replayed = await agent.resume(fork.sessionId);
* ```
*/
fork: (sessionId: string, options: Omit<ForkOptions, 'sessionId' | 'checkpointStore'>) => Promise<ForkResult>;
/**
* Tool calls this agent is paused on, waiting for approval (LOU-D21). A
* paused `send()` resolves with `finishReason: 'awaiting-approval'` and an
Expand Down Expand Up @@ -541,6 +554,8 @@ export function createAgent<TOutput extends z.ZodTypeAny = z.ZodUnknown>(
if (checkpoint.status === 'finished') return null;
return approvals.settle(await run([], signal, durable(sessionId)), signal) as Promise<ExecutionResult<Typed>>;
},
// durable() throws LOUSHY_CONFIG_MISSING_CHECKPOINT_STORE without `store.checkpoints`.
fork: async (sessionId, options) => AgentExecutor.fork({ ...options, ...(durable(sessionId) as SessionTurnCheckpoint) }),
approvals: approvals.approvals,
ready: mcp.ready,
close: mcp.close,
Expand Down
30 changes: 29 additions & 1 deletion src/evals/drift.test.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import { describe, it, expect } from 'vitest';
import type { Cassette } from '../testing/cassette';
import { diffTrajectories, trajectoryOf, type Trajectory } from './drift';
import type { Message } from '../providers';
import { compareTrajectories, diffTrajectories, trajectoryOf, type Trajectory } from './drift';

type Entry = Cassette['entries'][number];

Expand Down Expand Up @@ -77,3 +78,30 @@ describe('diffTrajectories', () => {
expect(diffTrajectories(base, pricier, { usage: true })).toEqual([{ field: 'tokens', committed: '15', current: '905' }]);
});
});

describe('compareTrajectories (LOU-D44)', () => {
const transcript = (callId: string, args: string, result: string): Message[] => [
{ role: 'user', content: 'refund order 42' },
{ role: 'assistant', content: '', toolCalls: [{ id: callId, type: 'function', function: { name: 'lookup', arguments: args } }] },
{ role: 'tool', content: result, toolCallId: callId, toolName: 'lookup' },
{ role: 'assistant', content: 'Done.' },
];

it('matches transcripts that differ only in call ids and argument key order', () => {
const diff = compareTrajectories(transcript('a', '{"id":42,"x":1}', 'ok'), transcript('b', '{"x":1,"id":42}', 'ok'));
expect(diff.divergedAt).toBeUndefined();
expect(diff.drift).toEqual([]);
expect(diff.a).toEqual([
{ text: '', tools: [{ id: 'a', name: 'lookup', args: '{"id":42,"x":1}', result: 'ok' }] },
{ text: 'Done.', tools: [] },
]);
});

it('reports the first step whose tool result, arguments or text differ', () => {
expect(compareTrajectories(transcript('a', '{}', 'ok'), transcript('a', '{}', 'gone')).divergedAt).toBe(1);
const args = compareTrajectories(transcript('a', '{"id":1}', 'ok'), transcript('a', '{"id":2}', 'ok'));
expect(args.drift).toEqual([{ field: 'args', committed: 'lookup {"id":1}', current: 'lookup {"id":2}' }]);
const shorter = compareTrajectories(transcript('a', '{}', 'ok'), transcript('a', '{}', 'ok').slice(0, 3));
expect(shorter).toMatchObject({ divergedAt: 2, drift: [{ field: 'steps', committed: '2', current: '1' }] });
});
});
Loading
Loading