From 3d29965ba73a072a6c6b21df9933b84daaeb38df Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 1 Oct 2026 16:18:20 +0000 Subject: [PATCH] [EPIC-W][LOU-W3.2] compaction.start/compaction.done stream events and createAgent({ compaction }) The compaction hook now emits typed compaction.start / compaction.done events into streamed runs (via a new optional GenerateHookContext.emit), and createAgent() takes `compaction` (true or an options object, where `summarizer` selects twoPhaseStrategy) and `hooks`. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_016bY29ibQgv3MDdjAbUT4YR --- CHANGELOG.md | 1 + docs/api-overview.md | 5 +- docs/compaction.md | 60 ++++++- docs/configuration.md | 2 + docs/streaming.md | 7 +- llms-full.txt | 74 +++++++- src/context/agentCompaction.ts | 40 +++++ src/context/compaction.ts | 15 +- src/context/compactionEvents.test.ts | 247 +++++++++++++++++++++++++++ src/context/index.ts | 1 + src/createAgent.ts | 41 ++++- src/execution/agentEvents.test-d.ts | 11 ++ src/execution/agentEvents.ts | 38 +++++ src/execution/agentRun.ts | 5 +- src/execution/generateStep.ts | 1 + src/execution/hooks.ts | 11 ++ 16 files changed, 543 insertions(+), 16 deletions(-) create mode 100644 src/context/agentCompaction.ts create mode 100644 src/context/compactionEvents.test.ts diff --git a/CHANGELOG.md b/CHANGELOG.md index e0b2919ec..0ef52970f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] - 2026-09-28 ### Added +- Compaction events and the `compaction` option (LOU-W3.2): `agent.stream()`, `session.stream()` and `AgentExecutor.stream()` emit two new typed events whenever the compaction hook compacts a request: `compaction.start` (`strategy`, `tokensBefore`, `contextWindow`, `thresholdTokens`) before the work and `compaction.done` (`strategy`, `tokensBefore`, `tokensAfter`, `prunedToolCallIds`, `summary?`, `error?: { message }`) after it, inside the step and before the model call; `error` is set when the strategy failed and the hook fell back. Additive: the event schema version stays 1. `GenerateHookContext` gains the optional `emit(event)` (set in streamed runs only) that the hook uses. `createAgent({ compaction })` takes `true` (the prune hook with its defaults) or `{ strategy?, thresholdPercent?, contextWindow?, protectedTokens?, summarizer? }`, where `summarizer` (a `'provider/model'` string or an `LLMProvider`) selects `twoPhaseStrategy()` with that model; `summarizer` together with `strategy` is a `ConfigurationError`. `createAgent({ hooks })` (new) takes any `AgentHook[]`, run before the compaction hook. See docs/compaction.md. - Connect MCP servers from config (LOU-Z4): `connectMcp(servers, { lazy?, onError?, logger? })` (`@loushy/build-ai-agent/mcp` and the root) connects stdio (`command`/`args`/`env`) and streamable HTTP (`url`/`headers`) servers, lists their tools as `__` descriptors and returns `{ tools, close(), status() }`. Every server connects up front (listing tools needs a connection); `lazy` (default `true`) reconnects on the next tool call after `close()` or a dropped connection. `onError: 'skip'` leaves a failing server out with a warning instead of rejecting. `createAgent({ mcpServers })` connects them on the new `agent.ready()` or the first `send()` / `stream()` and adds their tools; the new `agent.close()` disconnects them (both are no-ops without `mcpServers`). `specToAgent()` now connects `spec.mcpServers` the same way (closes TODO(LOU-D20.2)); `agent.mcpServers` still lists them. `loadMcpTools()` accepts any `{ listTools, callTool }` (`McpClientLike`). See docs/configuration.md#connect-mcp-servers-mcpservers-connectmcp. - Error codes with fixes (LOU-D2): every `SDKError` now has a stable `code` (`LOUSHY__`, e.g. `LOUSHY_CONFIG_MISSING_PROVIDER`, `LOUSHY_PEER_MISSING`, `LOUSHY_SPEC_INVALID`, `LOUSHY_SESSION_AWAITING_APPROVAL`), a one-sentence `hint` and a `docs` link to its section of the new docs/errors.md; its message ends with a `[code] hint (docs)` line (`detail` is the message without it), and `toString()` always shows it. `ERROR_CODES` / `ErrorCode` (new exports) are the registry, and a test keeps it, the codes used in `src/` and docs/errors.md in sync. The plain `Error`s thrown by `createAgent()` (no model, `instructions` + `prompt`, `sessionId` without checkpoints), `resolveProvider()` (bad spec, unknown prefix, missing API key, missing peer), `AgentExecutor.execute()` / `stream()` (missing `provider` / `agent` / `input`, approval with no `approvalStore`), `resumeAfterApproval()` (unknown approval), sessions (invalid id, corrupt file, second iteration, no stream runner), `loadSpec()` and `specToAgent()` are now `ConfigurationError` / `ValidationError` / `SDKError` with codes; `MissingPeerDependencyError` extends `SDKError`. Existing message text is unchanged (the help line is appended). `loadSpec()` suggests the field a top-level typo meant (`unknown field 'promt' (did you mean 'prompt'?)`). See docs/errors.md. - Checkpoint history (LOU-D43): `memoryStore()`, `SqliteStore` and `LocalStorageCheckpointStore` keep a bounded history per session: every `save()` also appends the record to a ring (`historyLimit` option, default 50, `0` keeps none; the oldest are dropped), and the new optional `CheckpointStore.history(sessionId, { limit? })` returns `{ step, savedAt, status, checkpoint }` entries newest first. `CheckpointStore.delete(sessionId, { keepHistory: true })` keeps the history (by default `delete()` clears it too), `getCheckpointHistory(store, sessionId)` reads it and returns `undefined` for a store without `history()`, and `SqliteStore` gains the `checkpoint_history` table (added when an existing database file is opened; `prune()` removes old entries). New types `CheckpointHistoryEntry`, `CheckpointHistoryOptions`, `CheckpointDeleteOptions` and `MemoryStoreOptions`. `KVCheckpointStore` and Agent Forge's file store keep no history yet (LOU-D43.2). This is the first step of time-travel from a past step. See docs/durable-execution.md#checkpoint-history. diff --git a/docs/api-overview.md b/docs/api-overview.md index 3efaaccfb..3c5472ef3 100644 --- a/docs/api-overview.md +++ b/docs/api-overview.md @@ -517,7 +517,10 @@ and, if the run is still too big, replaces old turns with a summary written by `model`; `summarizeStrategy()` only summarizes. `pinMessage(message)` marks a message that is never pruned or summarized. `compactMessages(messages, options)` does the same once, by hand (async), and `CompactionStrategy` is the -pluggable interface (`compact()` may be async). See [Context compaction](./compaction.md). +pluggable interface (`compact()` may be async). `createAgent({ compaction: true })` +installs the hook on an agent (`createAgent({ hooks })` takes any other +`AgentHook`s), and `stream()` emits `compaction.start` / `compaction.done`. +See [Context compaction](./compaction.md). ## Flows, evals, observability and security diff --git a/docs/compaction.md b/docs/compaction.md index 16605af6d..ccd7469fb 100644 --- a/docs/compaction.md +++ b/docs/compaction.md @@ -7,6 +7,60 @@ first it replaces old tool results with a short marker, and if that is not enough it replaces the oldest turns with a summary written by a (cheaper) model. +## Compacting an agent + +With `createAgent()`, turn compaction on with the `compaction` option. `true` +installs the hook with its defaults (prune old tool results above 90% of the +model's window); an object configures it, and `summarizer` selects +`twoPhaseStrategy()` with that model (a `"provider/model"` string or an +`LLMProvider`): + +```ts +import { createAgent } from '@loushy/build-ai-agent'; + +const simple = createAgent({ model: 'openai/gpt-4o', compaction: true }); + +const agent = createAgent({ + model: 'openai/gpt-4o', + compaction: { + thresholdPercent: 0.8, // compact above 80% of the context window (default 0.9) + protectedTokens: 20_000, // never change the newest 20K tokens (default 40_000) + summarizer: 'openai/gpt-4o-mini', // prune, then summarize if still too big + }, +}); + +for await (const event of agent.stream('Audit the repository.')) { + if (event.type === 'compaction.done') console.log(`${event.tokensBefore} -> ${event.tokensAfter} tokens`); +} +``` + +The object takes `strategy`, `thresholdPercent`, `contextWindow` and +`protectedTokens` (as in the table below) and `summarizer`; combining +`summarizer` with `strategy` is a configuration error (pass +`twoPhaseStrategy({ model })` as the `strategy` instead). `createAgent({ hooks })` +takes any other `AgentHook`s, which run before the compaction hook, so +compaction sees what they added. Hooks and compaction apply to `send()`, +`stream()` and sessions. + +### Stream events + +`agent.stream()`, `session.stream()` and `AgentExecutor.stream()` report each +compaction as two events, inside the step and before the model call that +triggered it (see [Streaming](./streaming.md#event-schema-version-1)): + +| `type` | Fields | +| --- | --- | +| `compaction.start` | `strategy`, `tokensBefore`, `contextWindow`, `thresholdTokens` | +| `compaction.done` | `strategy`, `tokensBefore`, `tokensAfter`, `prunedToolCallIds`, `summary?: boolean`, `error?: { message }` | + +Every `compaction.start` is followed by exactly one `compaction.done`. When the +strategy could not shrink anything, `tokensAfter` equals `tokensBefore`; when it +failed (or the summarizer failed and the hook fell back to pruning), `error` is +set and the run continues. `summary` is `true` when old turns were replaced by +a summary (the text itself is not sent; use `onCompaction` for it). A +non-streaming `send()` emits no events. Hooks add their own events with +`ctx.emit?.(...)` on the `preGenerate` context, which exists only in streamed runs. + ## Compacting a run `createCompactionHook()` returns an `AgentHook` named `compaction`. Register it @@ -192,6 +246,6 @@ const keepRecent: CompactionStrategy = { const hook = createCompactionHook({ strategy: keepRecent }); ``` -Typed `compaction.*` stream events and a `createAgent({ compaction })` option -are planned (LOU-W3.2); until then `onCompaction` is where to observe -compactions. +A custom strategy gets the same `compaction.start` / `compaction.done` events as +the built-in ones. `onCompaction` (on `createCompactionHook()`) still receives +the summary text and the `Error`. diff --git a/docs/configuration.md b/docs/configuration.md index 8d49d77df..e1dbedb79 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -334,6 +334,8 @@ const agent = createAgent({ prompt: 'You are helpful.', provider }); | `projectInstructions` | `true` or `{ cwd?, files? }`: append the nearest `AGENTS.md` / `CLAUDE.md` to the instructions (off by default; see [Project instructions](#project-instructions)). | | `retry` | `withRetry()` options for failed model calls, or `false`. Default `{ maxRetries: 2 }` for `model` strings; a `provider` instance is wrapped only when set. See [Provider retries and fallback](#provider-retries-and-fallback). | | `fallbackModels` | `provider/model` strings tried in order when the model still fails after its retries. | +| `hooks` | `AgentHook[]` run around every model call and tool call, in order, before the compaction hook (see `AgentHook` in the [API overview](./api-overview.md)). | +| `compaction` | `true` (prune old tool results above 90% of the context window) or `{ strategy?, thresholdPercent?, contextWindow?, protectedTokens?, summarizer? }`; `summarizer` (`'provider/model'` or an `LLMProvider`) selects the two-phase strategy. `stream()` reports `compaction.start` / `compaction.done`. See [Context compaction](./compaction.md#compacting-an-agent). | With neither `model` nor `provider`, `createAgent()` resolves from the environment: `LOUSHY_MODEL` (a `'provider/model'` string) if set, otherwise diff --git a/docs/streaming.md b/docs/streaming.md index ce1c34eb2..265a8d4cc 100644 --- a/docs/streaming.md +++ b/docs/streaming.md @@ -111,6 +111,8 @@ The event types and their extra fields: | `error` | `error: { name: string, message: string }` | An error. If it ends the run, `run.done` with `finishReason: 'error'` follows. A provider error retried under `surfaceRetryableProviderErrors` is followed by further steps instead. | | `provider.retry` | `attempt: number`, `maxRetries: number`, `delayMs: number`, `error: { message: string, category?: string }`, `provider: string` | A model call failed and is retried after `delayMs` (`createAgent({ retry })` or any `withRetry()` provider). `attempt` is the attempt that failed (1 = first); `category` is `'rate-limit'`, `'timeout'`, ... and absent when unknown (a 5xx, for example). | | `provider.fallback` | `from: string`, `to: string`, `error: { message: string }` | A model call still failed after its retries and the next provider takes over (`createAgent({ fallbackModels })` or any `withFallback()` provider). `from` and `to` are provider names. | +| `compaction.start` | `strategy: string`, `tokensBefore: number`, `contextWindow: number`, `thresholdTokens: number` | The compaction hook (`createAgent({ compaction })`) found the next model request above its threshold and starts compacting it. Exactly one `compaction.done` follows. See [Context compaction](./compaction.md). | +| `compaction.done` | `strategy: string`, `tokensBefore: number`, `tokensAfter: number`, `prunedToolCallIds: string[]`, `summary?: boolean`, `error?: { message: string }` | A compaction ended. `tokensAfter` equals `tokensBefore` when nothing could be compacted; `summary` is set when old turns were replaced by a summary; `error` is set when the strategy failed or fell back (the run continues). | | `run.done` | `finishReason: string`, `text: string`, `usage?: { promptTokens, completionTokens, totalTokens }`, `object?: unknown` | Last event of every run, exactly once, including aborted, failed and awaiting-approval runs. `finishReason` and `text` match `run.result` (`'max-steps'` when the `maxSteps` budget ran out while the model still wanted to continue); a failed run has `finishReason: 'error'`, `text: ''` and no `usage`. `object` is `run.result`'s validated `object` for an agent with an `output` schema (see [Structured output](./structured-output.md)), absent otherwise. | Optional fields are left out when they have no value. They are never @@ -122,8 +124,9 @@ Optional fields are left out when they have no value. They are never - Each `step.start` is followed by exactly one `step.done` with the same `step`, before the next `step.start`. Everything a step does happens between the two. -- Inside a step: `provider.retry` / `provider.fallback` events (if the model - call fails), then `text.delta` events, then `text.done`, then the tool events. +- Inside a step: `compaction.start` / `compaction.done` (if the request was + compacted, before the model call), `provider.retry` / `provider.fallback` + events (if the model call fails), then `text.delta` events, then `text.done`, then the tool events. - Tool calls of one step run in parallel (see `toolConcurrency`): `tool.start` events come in the model's call order and `tool.done` / `tool.error` events in completion order. Match them by `toolCallId`. diff --git a/llms-full.txt b/llms-full.txt index 99718831b..68d18d709 100644 --- a/llms-full.txt +++ b/llms-full.txt @@ -1195,7 +1195,10 @@ and, if the run is still too big, replaces old turns with a summary written by `model`; `summarizeStrategy()` only summarizes. `pinMessage(message)` marks a message that is never pruned or summarized. `compactMessages(messages, options)` does the same once, by hand (async), and `CompactionStrategy` is the -pluggable interface (`compact()` may be async). See [Context compaction](https://github.com/LinuxDevil/agent-sdk/blob/main/docs/compaction.md). +pluggable interface (`compact()` may be async). `createAgent({ compaction: true })` +installs the hook on an agent (`createAgent({ hooks })` takes any other +`AgentHook`s), and `stream()` emits `compaction.start` / `compaction.done`. +See [Context compaction](https://github.com/LinuxDevil/agent-sdk/blob/main/docs/compaction.md). ## Flows, evals, observability and security @@ -2047,6 +2050,60 @@ first it replaces old tool results with a short marker, and if that is not enough it replaces the oldest turns with a summary written by a (cheaper) model. +## Compacting an agent + +With `createAgent()`, turn compaction on with the `compaction` option. `true` +installs the hook with its defaults (prune old tool results above 90% of the +model's window); an object configures it, and `summarizer` selects +`twoPhaseStrategy()` with that model (a `"provider/model"` string or an +`LLMProvider`): + +```ts +import { createAgent } from '@loushy/build-ai-agent'; + +const simple = createAgent({ model: 'openai/gpt-4o', compaction: true }); + +const agent = createAgent({ + model: 'openai/gpt-4o', + compaction: { + thresholdPercent: 0.8, // compact above 80% of the context window (default 0.9) + protectedTokens: 20_000, // never change the newest 20K tokens (default 40_000) + summarizer: 'openai/gpt-4o-mini', // prune, then summarize if still too big + }, +}); + +for await (const event of agent.stream('Audit the repository.')) { + if (event.type === 'compaction.done') console.log(`${event.tokensBefore} -> ${event.tokensAfter} tokens`); +} +``` + +The object takes `strategy`, `thresholdPercent`, `contextWindow` and +`protectedTokens` (as in the table below) and `summarizer`; combining +`summarizer` with `strategy` is a configuration error (pass +`twoPhaseStrategy({ model })` as the `strategy` instead). `createAgent({ hooks })` +takes any other `AgentHook`s, which run before the compaction hook, so +compaction sees what they added. Hooks and compaction apply to `send()`, +`stream()` and sessions. + +### Stream events + +`agent.stream()`, `session.stream()` and `AgentExecutor.stream()` report each +compaction as two events, inside the step and before the model call that +triggered it (see [Streaming](https://github.com/LinuxDevil/agent-sdk/blob/main/docs/streaming.md#event-schema-version-1)): + +| `type` | Fields | +| --- | --- | +| `compaction.start` | `strategy`, `tokensBefore`, `contextWindow`, `thresholdTokens` | +| `compaction.done` | `strategy`, `tokensBefore`, `tokensAfter`, `prunedToolCallIds`, `summary?: boolean`, `error?: { message }` | + +Every `compaction.start` is followed by exactly one `compaction.done`. When the +strategy could not shrink anything, `tokensAfter` equals `tokensBefore`; when it +failed (or the summarizer failed and the hook fell back to pruning), `error` is +set and the run continues. `summary` is `true` when old turns were replaced by +a summary (the text itself is not sent; use `onCompaction` for it). A +non-streaming `send()` emits no events. Hooks add their own events with +`ctx.emit?.(...)` on the `preGenerate` context, which exists only in streamed runs. + ## Compacting a run `createCompactionHook()` returns an `AgentHook` named `compaction`. Register it @@ -2232,9 +2289,9 @@ const keepRecent: CompactionStrategy = { const hook = createCompactionHook({ strategy: keepRecent }); ``` -Typed `compaction.*` stream events and a `createAgent({ compaction })` option -are planned (LOU-W3.2); until then `onCompaction` is where to observe -compactions. +A custom strategy gets the same `compaction.start` / `compaction.done` events as +the built-in ones. `onCompaction` (on `createCompactionHook()`) still receives +the summary text and the `Error`. # Configuration @@ -2574,6 +2631,8 @@ const agent = createAgent({ prompt: 'You are helpful.', provider }); | `projectInstructions` | `true` or `{ cwd?, files? }`: append the nearest `AGENTS.md` / `CLAUDE.md` to the instructions (off by default; see [Project instructions](#project-instructions)). | | `retry` | `withRetry()` options for failed model calls, or `false`. Default `{ maxRetries: 2 }` for `model` strings; a `provider` instance is wrapped only when set. See [Provider retries and fallback](#provider-retries-and-fallback). | | `fallbackModels` | `provider/model` strings tried in order when the model still fails after its retries. | +| `hooks` | `AgentHook[]` run around every model call and tool call, in order, before the compaction hook (see `AgentHook` in the [API overview](https://github.com/LinuxDevil/agent-sdk/blob/main/docs/api-overview.md)). | +| `compaction` | `true` (prune old tool results above 90% of the context window) or `{ strategy?, thresholdPercent?, contextWindow?, protectedTokens?, summarizer? }`; `summarizer` (`'provider/model'` or an `LLMProvider`) selects the two-phase strategy. `stream()` reports `compaction.start` / `compaction.done`. See [Context compaction](https://github.com/LinuxDevil/agent-sdk/blob/main/docs/compaction.md#compacting-an-agent). | With neither `model` nor `provider`, `createAgent()` resolves from the environment: `LOUSHY_MODEL` (a `'provider/model'` string) if set, otherwise @@ -4867,6 +4926,8 @@ The event types and their extra fields: | `error` | `error: { name: string, message: string }` | An error. If it ends the run, `run.done` with `finishReason: 'error'` follows. A provider error retried under `surfaceRetryableProviderErrors` is followed by further steps instead. | | `provider.retry` | `attempt: number`, `maxRetries: number`, `delayMs: number`, `error: { message: string, category?: string }`, `provider: string` | A model call failed and is retried after `delayMs` (`createAgent({ retry })` or any `withRetry()` provider). `attempt` is the attempt that failed (1 = first); `category` is `'rate-limit'`, `'timeout'`, ... and absent when unknown (a 5xx, for example). | | `provider.fallback` | `from: string`, `to: string`, `error: { message: string }` | A model call still failed after its retries and the next provider takes over (`createAgent({ fallbackModels })` or any `withFallback()` provider). `from` and `to` are provider names. | +| `compaction.start` | `strategy: string`, `tokensBefore: number`, `contextWindow: number`, `thresholdTokens: number` | The compaction hook (`createAgent({ compaction })`) found the next model request above its threshold and starts compacting it. Exactly one `compaction.done` follows. See [Context compaction](https://github.com/LinuxDevil/agent-sdk/blob/main/docs/compaction.md). | +| `compaction.done` | `strategy: string`, `tokensBefore: number`, `tokensAfter: number`, `prunedToolCallIds: string[]`, `summary?: boolean`, `error?: { message: string }` | A compaction ended. `tokensAfter` equals `tokensBefore` when nothing could be compacted; `summary` is set when old turns were replaced by a summary; `error` is set when the strategy failed or fell back (the run continues). | | `run.done` | `finishReason: string`, `text: string`, `usage?: { promptTokens, completionTokens, totalTokens }`, `object?: unknown` | Last event of every run, exactly once, including aborted, failed and awaiting-approval runs. `finishReason` and `text` match `run.result` (`'max-steps'` when the `maxSteps` budget ran out while the model still wanted to continue); a failed run has `finishReason: 'error'`, `text: ''` and no `usage`. `object` is `run.result`'s validated `object` for an agent with an `output` schema (see [Structured output](https://github.com/LinuxDevil/agent-sdk/blob/main/docs/structured-output.md)), absent otherwise. | Optional fields are left out when they have no value. They are never @@ -4878,8 +4939,9 @@ Optional fields are left out when they have no value. They are never - Each `step.start` is followed by exactly one `step.done` with the same `step`, before the next `step.start`. Everything a step does happens between the two. -- Inside a step: `provider.retry` / `provider.fallback` events (if the model - call fails), then `text.delta` events, then `text.done`, then the tool events. +- Inside a step: `compaction.start` / `compaction.done` (if the request was + compacted, before the model call), `provider.retry` / `provider.fallback` + events (if the model call fails), then `text.delta` events, then `text.done`, then the tool events. - Tool calls of one step run in parallel (see `toolConcurrency`): `tool.start` events come in the model's call order and `tool.done` / `tool.error` events in completion order. Match them by `toolCallId`. diff --git a/src/context/agentCompaction.ts b/src/context/agentCompaction.ts new file mode 100644 index 000000000..4a55600b9 --- /dev/null +++ b/src/context/agentCompaction.ts @@ -0,0 +1,40 @@ +/** + * `createAgent({ compaction })` (LOU-W3.2): turns the option into the + * compaction hook it installs. + */ + +import type { LLMProvider } from '../providers/llm'; +import type { AgentHook } from '../execution/hooks'; +import { ConfigurationError } from '../execution/errors'; +import { createCompactionHook, twoPhaseStrategy, type CompactionHookOptions } from './compaction'; + +/** The object form of `createAgent({ compaction })`. */ +export interface AgentCompactionOptions extends Pick { + /** + * A model that summarizes old turns: a `"provider/model"` spec or an + * `LLMProvider`. Selects `twoPhaseStrategy()` with it (prune first, then + * summarize if still too big). Do not combine it with `strategy`. + */ + summarizer?: LLMProvider | string; +} + +/** What `createAgent({ compaction })` takes: `true` for the defaults, or {@link AgentCompactionOptions}. */ +export type AgentCompaction = boolean | AgentCompactionOptions; + +/** The hook `createAgent({ compaction })` installs, or `undefined` when it is `false` or absent. */ +export function compactionHookFor(compaction: AgentCompaction | undefined): AgentHook | undefined { + if (!compaction) return undefined; + const { summarizer, ...options } = compaction === true ? ({} as AgentCompactionOptions) : compaction; + if (summarizer !== undefined && options.strategy) { + throw new ConfigurationError( + "createAgent: compaction has both 'strategy' and 'summarizer'. 'summarizer' selects twoPhaseStrategy(); " + + "drop it and pass your own strategy (e.g. twoPhaseStrategy({ model, protectedTokens })) as 'strategy'.", + 'compaction', + 'LOUSHY_CONFIG_CONFLICTING_OPTIONS' + ); + } + return createCompactionHook({ + ...options, + ...(summarizer !== undefined && { strategy: twoPhaseStrategy({ model: summarizer }) }), + }); +} diff --git a/src/context/compaction.ts b/src/context/compaction.ts index ac7fee2b2..a8f2d2fc6 100644 --- a/src/context/compaction.ts +++ b/src/context/compaction.ts @@ -341,16 +341,27 @@ export function createCompactionHook(options: CompactionHookOptions = {}): Agent const { strategy, input } = prepare(messages, { ...options, model: ctx.request.model }, ctx.request.signal); const tokens = input.estimateTokens(messages); if (tokens <= input.thresholdTokens) return; + const { contextWindow, thresholdTokens } = input; + ctx.emit?.({ type: 'compaction.start', strategy: strategy.name, tokensBefore: tokens, contextWindow, thresholdTokens }); let result: CompactionResult; try { result = await strategy.compact(input); } catch (cause) { result = { messages, tokensBefore: tokens, tokensAfter: tokens, prunedToolCallIds: [], error: toError(cause) }; } - if (result.messages === messages && !result.error) return; + const { tokensBefore, tokensAfter, prunedToolCallIds, summary, error } = result; + ctx.emit?.({ + type: 'compaction.done', + strategy: strategy.name, + tokensBefore, + tokensAfter, + prunedToolCallIds, + ...(summary && { summary: true }), + ...(error && { error: { message: error.message } }), + }); + if (result.messages === messages && !error) return; // In place: request.messages is the run's transcript (see the file comment). if (result.messages !== messages) messages.splice(0, messages.length, ...result.messages); - const { tokensBefore, tokensAfter, prunedToolCallIds, summary, error } = result; onCompaction?.({ tokensBefore, tokensAfter, prunedToolCallIds, strategy: strategy.name, summary, error }); }, }; diff --git a/src/context/compactionEvents.test.ts b/src/context/compactionEvents.test.ts new file mode 100644 index 000000000..e6bf3dc48 --- /dev/null +++ b/src/context/compactionEvents.test.ts @@ -0,0 +1,247 @@ +/** + * LOU-W3.2: `compaction.start` / `compaction.done` stream events and + * `createAgent({ compaction, hooks })`. No network: scripted `mockModel`s. + */ +import { afterEach, describe, expect, it, vi } from 'vitest'; +import { z } from 'zod'; +import { createAgent, type CreateAgentConfig } from '../createAgent'; +import { LLMProviderRegistry } from '../providers/llm'; +import type { Message } from '../providers'; +import { estimateTokens } from '../models'; +import { defineTool } from '../tools/defineTool'; +import type { AgentEvent, AgentEventOf, AgentRun, AgentHook } from '../execution'; +import { mockModel } from '../testing'; +import { createCompactionHook, type CompactionStrategy } from './compaction'; + +const BIG = 'x'.repeat(4_000); // about 1,000 tokens +const HUGE = 'y'.repeat(150_000); // about 37,500 tokens + +const fetchPage = (size: string) => + defineTool({ + name: 'fetch_page', + description: 'Fetch a page', + input: z.object({ n: z.number() }), + execute: async ({ n }) => `${n}:${size}`, + }); + +const pageCall = (n: number) => ({ toolCalls: [{ name: 'fetch_page', args: { n }, id: `call_${n}` }] }); + +async function collect(run: AgentRun): Promise { + const events: AgentEvent[] = []; + for await (const event of run) events.push(event); + return events; +} + +const ofType = (events: AgentEvent[], type: T) => + events.filter((e): e is AgentEventOf => e.type === type); + +/** A small window: compaction starts once a request passes 2,700 tokens. */ +const SMALL = { contextWindow: 3_000, protectedTokens: 1_500 }; +const SUMMARIZING = { contextWindow: 3_000, protectedTokens: 1_000, thresholdPercent: 0.3 }; + +function agentWith(compaction: CreateAgentConfig['compaction'], size = BIG, extra: Partial = {}) { + const model = mockModel([pageCall(1), pageCall(2), pageCall(3), pageCall(4), 'done']); + const agent = createAgent({ provider: model, tools: [fetchPage(size)], maxSteps: 10, compaction, ...extra }); + return { agent, model }; +} + +describe('compaction stream events (LOU-W3.2)', () => { + afterEach(() => { + vi.unstubAllEnvs(); + vi.restoreAllMocks(); + }); + + it('emits compaction.start then compaction.done inside the step, before the model call that triggered them', async () => { + const { agent, model } = agentWith(SMALL); + const run = agent.stream('read four pages'); + const events = await collect(run); + const result = await run.result; + + expect(result.text).toBe('done'); + const types = events.map((e) => e.type); + const start = types.indexOf('compaction.start'); + expect(start).toBeGreaterThan(0); + expect(types[start - 1]).toBe('step.start'); + expect(types[start + 1]).toBe('compaction.done'); + // The model call of that step comes after: its text events follow the compaction. + expect(types.slice(start + 2)).toContain('text.delta'); + expect(events.map((e) => e.seq)).toEqual(events.map((_, i) => i)); + + const [started] = ofType(events, 'compaction.start'); + const [done] = ofType(events, 'compaction.done'); + expect(started).toMatchObject({ strategy: 'prune-tool-results', contextWindow: 3_000, thresholdTokens: 2_700 }); + expect(started.tokensBefore).toBeGreaterThan(2_700); + expect(done).toMatchObject({ + strategy: 'prune-tool-results', + tokensBefore: started.tokensBefore, + prunedToolCallIds: ['call_1', 'call_2'], + }); + expect(done.tokensAfter).toBeLessThan(done.tokensBefore); + expect(done).not.toHaveProperty('summary'); + expect(done).not.toHaveProperty('error'); + // Every start has exactly one done, and the request of that step was already compacted. + expect(ofType(events, 'compaction.start')).toHaveLength(ofType(events, 'compaction.done').length); + expect(estimateTokens(model.calls.at(-1)?.messages as Message[])).toBeLessThan(2_700); + // Events are plain JSON. + expect(JSON.parse(JSON.stringify(started))).toEqual(started); + }); + + it('emits nothing below the threshold, and nothing from a non-streaming send()', async () => { + const quiet = agentWith({ contextWindow: 1_000_000 }); + expect(ofType(await collect(quiet.agent.stream('go')), 'compaction.start')).toEqual([]); + + const { agent, model } = agentWith(SMALL); + const result = await agent.send('read four pages'); + expect(result.text).toBe('done'); + expect(result.messages.some((m) => m.content.startsWith('[pruned: fetch_page result'))).toBe(true); + expect(model.calls).toHaveLength(5); + }); + + it('createAgent({ compaction: true }) compacts a long transcript with the defaults', async () => { + const { agent, model } = agentWith(true, HUGE); + const run = agent.stream('read four pages'); + const events = await collect(run); + const result = await run.result; + + expect(result.text).toBe('done'); + const [started] = ofType(events, 'compaction.start'); + const [done] = ofType(events, 'compaction.done'); + expect(started).toMatchObject({ strategy: 'prune-tool-results', contextWindow: 128_000 }); + expect(done.prunedToolCallIds.length).toBeGreaterThan(0); + expect(done.tokensAfter).toBeLessThan(started.thresholdTokens); + const markers = result.messages.filter((m) => m.role === 'tool' && m.content.startsWith('[pruned: fetch_page result')); + expect(markers.map((m) => m.toolCallId)).toEqual(done.prunedToolCallIds); + expect(model.calls).toHaveLength(5); + }); + + it('an object with a summarizer provider runs twoPhaseStrategy with it', async () => { + const summarizer = mockModel(['The user wants four pages read.']); + const { agent, model } = agentWith({ ...SUMMARIZING, summarizer }); + const run = agent.stream('read four pages'); + const events = await collect(run); + + expect((await run.result).text).toBe('done'); + expect(summarizer.calls.length).toBeGreaterThan(0); + const done = ofType(events, 'compaction.done').find((e) => e.summary); + expect(done).toMatchObject({ strategy: 'two-phase', summary: true }); + expect(done).not.toHaveProperty('error'); + expect(JSON.stringify(model.calls.at(-1)?.messages)).toContain('[Conversation summary]\\nThe user wants four pages read.'); + }); + + it("resolves a 'provider/model' summarizer with the provider registry", async () => { + vi.stubEnv('OPENAI_API_KEY', 'sk-test'); + const summarizer = mockModel(['A summary.']); + const create = vi.spyOn(LLMProviderRegistry, 'create').mockReturnValue(summarizer); + const { agent } = agentWith({ ...SUMMARIZING, summarizer: 'openai/gpt-4o-mini' }); + + const events = await collect(agent.stream('read four pages')); + + expect(create.mock.calls[0]?.[0]).toBe('openai'); + expect(summarizer.calls.length).toBeGreaterThan(0); + expect(ofType(events, 'compaction.done').some((e) => e.summary)).toBe(true); + }); + + it('reports a failing strategy as compaction.done with an error and keeps the run going', async () => { + const strategy: CompactionStrategy = { + name: 'boom', + compact() { + throw new Error('strategy exploded'); + }, + }; + const { agent } = agentWith({ ...SMALL, strategy }); + const run = agent.stream('read four pages'); + const events = await collect(run); + + expect((await run.result).text).toBe('done'); + const [done] = ofType(events, 'compaction.done'); + expect(done).toMatchObject({ strategy: 'boom', prunedToolCallIds: [], error: { message: 'strategy exploded' } }); + expect(done.tokensAfter).toBe(done.tokensBefore); + }); + + it('reports a failed summarizer (fallback to pruning) on compaction.done', async () => { + const summarizer = mockModel([{ error: new Error('summarizer down') }]); + const { agent } = agentWith({ ...SUMMARIZING, summarizer }); + const events = await collect(agent.stream('read four pages')); + + const failed = ofType(events, 'compaction.done').find((e) => e.error); + expect(failed).toMatchObject({ strategy: 'two-phase', error: { message: 'summarizer down' } }); + expect(failed?.summary).toBeUndefined(); + }); + + it("tags a sub-agent's compaction events with `subagent`", async () => { + const child = createAgent({ + provider: mockModel([pageCall(1), pageCall(2), pageCall(3), pageCall(4), 'child done']), + tools: [fetchPage(BIG)], + description: 'Reads pages', + maxSteps: 10, + }); + const lead = createAgent({ + provider: mockModel([{ toolCalls: [{ name: 'task', args: { agent: 'reader', prompt: 'read', description: 'read pages' }, id: 't1' }] }, 'lead done']), + subagents: { reader: child }, + hooks: [createCompactionHook(SMALL)], + }); + const events = await collect(lead.stream('go')); + const started = ofType(events, 'compaction.start'); + expect(started.length).toBeGreaterThan(0); + expect(started.every((e) => e.subagent?.name === 'reader')).toBe(true); + }); +}); + +describe('createAgent({ hooks, compaction }) options (LOU-W3.2)', () => { + it('forwards `hooks`: they run around every model call, in order, before the compaction hook', async () => { + const seen: string[] = []; + const { agent } = agentWith(SMALL, BIG, { + hooks: [ + { name: 'a', preGenerate: () => void seen.push('a') }, + { + name: 'b', + preToolCall: (ctx) => void seen.push(`tool:${ctx.toolName}`), + // Runs before compaction, so compaction sees what it added. + preGenerate: (ctx) => void seen.push(`b:${ctx.request.messages.length}`), + }, + ], + }); + await agent.send('read four pages'); + expect(seen.slice(0, 3)).toEqual(['a', 'b:2', 'tool:fetch_page']); + expect(seen.filter((s) => s === 'a')).toHaveLength(5); + expect(seen.filter((s) => s === 'tool:fetch_page')).toHaveLength(4); + }); + + it('lets a hook add events with ctx.emit in streamed runs only', async () => { + const hook: AgentHook = { + name: 'emitter', + preGenerate: (ctx) => + ctx.emit?.({ type: 'compaction.start', strategy: 'custom', tokensBefore: 1, contextWindow: 2, thresholdTokens: 3 }), + }; + const agent = createAgent({ provider: mockModel(['hi', 'hi']), hooks: [hook] }); + const events = await collect(agent.stream('hello')); + expect(events.map((e) => e.type)).toEqual([ + 'run.start', + 'step.start', + 'compaction.start', + 'text.delta', + 'text.done', + 'step.done', + 'run.done', + ]); + expect((await agent.send('hello')).text).toBe('hi'); + }); + + it('rejects a summarizer together with a strategy, and a threshold outside (0, 1]', () => { + const strategy: CompactionStrategy = { + name: 's', + compact: (input) => ({ messages: input.messages, tokensBefore: 0, tokensAfter: 0, prunedToolCallIds: [] }), + }; + const provider = mockModel(['x']); + expect(() => createAgent({ provider, compaction: { strategy, summarizer: 'openai/gpt-4o-mini' } })).toThrow( + /both 'strategy' and 'summarizer'/ + ); + expect(() => createAgent({ provider, compaction: { thresholdPercent: 2 } })).toThrow(RangeError); + }); + + it('installs nothing for compaction: false', async () => { + const { agent } = agentWith(false, BIG); + const events = await collect(agent.stream('read four pages')); + expect(ofType(events, 'compaction.start')).toEqual([]); + }); +}); diff --git a/src/context/index.ts b/src/context/index.ts index e0c582839..39b8cc6a6 100644 --- a/src/context/index.ts +++ b/src/context/index.ts @@ -3,3 +3,4 @@ */ export * from './compaction'; +export * from './agentCompaction'; diff --git a/src/createAgent.ts b/src/createAgent.ts index 9017034fc..bcf24f7eb 100644 --- a/src/createAgent.ts +++ b/src/createAgent.ts @@ -43,6 +43,8 @@ import { createAgentApprovals, type AgentApprovals, type ApproveToolCall } from import type { z } from 'zod'; import type { McpServerSpec } from './spec/schema'; import { agentMcp, streamAfter } from './tools/mcp/agentMcp'; +import { HookRegistry, type AgentHook } from './execution/hooks'; +import { compactionHookFor, type AgentCompaction } from './context/agentCompaction'; /** * Options for createAgent() that do not depend on how the instructions and @@ -223,6 +225,32 @@ export interface CreateAgentBase { * ``` */ output?: TOutput; + /** + * Hooks run around every model call and tool call (LOU-W3.2), in the order + * given, before the hook `compaction` installs. See `AgentHook` and + * docs/api-overview.md. They apply to this agent's runs and to its sub-agents'. + * + * @example + * ```ts + * createAgent({ prompt: '...', provider, hooks: [{ name: 'audit', preToolCall: (ctx) => console.log(ctx.toolName) }] }); + * ``` + */ + hooks?: readonly AgentHook[]; + /** + * Keeps long runs under the model's context window (LOU-W3.2): `true` + * installs `createCompactionHook()` with its defaults (prune old tool + * results above 90% of the window); an object sets `strategy`, + * `thresholdPercent`, `contextWindow` and `protectedTokens`, and + * `summarizer` (a `'provider/model'` string or an `LLMProvider`) selects + * `twoPhaseStrategy()` with that model. `stream()` reports each compaction + * as `compaction.start` / `compaction.done` events. See docs/compaction.md. + * + * @example + * ```ts + * createAgent({ model: 'openai/gpt-4o', compaction: { thresholdPercent: 0.8, summarizer: 'openai/gpt-4o-mini' } }); + * ``` + */ + compaction?: AgentCompaction; } /** @@ -437,6 +465,7 @@ export function createAgent( toolConcurrency: config.toolConcurrency, }; const spec: SubagentSpec = { agent, provider, toolRegistry, ...runOptions }; + const hooks = agentHooks(config); const checkpoints = config.store?.checkpoints; const approvals = createAgentApprovals({ store: config.approvalStore ?? config.store?.approvals ?? new InMemoryApprovalStore(), @@ -447,7 +476,7 @@ export function createAgent( approvalStore, toolRegistry ?? new ToolRegistry(), provider, - { ...runOptions, output: config.output, approvalStore, signal }, + { ...runOptions, output: config.output, hooks, approvalStore, signal }, // A run paused under a `sessionId` keeps checkpointing after the decision. checkpointStore ?? checkpoints ), @@ -472,6 +501,7 @@ export function createAgent( ): ExecuteOptions => ({ ...spec, output: config.output, + hooks, approvalStore: approvals.store, input, signal, @@ -515,6 +545,15 @@ export function createAgent( return simpleAgent; } +/** The agent's hooks: `hooks`, then the one `compaction` installs; `undefined` when there are none. */ +function agentHooks({ hooks = [], compaction }: CreateAgentBase): HookRegistry | undefined { + const compactionHook = compactionHookFor(compaction); + if (hooks.length === 0 && !compactionHook) return undefined; + const registry = new HookRegistry(); + registry.registerMany([...hooks, ...(compactionHook ? [compactionHook] : [])]); + return registry; +} + /** Appends the nearest AGENTS.md / CLAUDE.md to `instructions` when `projectInstructions` is set. */ function withProjectInstructions( instructions: string, diff --git a/src/execution/agentEvents.test-d.ts b/src/execution/agentEvents.test-d.ts index eb0d8c249..7f0447790 100644 --- a/src/execution/agentEvents.test-d.ts +++ b/src/execution/agentEvents.test-d.ts @@ -66,6 +66,15 @@ describe('AgentEvent types', () => { if (event.type === 'provider.fallback') expectTypeOf(event.error).toEqualTypeOf<{ message: string }>(); }); + it('narrows compaction events on event.type (LOU-W3.2)', () => { + if (event.type === 'compaction.start') expectTypeOf(event.thresholdTokens).toBeNumber(); + if (event.type === 'compaction.done') { + expectTypeOf(event.prunedToolCallIds).toEqualTypeOf(); + expectTypeOf(event.summary).toEqualTypeOf(); + expectTypeOf(event.error).toEqualTypeOf<{ message: string } | undefined>(); + } + }); + it('covers every event type', () => { expectTypeOf().toEqualTypeOf< | 'run.start' @@ -80,6 +89,8 @@ describe('AgentEvent types', () => { | 'error' | 'provider.retry' | 'provider.fallback' + | 'compaction.start' + | 'compaction.done' | 'run.done' >(); }); diff --git a/src/execution/agentEvents.ts b/src/execution/agentEvents.ts index 596dc4377..bd63ea629 100644 --- a/src/execution/agentEvents.ts +++ b/src/execution/agentEvents.ts @@ -181,6 +181,40 @@ export interface ProviderFallbackEvent extends AgentEventBase<'provider.fallback error: { message: string }; } +/** + * The compaction hook (`createAgent({ compaction })`, `createCompactionHook()`) + * found the next model request above its threshold and starts compacting it + * (LOU-W3.2). Emitted inside the step, before the model call; exactly one + * `compaction.done` follows. + */ +export interface CompactionStartEvent extends AgentEventBase<'compaction.start'> { + /** The strategy's name, e.g. `'prune-tool-results'` or `'two-phase'`. */ + strategy: string; + /** Estimated tokens of the request before compaction. */ + tokensBefore: number; + /** The model's context window, in tokens. */ + contextWindow: number; + /** The size (`thresholdPercent` of the window) the request is compared with. */ + thresholdTokens: number; +} + +/** + * A compaction ended (LOU-W3.2). `tokensAfter` equals `tokensBefore` when + * nothing could be compacted. `error` is set when the strategy failed or fell + * back (e.g. the summarizer failed and only tool results were pruned); the + * run continues either way. + */ +export interface CompactionDoneEvent extends AgentEventBase<'compaction.done'> { + strategy: string; + tokensBefore: number; + tokensAfter: number; + /** `toolCallId`s whose results were replaced by a marker. */ + prunedToolCallIds: string[]; + /** `true` when old turns were replaced by a model-written summary (the text is not sent). */ + summary?: boolean; + error?: { message: string }; +} + /** * Last event of every run, emitted exactly once - also for aborted, failed * and awaiting-approval runs. @@ -220,6 +254,8 @@ export type AgentEvent = | AgentErrorEvent | ProviderRetryEvent | ProviderFallbackEvent + | CompactionStartEvent + | CompactionDoneEvent | RunDoneEvent; /** The `type` of an {@link AgentEvent}. */ @@ -253,6 +289,8 @@ const EVENT_TYPES: ReadonlySet = new Set([ 'error', 'provider.retry', 'provider.fallback', + 'compaction.start', + 'compaction.done', 'run.done', ]); diff --git a/src/execution/agentRun.ts b/src/execution/agentRun.ts index d5becc347..215289e18 100644 --- a/src/execution/agentRun.ts +++ b/src/execution/agentRun.ts @@ -18,7 +18,7 @@ import { newId } from '../utils/id'; import type { GenerateOptions, GenerateResult, LLMProvider } from '../providers'; import type { ExecuteOptions, ExecutionEvent, ExecutionResult } from './AgentExecutor'; import type { PendingApproval } from './ApprovalGate'; -import type { SubagentInfo } from './hooks'; +import type { HookEventPayload, SubagentInfo } from './hooks'; import { AGENT_EVENT_SCHEMA_VERSION, AgentEvent, @@ -72,6 +72,8 @@ export interface RunEventSink { /** `finishReason` overrides the step's own (the model's) finish reason. */ stepDone(step: number, finishReason?: string): void; approvalRequested(pending: PendingApproval): void; + /** LOU-W3.2: an event a hook emits (`GenerateHookContext.emit`). */ + hookEvent(event: HookEventPayload): void; /** Obtains one model step - streamed when the provider can. */ generate(provider: LLMProvider, request: GenerateOptions): Promise; /** LOU-Y1: the sink for a sub-agent's run, whose events carry `subagent`. */ @@ -334,6 +336,7 @@ class AgentRunImpl implements AgentRun { args: toJsonValue(pending.args) as Record, }); }, + hookEvent: (event) => this.emit(event, subagent), generate: async (provider, request) => { const call = withProviderEvents(request, this.providerEvents(subagent)); const generated = await this.generateStep(provider, call, subagent); diff --git a/src/execution/generateStep.ts b/src/execution/generateStep.ts index 521d297be..059fb1707 100644 --- a/src/execution/generateStep.ts +++ b/src/execution/generateStep.ts @@ -70,6 +70,7 @@ function generateHookContext( sessionId: options.sessionId, messages, request, + emit: runEventsOf(options)?.hookEvent, }; } diff --git a/src/execution/hooks.ts b/src/execution/hooks.ts index e38bd1653..fd62f5eec 100644 --- a/src/execution/hooks.ts +++ b/src/execution/hooks.ts @@ -40,6 +40,7 @@ */ import { GenerateOptions, GenerateResult, Message, ToolCall } from '../providers'; +import type { AgentEventPayload } from './agentEvents'; /** * Fields common to every hook invocation. @@ -115,8 +116,18 @@ export interface ToolCallHookResult { requiresApproval?: boolean; } +/** The stream events a hook may emit with `GenerateHookContext.emit` (LOU-W3.2). */ +export type HookEventPayload = Extract; + /** Context passed to `AgentHook.preGenerate` / `postGenerate`. */ export interface GenerateHookContext extends HookContext { + /** + * Adds an event to the run's stream, inside the current step (LOU-W3.2). + * Set only when the run is streamed (`agent.stream()`, `session.stream()`, + * `AgentExecutor.stream()`); `ctx.emit?.(...)` is a no-op otherwise. The run + * fills in `runId`, `seq`, `timestamp` and `v`, and tags a sub-agent's events. + */ + emit?: (event: HookEventPayload) => void; /** * Live reference to the request about to be sent to * `provider.generate()`. Mutating it (e.g. appending a message, changing