LOU-V5: usage and cost accounting on every run

README.md

@@ -30,6 +30,7 @@ dashboard — any provider, any deploy target, no lock-in.
 - **Project instructions** — `createAgent({ projectInstructions: true })` appends the nearest `AGENTS.md` / `CLAUDE.md` to the instructions (opt-in)
 >>>>>>> origin/main
 - **Parallel tool calls** — when the model asks for several tools in one turn they run concurrently (cap it with `toolConcurrency`, or `1` for sequential), and results still reach the transcript in the model's call order
+- **Usage & cost** — every result carries `usage` (tokens, USD cost, per-model breakdown, delegated children included, continuing across resumes) and `formatUsage(result.usage)` for logs
 - **Skills** — `defineSkill()` / `loadSkills('./skills')`: only each skill's name and description sit in the system prompt; the model loads the full markdown on demand through an auto-registered `load_skill` tool
 - **Agents as directories** — `loadAgentDir('./my-agent')` turns `instructions.md`, `tools/*.ts`, `skills/` and `subagents/<name>/` into the same object `createAgent()` returns, so you can start with files and drop to code later — see [Agent directories](docs/agent-directories.md)
 - **Multi-agent delegation** — wrap a child agent as a tool with `createDelegateTool()`, with a `maxDepth` guard against delegation loops

docs/api-overview.md

@@ -333,6 +333,45 @@ const usd = estimateCost({ inputTokens: used, outputTokens: 500 }, 'my-llama');
 
 The built-in context windows and prices are a dated snapshot (see the retrieval date and sources at the top of `src/models/modelData.ts`). Providers change prices and models, so override entries with `registerModel` when you need billing-grade numbers.
 
+### Usage and cost of a run
+
+Every `ExecutionResult` (and `agent.send()` result) carries `usage`, the running total of the whole run:
+
+```ts
+import { AgentExecutor, formatUsage } from '@loushy/build-ai-agent';
+
+const result = await AgentExecutor.execute({ agent, input: 'Compare 3 cities', provider });
+
+result.usage.inputTokens; // all model calls of the run, delegated children included
+result.usage.costUsd; // number, or undefined if any model used has no known price
+result.usage.estimated; // true if some call reported no usage and was estimated
+result.usage.byModel['gpt-4o-mini']; // { inputTokens, outputTokens, costUsd?, calls }
+result.stepUsage?.[0]; // { step, model, usage, estimated, costUsd? } per model call
+console.log(formatUsage(result.usage)); // 1,234 in / 567 out tokens · $0.0042 (2 model calls)
+```
+
+- **Reported vs estimated.** Providers report `promptTokens`/`completionTokens`; the built-in providers also pass on `cachedInputTokens`/`reasoningTokens` when the 'ai' SDK's provider metadata carries them (`usage.cachedInputTokens` and `usage.reasoningTokens` only appear then). A backend that reports nothing gives `undefined` usage, never zeros. For that step the executor falls back to `estimateTokens` and sets `usage.estimated` (a leading `~` in `formatUsage`). A custom provider should leave `GenerateResult.usage` unset rather than fill in zeros.
+- **Cost.** `costUsd` is the sum of `estimateCost` per model. It is `undefined`, never a misleading partial sum, as soon as any model used has unknown pricing; `byModel` shows which ones are priced.
+- **Delegation.** A delegated child's usage is added to the parent's totals and `byModel`, and is also shown on its own as `usage.delegated` (`{ inputTokens, outputTokens, totalTokens, costUsd, modelCalls, estimated, runs }`).
+- **Resume.** A run resumed from a checkpoint, or after an approval, continues from the saved totals instead of restarting at zero. Checkpoints written by older versions start from their saved token counts with an unknown cost.
+- **Events and traces.** The `finish` event (and every lifecycle event that carried `usage`) now carries the running totals; `text-complete` also has `stepUsage`, and `onLLMResponse` receives the call's usage as a third argument. The `chat` span's `gen_ai.usage.*` attributes use the same numbers, with `loushy.usage.estimated` set to `true` when they are estimates.
+- **Streaming.** `agent.stream()` events carry the same accounting: `step.done` and `run.done` have `usage` with `inputTokens`, `outputTokens`, `estimated` and `costUsd` (run-level also `modelCalls`), alongside the older `promptTokens`/`completionTokens`.
+- `promptTokens` and `completionTokens` on `usage` remain as deprecated aliases of `inputTokens` and `outputTokens`.
+
+Prices come from the model registry above, so to get a cost for a custom or fine-tuned model, register it under the id you pass as the model:
+
+```ts
+import { registerModel } from '@loushy/build-ai-agent';
+
+registerModel({
+  id: 'ft:gpt-4o-mini:acme',
+  provider: 'openai',
+  contextWindow: 128000,
+  inputCostPerMTok: 0.3,
+  outputCostPerMTok: 1.2,
+});
+```
+
 ## Flows, evals, observability and security
 
 - `FlowBuilder` / `FlowExecutor` - multi-step workflow graphs.

llms-full.txt

@@ -28,6 +28,7 @@ dashboard — any provider, any deploy target, no lock-in.
 - **Project instructions** — `createAgent({ projectInstructions: true })` appends the nearest `AGENTS.md` / `CLAUDE.md` to the instructions (opt-in)
 >>>>>>> origin/main
 - **Parallel tool calls** — when the model asks for several tools in one turn they run concurrently (cap it with `toolConcurrency`, or `1` for sequential), and results still reach the transcript in the model's call order
+- **Usage & cost** — every result carries `usage` (tokens, USD cost, per-model breakdown, delegated children included, continuing across resumes) and `formatUsage(result.usage)` for logs
 - **Skills** — `defineSkill()` / `loadSkills('./skills')`: only each skill's name and description sit in the system prompt; the model loads the full markdown on demand through an auto-registered `load_skill` tool
 - **Agents as directories** — `loadAgentDir('./my-agent')` turns `instructions.md`, `tools/*.ts`, `skills/` and `subagents/<name>/` into the same object `createAgent()` returns, so you can start with files and drop to code later — see [Agent directories](https://github.com/LinuxDevil/agent-sdk/blob/main/docs/agent-directories.md)
 - **Multi-agent delegation** — wrap a child agent as a tool with `createDelegateTool()`, with a `maxDepth` guard against delegation loops
@@ -1481,6 +1482,45 @@ const usd = estimateCost({ inputTokens: used, outputTokens: 500 }, 'my-llama');
 
 The built-in context windows and prices are a dated snapshot (see the retrieval date and sources at the top of `src/models/modelData.ts`). Providers change prices and models, so override entries with `registerModel` when you need billing-grade numbers.
 
+### Usage and cost of a run
+
+Every `ExecutionResult` (and `agent.send()` result) carries `usage`, the running total of the whole run:
+
+```ts
+import { AgentExecutor, formatUsage } from '@loushy/build-ai-agent';
+
+const result = await AgentExecutor.execute({ agent, input: 'Compare 3 cities', provider });
+
+result.usage.inputTokens; // all model calls of the run, delegated children included
+result.usage.costUsd; // number, or undefined if any model used has no known price
+result.usage.estimated; // true if some call reported no usage and was estimated
+result.usage.byModel['gpt-4o-mini']; // { inputTokens, outputTokens, costUsd?, calls }
+result.stepUsage?.[0]; // { step, model, usage, estimated, costUsd? } per model call
+console.log(formatUsage(result.usage)); // 1,234 in / 567 out tokens · $0.0042 (2 model calls)
+```
+
+- **Reported vs estimated.** Providers report `promptTokens`/`completionTokens`; the built-in providers also pass on `cachedInputTokens`/`reasoningTokens` when the 'ai' SDK's provider metadata carries them (`usage.cachedInputTokens` and `usage.reasoningTokens` only appear then). A backend that reports nothing gives `undefined` usage, never zeros. For that step the executor falls back to `estimateTokens` and sets `usage.estimated` (a leading `~` in `formatUsage`). A custom provider should leave `GenerateResult.usage` unset rather than fill in zeros.
+- **Cost.** `costUsd` is the sum of `estimateCost` per model. It is `undefined`, never a misleading partial sum, as soon as any model used has unknown pricing; `byModel` shows which ones are priced.
+- **Delegation.** A delegated child's usage is added to the parent's totals and `byModel`, and is also shown on its own as `usage.delegated` (`{ inputTokens, outputTokens, totalTokens, costUsd, modelCalls, estimated, runs }`).
+- **Resume.** A run resumed from a checkpoint, or after an approval, continues from the saved totals instead of restarting at zero. Checkpoints written by older versions start from their saved token counts with an unknown cost.
+- **Events and traces.** The `finish` event (and every lifecycle event that carried `usage`) now carries the running totals; `text-complete` also has `stepUsage`, and `onLLMResponse` receives the call's usage as a third argument. The `chat` span's `gen_ai.usage.*` attributes use the same numbers, with `loushy.usage.estimated` set to `true` when they are estimates.
+- **Streaming.** `agent.stream()` events carry the same accounting: `step.done` and `run.done` have `usage` with `inputTokens`, `outputTokens`, `estimated` and `costUsd` (run-level also `modelCalls`), alongside the older `promptTokens`/`completionTokens`.
+- `promptTokens` and `completionTokens` on `usage` remain as deprecated aliases of `inputTokens` and `outputTokens`.
+
+Prices come from the model registry above, so to get a cost for a custom or fine-tuned model, register it under the id you pass as the model:
+
+```ts
+import { registerModel } from '@loushy/build-ai-agent';
+
+registerModel({
+  id: 'ft:gpt-4o-mini:acme',
+  provider: 'openai',
+  contextWindow: 128000,
+  inputCostPerMTok: 0.3,
+  outputCostPerMTok: 1.2,
+});
+```
+
 ## Flows, evals, observability and security
 
 - `FlowBuilder` / `FlowExecutor` - multi-step workflow graphs.

src/execution/AgentExecutor.ts

@@ -12,6 +12,8 @@ import { ToolRegistry } from '../tools';
 import { SandboxAdapter, NoopSandbox } from '../security/sandboxCore';
 import { ApprovalStore, ExecutionSnapshot, PendingApproval } from './ApprovalGate';
 import { CheckpointStore } from './checkpoint';
+import type { CallUsage, RunUsage, StepUsage } from '../models/usage';
+import { mergeDelegatedUsage } from './runUsage';
 import { TraceExporter, withSpan } from './tracing';
 import {
   agentRunSpanInit,
@@ -23,6 +25,7 @@ import { HookRegistry } from './hooks';
 import { isAbortError } from './errors';
 import {
   PreparedToolCall,
+  ToolCallContext,
   ToolCallOutcome,
   parseToolArguments,
   runToolCall,
@@ -37,17 +40,18 @@ import {
 import {
   buildTools,
   compactGenerateError,
+  GeneratedStep,
   generateInSpan,
   prepareGenerateRequest,
   providerErrorMessage,
   shouldSurfaceToModel,
 } from './generateStep';
 import {
   AgentRunState,
-  addUsage,
   loadRunState,
   pushAbortedBatchResults,
   pushToolResult,
+  recordStep,
   saveStepCheckpoint,
   toExecutionResult,
 } from './agentRunState';
@@ -103,11 +107,10 @@ export interface ExecutionEvent {
     error?: string;
   };
   finishReason?: ExecutionFinishReason;
-  usage?: {
-    promptTokens: number;
-    completionTokens: number;
-    totalTokens: number;
-  };
+  /** Running usage of the whole run so far (LOU-V5); on `finish`, the final total. */
+  usage?: RunUsage;
+  /** On `text-complete`: what the model call that produced the text spent (LOU-V5). */
+  stepUsage?: StepUsage;
   error?: Error;
   /**
    * On an `abort` event: the `reason` of the aborted signal (a
@@ -164,6 +167,12 @@ export interface ExecuteOptions {
    * 0 (a genuinely fresh run) when omitted.
    */
   initialSteps?: number;
+  /**
+   * Usage already spent by earlier work this run continues (LOU-V5), used
+   * when there is no checkpoint to rehydrate it from - e.g. resume.ts
+   * resuming a run paused for approval. Ignored when a checkpoint is loaded.
+   */
+  initialUsage?: RunUsage;
   /**
    * Tracing/observability hooks (LOU-E1/E2). These are invoked immediately
    * before/after the underlying provider.generate() call and each tool
@@ -176,9 +185,14 @@ export interface ExecuteOptions {
   onLLMRequest?: (request: GenerateOptions) => void | Promise<void>;
   /**
    * Invoked immediately after each provider.generate() call resolves,
-   * with the elapsed wall-clock time in milliseconds.
+   * with the elapsed wall-clock time in milliseconds and what the call
+   * spent (LOU-V5: reported usage, or an estimate flagged `estimated`).
    */
-  onLLMResponse?: (response: GenerateResult, latencyMs: number) => void | Promise<void>;
+  onLLMResponse?: (
+    response: GenerateResult,
+    latencyMs: number,
+    usage: CallUsage
+  ) => void | Promise<void>;
   /** Invoked immediately before each tool execution. */
   onToolCall?: (toolCall: ToolCall) => void | Promise<void>;
   /**
@@ -368,11 +382,13 @@ export interface ExecutionResult {
   text: string;
   messages: Message[];
   toolCalls: ToolCall[];
-  usage: {
-    promptTokens: number;
-    completionTokens: number;
-    totalTokens: number;
-  };
+  /**
+   * Tokens, cost and per-model breakdown of the whole run, including
+   * delegated children and steps before a resume (LOU-V5). See `formatUsage()`.
+   */
+  usage: RunUsage;
+  /** One entry per model call of this process's run, in order (LOU-V5). */
+  stepUsage?: StepUsage[];
   finishReason: ExecutionFinishReason;
   steps: number;
   approvalId?: string;
@@ -470,7 +486,7 @@ export class AgentExecutor {
     options: ExecuteOptions,
     agentSpanId: string
   ): Promise<ExecutionResult> {
-    const { agent, toolRegistry, maxSteps = 10, onEvent, signal } = options;
+    const { agent, toolRegistry, onEvent } = options;
 
     // Emit start event
     this.emitEvent(onEvent, {
@@ -485,6 +501,18 @@ export class AgentExecutor {
 
     const state = await loadRunState(options);
 
+    return this.runSteps(options, state, tools, agentSpanId);
+  }
+
+  /** The generate -> tools loop of runAgentLoop(), run once `state` is loaded. */
+  private static async runSteps(
+    options: ExecuteOptions,
+    state: AgentRunState,
+    tools: ToolDefinition[],
+    agentSpanId: string
+  ): Promise<ExecutionResult> {
+    const { maxSteps = 10, signal } = options;
+
     // Execution loop with tool calling. LOU-V1: the signal is checked
     // before every model call (here) and every tool call (runToolCalls()).
     while (state.steps < maxSteps) {
@@ -550,17 +578,18 @@ export class AgentExecutor {
     tools: ToolDefinition[],
     agentSpanId: string
   ): Promise<'continue' | 'stop' | ExecutionResult> {
-    const result = await this.generateOrSurfaceError(options, state, tools, agentSpanId);
-    if (!result) {
+    const generatedStep = await this.generateOrSurfaceError(options, state, tools, agentSpanId);
+    if (!generatedStep) {
       return 'continue';
     }
+    const { generated: result, measured } = generatedStep;
 
     // A turn produced a real result - any pending "the last thing that
     // happened was a provider failure" tracking no longer applies.
     state.lastSurfacedProviderError = undefined;
 
-    // Update usage
-    addUsage(state.usage, result.usage);
+    // Update usage (LOU-V5)
+    const stepUsage = recordStep(state, measured);
 
     // Handle text response
     if (result.text) {
@@ -569,6 +598,7 @@ export class AgentExecutor {
         type: 'text-complete',
         timestamp: new Date(),
         text: result.text,
+        stepUsage,
       });
     }
 
@@ -613,7 +643,7 @@ export class AgentExecutor {
     state: AgentRunState,
     tools: ToolDefinition[],
     agentSpanId: string
-  ): Promise<GenerateResult | undefined> {
+  ): Promise<GeneratedStep | undefined> {
     const generateRequest = await prepareGenerateRequest(options, state.messages, tools);
 
     try {
@@ -787,7 +817,9 @@ export class AgentExecutor {
           sessionId,
           state.messages,
           signal,
-          onPrepared
+          onPrepared,
+          // LOU-V5: a delegated child's usage is added to this run's totals.
+          (child) => mergeDelegatedUsage(state.usage, child)
         );
         const parsedArgs =
           executed.args === undefined
@@ -842,6 +874,7 @@ export class AgentExecutor {
       pendingToolCall: pending,
       steps: state.steps,
       sessionId,
+      usage: structuredClone(state.usage),
     };
 
     await approvalStore.save(pending, snapshot);
@@ -952,7 +985,8 @@ export class AgentExecutor {
     sessionId?: string,
     messages: Message[] = [],
     signal?: AbortSignal,
-    onPrepared?: (prepared: PreparedToolCall) => void
+    onPrepared?: (prepared: PreparedToolCall) => void,
+    onDelegatedUsage?: ToolCallContext['onDelegatedUsage']
   ): Promise<ToolCallOutcome> {
     return runToolCall(
       toolCall,
@@ -966,6 +1000,7 @@ export class AgentExecutor {
         sessionId,
         messages,
         signal,
+        onDelegatedUsage,
       },
       onPrepared
     );

src/execution/ApprovalGate.ts

@@ -6,6 +6,7 @@
 import { Message } from '../providers';
 import { AgentConfig } from '../types';
 import { StorageService } from '../storage';
+import type { RunUsage } from '../models/usage';
 
 /**
  * A tool call that is waiting on a human decision before it can execute.
@@ -48,6 +49,8 @@ export interface ExecutionSnapshot {
    * execution at all.
    */
   sessionId?: string;
+  /** Usage the paused run had spent (LOU-V5), so the resumed run continues its totals. */
+  usage?: RunUsage;
 }
 
 /**

src/execution/DelegationTool.ts

@@ -11,6 +11,7 @@ import { AgentExecutor, PropagatingToolError } from './AgentExecutor';
 import { LLMProvider, Message } from '../providers';
 import { AgentConfig, ToolDescriptor } from '../types';
 import { ToolRegistry } from '../tools';
+import type { ToolRunContext } from './sandboxGuard';
 
 /**
  * Thrown when a delegation chain exceeds the configured maxDepth without
@@ -119,7 +120,7 @@ export function createDelegateTool(opts: DelegateAgentOptions): ToolDescriptor {
           task: string;
           context?: Message[];
         },
-        options?: { abortSignal?: AbortSignal }
+        options?: { abortSignal?: AbortSignal } & ToolRunContext
       ): Promise<DelegateAgentResult> => {
         const currentDepth = delegationDepthStorage.getStore() ?? 0;
 
@@ -143,9 +144,13 @@ export function createDelegateTool(opts: DelegateAgentOptions): ToolDescriptor {
             signal: options?.abortSignal,
           });
 
+          // LOU-V5: the child's full usage rolls up into the parent run's
+          // totals (`usage.delegated`); the model only sees the token counts.
+          options?.onDelegatedUsage?.(result.usage);
+          const { promptTokens, completionTokens, totalTokens } = result.usage;
           return {
             text: result.text,
-            usage: result.usage,
+            usage: { promptTokens, completionTokens, totalTokens },
           };
         });
       },

src/execution/agentEvents.ts

@@ -18,9 +18,19 @@ export const AGENT_EVENT_SCHEMA_VERSION = 1 as const;
 
 /** Token usage, as reported on `step.done` and `run.done`. */
 export interface AgentEventUsage {
+  /** Alias of `inputTokens`. */
   promptTokens: number;
+  /** Alias of `outputTokens`. */
   completionTokens: number;
   totalTokens: number;
+  inputTokens: number;
+  outputTokens: number;
+  /** `true` when the provider reported no usage and the tokens are an `estimateTokens` estimate. */
+  estimated: boolean;
+  /** USD, or absent when a model used has unknown pricing (see `registerModel`). */
+  costUsd?: number;
+  /** On `run.done`: model calls of the whole run. */
+  modelCalls?: number;
 }
 
 /** A JSON-safe error: the `name` and `message` of the original error. */