LOU-V3: run a turn's tool calls in parallel (toolConcurrency)

README.md

@@ -22,6 +22,7 @@ dashboard — any provider, any deploy target, no lock-in.
 - **Human-in-the-loop** — flag a tool `needsApproval` and pause execution until a human approves or rejects it, then `resumeAfterApproval()` from any process
 - **Durable execution** — pass a `sessionId` + `checkpointStore` and a crash mid-conversation resumes instead of restarting
 - **Cancellation** — pass an `AbortSignal` (`agent.send(input, { signal })`) to stop a run; it resolves with `finishReason: 'aborted'` and the transcript so far, and the signal reaches the provider, tools and delegated agents
+- **Parallel tool calls** — when the model asks for several tools in one turn they run concurrently (cap it with `toolConcurrency`, or `1` for sequential), and results still reach the transcript in the model's call order
 - **Skills** — `defineSkill()` / `loadSkills('./skills')`: only each skill's name and description sit in the system prompt; the model loads the full markdown on demand through an auto-registered `load_skill` tool
 - **Multi-agent delegation** — wrap a child agent as a tool with `createDelegateTool()`, with a `maxDepth` guard against delegation loops
 - **Pre/post hooks** — a `HookRegistry` of `AgentHook`s that can inspect or mutate a tool call/LLM generate step, or throw to abort it, run sandboxed by Agent Forge's hook editor

docs/api-overview.md

@@ -70,6 +70,63 @@ How it behaves:
   cancelled, so the conversation stays valid for the provider.
 - An already-aborted signal returns at once without calling the provider.
 
+### Parallel tool calls
+
+When the model asks for several tools in one turn, they run concurrently.
+`toolConcurrency` (on `createAgent()` and `AgentExecutor.execute()`) caps how
+many run at once: a positive integer, or `'unbounded'` (the default). Use `1`
+for strictly sequential execution, e.g. when your tools share state that is
+not safe to touch concurrently.
+
+```ts
+import { createAgent, createMockProvider, defineTool } from '@loushy/build-ai-agent';
+import { z } from 'zod';
+
+const getWeather = defineTool({
+  name: 'get_weather',
+  description: 'Current weather for a city',
+  input: z.object({ city: z.string() }),
+  execute: async ({ city }) => ({ city, tempC: 21 }),
+});
+
+const agent = createAgent({
+  prompt: 'You are a travel assistant.',
+  provider: createMockProvider(),
+  tools: [getWeather],
+  toolConcurrency: 4, // at most 4 tool calls of a turn in flight
+});
+```
+
+Guarantees, whatever the limit:
+
+- **Transcript order is call order.** Tool results are appended in the order
+  the model requested the calls, not the order they finish, so the next
+  provider request is deterministic.
+- **Events.** Calls start in call order. A call's `tool-call` event,
+  `onToolCall`, argument validation, `preToolCall` hooks and `needsApproval`
+  check run just before it starts, one call at a time. Its `tool-result`
+  event fires when it finishes, so results arrive in completion order. With
+  `toolConcurrency: 1` events alternate call/result exactly as before.
+- **Approvals.** The first call that needs approval stops the batch: the
+  calls before it run (concurrently) and their results are recorded, then the
+  run pauses on that call (`finishReason: 'awaiting-approval'`). Calls after
+  it never start in this run. **Known gap:** after `resumeAfterApproval()`
+  those later calls are still not run and get no result (tracked as LOU-U7).
+  Until that lands, use `toolConcurrency: 1` and keep approval tools out of
+  multi-call turns if this matters to you.
+- **Failures are isolated.** A tool that throws gets its own error result;
+  its siblings carry on. A propagating error (`PropagatingToolError`, such as
+  the delegation depth guard, or a throwing hook) stops new calls from
+  starting, waits for the running ones to settle, then rejects the run. No
+  tool is left running detached.
+- **Cancellation.** An abort while a batch runs resolves with
+  `finishReason: 'aborted'`. Calls that finished keep their results; the rest
+  get a "cancelled" result. Running tools see the abort through their
+  `abortSignal`.
+- **Checkpoints.** With `sessionId` + `checkpointStore`, a checkpoint is
+  written each time the in-order run of finished calls grows (with `1`, after
+  every call, as before). A resumed run never runs a recorded call again.
+
 ## Declarative specs
 
 | Export             | Description                                                    |

src/createAgent.ts

@@ -23,6 +23,7 @@ import { ToolRegistry } from './tools/ToolRegistry';
 import { ToolDescriptor } from './types';
 import { modelFromEnv, resolveProviderSpec } from './providers/providerSpec';
 import type { DefinedTool } from './tools/defineTool';
+import { ToolConcurrency, assertToolConcurrency } from './execution/toolBatch';
 import type { Skill } from './skills/defineSkill';
 
 /**
@@ -53,6 +54,18 @@ export interface CreateAgentBase {
   name?: string;
   /** Optional maxSteps passed through to AgentExecutor.execute(). */
   maxSteps?: number;
+  /**
+   * How many tool calls from one model turn may run at once (LOU-V3).
+   * Defaults to `'unbounded'`; `1` runs them one at a time. Results always
+   * reach the transcript in the model's call order. See
+   * `ExecuteOptions.toolConcurrency` for the full contract.
+   *
+   * @example
+   * ```ts
+   * const agent = createAgent({ prompt: '...', provider, tools: [sendEmail], toolConcurrency: 1 });
+   * ```
+   */
+  toolConcurrency?: ToolConcurrency;
 }
 
 /**
@@ -151,6 +164,7 @@ export interface SimpleAgent {
  * const { text } = await agent.send('Hello!');
  */
 export function createAgent(config: CreateAgentConfig = {}): SimpleAgent {
+  assertToolConcurrency(config.toolConcurrency, 'createAgent');
   const instructions = resolveInstructions(config);
   const provider = resolveModelSource(config);
 
@@ -174,6 +188,7 @@ export function createAgent(config: CreateAgentConfig = {}): SimpleAgent {
         toolRegistry,
         skills: config.skills,
         maxSteps: config.maxSteps,
+        toolConcurrency: config.toolConcurrency,
         signal: options.signal,
       });
     },

src/execution/AgentExecutor.ts

@@ -15,7 +15,19 @@ import { CheckpointStore } from './checkpoint';
 import { TraceExporter, withSpan } from './tracing';
 import { HookRegistry } from './hooks';
 import { isAbortError } from './errors';
-import { ToolCallOutcome, parseToolArguments, runToolCall } from './toolCallExecution';
+import {
+  PreparedToolCall,
+  ToolCallOutcome,
+  parseToolArguments,
+  runToolCall,
+} from './toolCallExecution';
+import {
+  StartedToolCall,
+  ToolBatchResult,
+  ToolConcurrency,
+  assertToolConcurrency,
+  runToolBatch,
+} from './toolBatch';
 import {
   buildTools,
   compactGenerateError,
@@ -28,7 +40,8 @@ import {
   AgentRunState,
   addUsage,
   loadRunState,
-  pushCancelledToolResults,
+  pushAbortedBatchResults,
+  pushToolResult,
   saveStepCheckpoint,
   toExecutionResult,
 } from './agentRunState';
@@ -294,6 +307,40 @@ export interface ExecuteOptions {
    * ```
    */
   signal?: AbortSignal;
+  /**
+   * LOU-V3: how many tool calls from ONE model turn may run at the same
+   * time. Defaults to `'unbounded'` (every call of the turn runs
+   * concurrently); `1` restores strictly sequential execution. Must be a
+   * positive integer or `'unbounded'`.
+   *
+   * Guarantees, whatever the limit:
+   * - Tool-result messages are appended to the transcript in the model's
+   *   call order, never completion order, so the next provider request is
+   *   deterministic.
+   * - Calls are started in call order. Each call's `tool-call` event,
+   *   `onToolCall`, argument validation, `preToolCall` hooks and
+   *   `needsApproval` check run before it starts, one call at a time; its
+   *   `tool-result` event fires as soon as it completes (completion order).
+   *   With `1`, events alternate call/result exactly as before.
+   * - Approval: the first call that needs approval stops the batch. Calls
+   *   before it run (concurrently) and are recorded; the run then pauses
+   *   on that call (`finishReason: 'awaiting-approval'`); calls after it
+   *   never start in this run.
+   * - A failing tool does not affect its siblings - each gets its own error
+   *   result. A propagating error (`PropagatingToolError`, a throwing hook)
+   *   stops new calls from starting, waits for the running ones, then
+   *   rejects `execute()` - no tool is left running detached.
+   * - An abort (`signal`) mid-batch resolves with `finishReason: 'aborted'`:
+   *   finished calls keep their results, the rest get a "cancelled" result.
+   * - With checkpointing, results are checkpointed as the in-order prefix
+   *   of finished calls grows, so a resumed run never re-runs a recorded call.
+   *
+   * @example
+   * ```ts
+   * await AgentExecutor.execute({ agent, input: 'Compare 3 cities', provider, toolConcurrency: 2 });
+   * ```
+   */
+  toolConcurrency?: ToolConcurrency;
 }
 
 /**
@@ -542,9 +589,11 @@ export class AgentExecutor {
   }
 
   /**
-   * Executes the tool calls of one assistant turn in order, appending each
-   * result to the conversation and checkpointing after it. Resolves to the
-   * ExecutionResult to return when a tool call requires approval.
+   * Executes the tool calls of one assistant turn - concurrently up to
+   * `toolConcurrency` (LOU-V3, see that option for the ordering contract) -
+   * appending the results to the conversation in call order and
+   * checkpointing as they are recorded. Resolves to the ExecutionResult to
+   * return when the run pauses for approval or is aborted.
    */
   private static async runToolCalls(
     options: ExecuteOptions,
@@ -563,53 +612,71 @@ export class AgentExecutor {
       toolCalls,
     });
 
-    // Execute tools and add results
-    for (const [index, toolCall] of toolCalls.entries()) {
-      if (options.signal?.aborted) {
-        pushCancelledToolResults(state, toolCalls.slice(index));
-        return this.abortRun(options, state);
-      }
-
-      this.emitEvent(onEvent, {
-        type: 'tool-call',
-        timestamp: new Date(),
-        toolCall,
-      });
-
-      const toolResult = await this.runToolCallInSpan(options, state, toolCall, agentSpanId);
-
-      if (toolResult.requiresApproval) {
-        return this.pauseForApproval(options, state, toolCall, toolResult);
-      }
-
-      this.emitEvent(onEvent, {
-        type: 'tool-result',
-        timestamp: new Date(),
-        toolResult,
-      });
+    const batch = await runToolBatch(
+      toolCalls,
+      options.toolConcurrency ?? 'unbounded',
+      {
+        start: (toolCall) => {
+          this.emitEvent(onEvent, { type: 'tool-call', timestamp: new Date(), toolCall });
+          return this.startToolCall(options, state, toolCall, agentSpanId);
+        },
+        onComplete: (toolResult) =>
+          this.emitEvent(onEvent, { type: 'tool-result', timestamp: new Date(), toolResult }),
+        record: (toolCall, outcome) => pushToolResult(state, toolCall, outcome),
+        persist: () => saveStepCheckpoint(options, state),
+      },
+      options.signal
+    );
 
-      // A failed tool carries its message as `{error}` (the same shape
-      // resume.ts uses) - `result` is null then, so the model would
-      // otherwise see a bare "null" and never learn the call failed. A
-      // failure that already has a structured result (argument validation)
-      // keeps it, so the model gets the per-issue detail.
-      const failed = toolResult.error !== undefined;
-      const failurePayload = toolResult.result ?? { error: toolResult.error };
-      state.messages.push({
-        role: 'tool',
-        content: JSON.stringify(failed ? failurePayload : toolResult.result),
-        name: toolCall.function.name,
-        toolCallId: toolCall.id,
-        toolName: toolCall.function.name,
-        ...(failed && { isError: true }),
-      });
+    return this.settleToolBatch(options, state, batch);
+  }
 
-      await saveStepCheckpoint(options, state);
+  /**
+   * Turns a finished batch into the run's next move: abort (finished calls
+   * keep their results, the rest are cancelled), reject with the first
+   * fatal error, pause for approval, or carry on (`undefined`).
+   */
+  private static settleToolBatch(
+    options: ExecuteOptions,
+    state: AgentRunState,
+    batch: ToolBatchResult
+  ): Promise<ExecutionResult> | undefined {
+    if (options.signal?.aborted) {
+      pushAbortedBatchResults(state, batch.unrecorded);
+      return this.abortRun(options, state);
+    }
+    if (batch.failure) {
+      throw batch.failure.error;
+    }
+    if (batch.approval) {
+      return this.pauseForApproval(options, state, batch.approval.toolCall, batch.approval.outcome);
     }
-
     return undefined;
   }
 
+  /**
+   * Starts one tool call. `gate` settles once it passed its pre-execution
+   * checks (or rejects with the error that stopped it); `done` settles with
+   * its outcome.
+   */
+  private static startToolCall(
+    options: ExecuteOptions,
+    state: AgentRunState,
+    toolCall: ToolCall,
+    agentSpanId: string
+  ): StartedToolCall {
+    let openGate!: (prepared: PreparedToolCall) => void;
+    let failGate!: (error: unknown) => void;
+    const gate = new Promise<PreparedToolCall>((resolve, reject) => {
+      openGate = resolve;
+      failGate = reject;
+    });
+    const done = this.runToolCallInSpan(options, state, toolCall, agentSpanId, openGate);
+    // Rejecting an already-resolved gate is a no-op.
+    done.catch(failGate);
+    return { gate, done };
+  }
+
   /**
    * Runs one tool call inside a `tool.call` span parented to the run's
    * `agent.run` span.
@@ -618,7 +685,8 @@ export class AgentExecutor {
     options: ExecuteOptions,
     state: AgentRunState,
     toolCall: ToolCall,
-    agentSpanId: string
+    agentSpanId: string,
+    onPrepared?: (prepared: PreparedToolCall) => void
   ): Promise<ToolCallOutcome> {
     const {
       agent,
@@ -649,7 +717,8 @@ export class AgentExecutor {
           hooks,
           sessionId,
           state.messages,
-          signal
+          signal,
+          onPrepared
         );
         const parsedArgs =
           executed.args === undefined
@@ -807,19 +876,24 @@ export class AgentExecutor {
     hooks?: HookRegistry,
     sessionId?: string,
     messages: Message[] = [],
-    signal?: AbortSignal
+    signal?: AbortSignal,
+    onPrepared?: (prepared: PreparedToolCall) => void
   ): Promise<ToolCallOutcome> {
-    return runToolCall(toolCall, {
-      agent,
-      toolRegistry,
-      onToolCall,
-      onToolResult,
-      sandbox,
-      hooks,
-      sessionId,
-      messages,
-      signal,
-    });
+    return runToolCall(
+      toolCall,
+      {
+        agent,
+        toolRegistry,
+        onToolCall,
+        onToolResult,
+        sandbox,
+        hooks,
+        sessionId,
+        messages,
+        signal,
+      },
+      onPrepared
+    );
   }
 
   /**
@@ -849,6 +923,7 @@ export class AgentExecutor {
           "Example: AgentExecutor.execute({ agent, input: 'hello', provider })"
       );
     }
+    assertToolConcurrency(options.toolConcurrency, 'AgentExecutor.execute');
   }
 
   /**