Repository navigation
LOU-V3: run a turn's tool calls in parallel (toolConcurrency) - #49
Merged
Merged
Conversation
When the model returned several tool calls in one turn, AgentExecutor ran them one after another. They now run concurrently, capped by a new `toolConcurrency?: number | 'unbounded'` option on AgentExecutor.execute() and createAgent() (default 'unbounded'; 1 restores sequential execution; anything else throws a descriptive error). Contract (TSDoc on ExecuteOptions.toolConcurrency, docs/api-overview.md): - Tool results are appended in the model's call order, not completion order, so the next provider request is deterministic. - Calls start in call order; each call's tool-call event, onToolCall, validation, preToolCall hooks and needsApproval check run just before it starts, one call at a time. tool-result events fire on completion. With toolConcurrency 1 the event order is exactly the old interleaved order. - Approval: the first call needing approval halts the batch. Earlier calls run (concurrently) and are recorded, then the run pauses on that call as before. Later calls never start (resume behavior for them is unchanged; LOU-U7). - A failing tool gets its own error result; siblings are unaffected. A propagating error stops new calls from starting, awaits running ones, then rejects. - An abort mid-batch keeps finished results and cancels the rest. - Checkpoints are written whenever the in-order prefix of finished calls grows (per call with concurrency 1, as before), so a recorded call is never re-run on resume. Implementation: runToolCall() is split into a gate (prepareToolCall) and execution (settleToolCall); the new toolBatch.ts schedules a batch with a small limiter. Tool-result message building moved to agentRunState. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…calls # Conflicts: # README.md # src/createAgent.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The tool calls of one model turn now run concurrently. A new option caps how many run at once:
Invalid values (
0,-1,1.5,NaN,'all') throw:AgentExecutor.execute: 'toolConcurrency' must be a positive integer or 'unbounded', got 0. Use 1 to run a turn's tool calls one at a time, ...(same forcreateAgent).ToolConcurrencyis exported as a type.Why
Before this change the executor ran parallel tool calls from the model one after another. Models emit several calls in one turn so that they can run concurrently.
How
runToolCall()(toolCallExecution.ts) is split into a gate (onToolCall, LOU-U4 validation, preToolCall hooks,needsApproval) and the execution (execute, post hook, onToolResult). Behavior for one call is unchanged.src/execution/toolBatch.tsschedules a batch. It starts calls in call order, and each call starts only after the previous one has passed its gate and a slot under the limit is free. Results are recorded in call order as the completed prefix grows.agentRunState.pushToolResult().Contract (TSDoc on
ExecuteOptions.toolConcurrency+ new "Parallel tool calls" section in docs/api-overview.md)tool-callevents (and the gate) run in call order, just before each call starts.tool-resultevents arrive in completion order. WithtoolConcurrency: 1the old interleaved order is kept exactly.finishReason: 'aborted'. Finished calls keep their results and the rest get "cancelled".1that is per call, as before. With concurrency, a result is only persisted once every call before it is persisted, so a resumed run never re-runs a recorded call.How verified
Ran after merging origin/main @ 89d199f:
npx tsc --noEmit: exit 0npm run lint:✖ 426 problems (0 errors, 426 warnings). All warnings were already there; none added in the touched files.npm run build && npm run build --workspace=packages/create-loushy-agent: exit 0npm run test:coverage:Test Files 99 passed (99)/Tests 1294 passed | 3 skipped (1297)npm run fallow: exit 0,✓ 0 above threshold · 2298 analyzed · maintainability 92.2 (good)npm run docs:verify-snippets:all 34 snippet(s) type-check against src/; 5 also run cleanlyNew tests in
src/execution/parallelToolCalls.test.ts(mockModel + defineTool + deferred promises) cover:1is sequential with interleaved eventsNo existing test was changed.
Notes for reviewer
resumeAfterApproval(), calls that came after the approval call in the same batch still never run and get no tool result. This was already the case before this PR. The approval tests assert it explicitly with a comment. Fixing it needsExecutionSnapshotto carry the remaining calls and resume to run them, possibly with another approval pause.needsApprovalnow runs in the gate, before execution starts. If it throws, the result is the same{error}outcome as before. The only difference is that its time is no longer counted inonToolResultlatency.src/skills/*,docs/skills.md, theskillsoption) from main. This branch follows main, so it doesn't have skills either.🤖 Generated with Claude Code