Repository navigation
[EPIC-W][LOU-W2] Context compaction: prune old tool results behind a pluggable strategy - #80
Merged
Merged
Conversation
…pluggable strategy Add src/context/compaction.ts (exported from the package root): - CompactionStrategy / CompactionResult: a pluggable compaction interface. - pruneToolResultsStrategy(): replaces tool results older than the protected tail (newest protectedTokens, default 40,000, plus the latest assistant turn's results) with a "[pruned: <tool> result, N chars]" marker. System, user and assistant messages are never changed, so the transcript stays valid; markers and short results are left alone, so it is idempotent. - compactMessages(): pure, manual compaction. - createCompactionHook(): an AgentHook that compacts before a model call estimated above thresholdPercent (default 0.9) of the context window (registry lookup by request.model, else 128,000), reported through onCompaction. GenerateHookContext.request.messages is the run's state.messages array, so the hook rewrites it in place and pruning persists in later steps, checkpoints and result.messages. Docs: docs/compaction.md, linked from docs/api-overview.md and the README; llms files regenerated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016bY29ibQgv3MDdjAbUT4YR
Move the per-message pruning decision into prunedToolResult() so the strategy's map callback stays under fallow's CRAP threshold. No behavior change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016bY29ibQgv3MDdjAbUT4YR
7 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
New module
src/context/compaction.ts(exported fromsrc/context/index.tsand the package root):CompactionStrategy { name; compact(input) },CompactionInput { messages, estimateTokens, contextWindow, protectedTokens },CompactionResult { messages, tokensBefore, tokensAfter, prunedToolCallIds }.pruneToolResultsStrategy(): replaces thecontentof tool results older than the protected tail with[pruned: <toolName> result, N chars]. The protected tail is the newest messages that fit inprotectedTokens(default 40,000), plus every message after the latest assistant turn (results the model has not read yet are never pruned, even if they are bigger than the budget). Onlytoolmessages change: system messages, user messages (the first one included) and assistant turns/tool calls are untouched, so every tool call keeps its result. Results that are already a marker, or shorter than one, are skipped, so the strategy is idempotent and returns the input array when nothing changes.compactMessages(messages, options): pure, manual compaction (strategy, contextWindow, protectedTokens, model).createCompactionHook({ strategy?, thresholdPercent? = 0.9, contextWindow?, protectedTokens?, onCompaction? }): anAgentHooknamedcompaction. ItspreGenerateestimates the request withestimateTokens(LOU-W1) and compacts when it is abovethresholdPercent * contextWindow.contextWindowdefaults togetModelInfo(request.model)?.contextWindow, else 128,000.onCompaction({ tokensBefore, tokensAfter, prunedToolCallIds, strategy })is the integration point for LOU-W3'scompaction.*events.Transcript or copy?
request.messagesis the run's transcript:AgentExecutorpassesstate.messagesitself toprepareGenerateRequest()(AgentExecutor.ts:745), which puts that array inGenerateOptions.messagesand also hands it to hooks asctx.messages. So the hook rewrites the array in place (splice), and pruning persists in later steps, in checkpoints and inresult.messages, like open-harness. No internal set of pruned ids is needed: already-pruned results are recognized by their marker. This is documented in the module comment and in docs/compaction.md.Docs: new
docs/compaction.md(type-checked snippets: hook setup,compactMessages, a custom strategy), linked fromdocs/api-overview.md(new "Context compaction" subsection) and the README docs list; CHANGELOG[Unreleased] > Addedentry;llms.txt/llms-full.txtregenerated.No execution-loop files were touched (
createAgent.ts,AgentExecutor.ts,generateStep.ts,resume.ts,toolCallExecution.ts,agentEvents.tsare unchanged).Why
Parity gap (Epic W): both eve (
compaction.thresholdPercent, default 0.9) and open-harness (DefaultCompactionStrategy, prune to a marker while keeping ~40K recent tokens) compact context; we did not, so long tool-heavy runs could overflow the model's context window.Acceptance criteria
CompactionStrategy/CompactionResultinterface and built-inpruneToolResultsStrategy()compactMessages(messages, options)pure function for manual usecreateCompactionHook()returns anAgentHooknamedcompaction; compacts abovethresholdPercent * contextWindow(default 0.9); context window from the model registry byrequest.model, else 128,000;onCompactioncallbackestimateTokensandgetModelInfofrom src/models (LOU-W1)request.messagesis the transcript (it is; pruned in place and persisted)HookRegistry.runPreGenerate, end-to-end run withmockModel+hooksthroughAgentExecutor.execute(createAgenthas nohooksoption)How verified
New tests:
src/context/compaction.test.ts(10 tests).Not in this PR (LOU-W3): the summarize strategy and typed
compaction.*stream events.CompactionStrategy.compactis synchronous as specified; a summarize strategy will need it to allowPromise<CompactionResult>.🤖 Generated with Claude Code
https://claude.ai/code/session_016bY29ibQgv3MDdjAbUT4YR
Generated by Claude Code