Skip to content

[EPIC-W][LOU-W2] Context compaction: prune old tool results behind a pluggable strategy - #80

Merged
LinuxDevil merged 3 commits into
mainfrom
lou-w2-compaction-prune
Oct 1, 2026
Merged

LinuxDevil merged 3 commits into
mainfrom
lou-w2-compaction-prune

Conversation

@LinuxDevil

Copy link
Copy Markdown
Owner

What

New module src/context/compaction.ts (exported from src/context/index.ts and the package root):

  • CompactionStrategy { name; compact(input) }, CompactionInput { messages, estimateTokens, contextWindow, protectedTokens }, CompactionResult { messages, tokensBefore, tokensAfter, prunedToolCallIds }.
  • pruneToolResultsStrategy(): replaces the content of tool results older than the protected tail with [pruned: <toolName> result, N chars]. The protected tail is the newest messages that fit in protectedTokens (default 40,000), plus every message after the latest assistant turn (results the model has not read yet are never pruned, even if they are bigger than the budget). Only tool messages change: system messages, user messages (the first one included) and assistant turns/tool calls are untouched, so every tool call keeps its result. Results that are already a marker, or shorter than one, are skipped, so the strategy is idempotent and returns the input array when nothing changes.
  • compactMessages(messages, options): pure, manual compaction (strategy, contextWindow, protectedTokens, model).
  • createCompactionHook({ strategy?, thresholdPercent? = 0.9, contextWindow?, protectedTokens?, onCompaction? }): an AgentHook named compaction. Its preGenerate estimates the request with estimateTokens (LOU-W1) and compacts when it is above thresholdPercent * contextWindow. contextWindow defaults to getModelInfo(request.model)?.contextWindow, else 128,000. onCompaction({ tokensBefore, tokensAfter, prunedToolCallIds, strategy }) is the integration point for LOU-W3's compaction.* events.

Transcript or copy? request.messages is the run's transcript: AgentExecutor passes state.messages itself to prepareGenerateRequest() (AgentExecutor.ts:745), which puts that array in GenerateOptions.messages and also hands it to hooks as ctx.messages. So the hook rewrites the array in place (splice), and pruning persists in later steps, in checkpoints and in result.messages, like open-harness. No internal set of pruned ids is needed: already-pruned results are recognized by their marker. This is documented in the module comment and in docs/compaction.md.

Docs: new docs/compaction.md (type-checked snippets: hook setup, compactMessages, a custom strategy), linked from docs/api-overview.md (new "Context compaction" subsection) and the README docs list; CHANGELOG [Unreleased] > Added entry; llms.txt/llms-full.txt regenerated.

No execution-loop files were touched (createAgent.ts, AgentExecutor.ts, generateStep.ts, resume.ts, toolCallExecution.ts, agentEvents.ts are unchanged).

Why

Parity gap (Epic W): both eve (compaction.thresholdPercent, default 0.9) and open-harness (DefaultCompactionStrategy, prune to a marker while keeping ~40K recent tokens) compact context; we did not, so long tool-heavy runs could overflow the model's context window.

Acceptance criteria

  • CompactionStrategy / CompactionResult interface and built-in pruneToolResultsStrategy()
  • Old tool results are replaced by a marker; the protected tail (default 40,000 estimated tokens) stays intact and in order
  • System messages, the first user message, and results that are already a marker are never touched
  • Idempotent; produces a valid transcript (every assistant tool call still has its result message)
  • compactMessages(messages, options) pure function for manual use
  • createCompactionHook() returns an AgentHook named compaction; compacts above thresholdPercent * contextWindow (default 0.9); context window from the model registry by request.model, else 128,000; onCompaction callback
  • Reuses estimateTokens and getModelInfo from src/models (LOU-W1)
  • Documented whether request.messages is the transcript (it is; pruned in place and persisted)
  • Tests: tail/order, system/user never pruned, idempotency, valid transcript, threshold math, hook via HookRegistry.runPreGenerate, end-to-end run with mockModel + hooks through AgentExecutor.execute (createAgent has no hooks option)
  • docs/compaction.md linked from api-overview and README; llms files regenerated

How verified

npx tsc --noEmit                                   -> exit 0
npm run lint                                       -> ✖ 433 problems (0 errors, 433 warnings)  (main: 433)
npm run test:types                                 -> Type Errors no errors; Tests 19 passed (19)
npx vitest run                                     -> Test Files 138 passed (138); Tests 2032 passed | 3 skipped (2035)
npm run build && npm run build --workspace=packages/create-loushy-agent  -> both exit 0
npm run docs:verify-snippets -- --skip-build       -> all 94 snippet(s) type-check against src/; 6 also run cleanly
npm run docs:llms:check                            -> exit 0
npm run test:coverage && npm run fallow            -> 138 files passed; fallow exit 0, "✓ 0 above threshold"

New tests: src/context/compaction.test.ts (10 tests).

Not in this PR (LOU-W3): the summarize strategy and typed compaction.* stream events. CompactionStrategy.compact is synchronous as specified; a summarize strategy will need it to allow Promise<CompactionResult>.

🤖 Generated with Claude Code

https://claude.ai/code/session_016bY29ibQgv3MDdjAbUT4YR


Generated by Claude Code

claude added 3 commits October 1, 2026 12:40
…pluggable strategy

Add src/context/compaction.ts (exported from the package root):

- CompactionStrategy / CompactionResult: a pluggable compaction interface.
- pruneToolResultsStrategy(): replaces tool results older than the
  protected tail (newest protectedTokens, default 40,000, plus the latest
  assistant turn's results) with a "[pruned: <tool> result, N chars]"
  marker. System, user and assistant messages are never changed, so the
  transcript stays valid; markers and short results are left alone, so it
  is idempotent.
- compactMessages(): pure, manual compaction.
- createCompactionHook(): an AgentHook that compacts before a model call
  estimated above thresholdPercent (default 0.9) of the context window
  (registry lookup by request.model, else 128,000), reported through
  onCompaction. GenerateHookContext.request.messages is the run's
  state.messages array, so the hook rewrites it in place and pruning
  persists in later steps, checkpoints and result.messages.

Docs: docs/compaction.md, linked from docs/api-overview.md and the README;
llms files regenerated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016bY29ibQgv3MDdjAbUT4YR
Move the per-message pruning decision into prunedToolResult() so the
strategy's map callback stays under fallow's CRAP threshold. No behavior
change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016bY29ibQgv3MDdjAbUT4YR
@LinuxDevil
LinuxDevil merged commit 7126757 into main Oct 1, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants