Skip to content

[EPIC-V][LOU-V7.1] Provider retry policy and model fallback wrappers - #74

Merged
LinuxDevil merged 2 commits into
mainfrom
lou-v7-provider-resilience
Oct 1, 2026
Merged

LinuxDevil merged 2 commits into
mainfrom
lou-v7-provider-resilience

Conversation

@LinuxDevil

Copy link
Copy Markdown
Owner

What

New src/providers/resilience.ts, exported from src/providers/index.ts and so from the package root:

  • withRetry(provider, { maxRetries = 2, backoff: { initialMs, maxMs, factor, jitter }, retryOn, onRetry, signal, timeoutMs }) returns an LLMProvider whose generate() and stream() retry transient failures with exponential backoff.
    • Retryability comes from the existing compactProviderError() classification, or .compacted on a CompactedLLMProviderError, through the new exported isRetryableProviderError(). Rate limits, timeouts, network errors and 5xx responses are retried. Auth failures, invalid requests and context-length errors are not.
    • A provider's retryAfterMs replaces the backoff delay.
    • It never retries an abort. The wrapper's signal or the call's signal stops retrying, including during the backoff wait.
    • A stream() call is retried only when it rejects. An error inside a stream that was already returned is not retried.
  • withFallback(providers, { fallbackOn, onFallback }) tries the providers in order and rethrows the last error when all fail.
    • name/defaultModel report the provider that served the latest call. supportsTools/supportsStreaming/getModels ask the first provider.
    • Each fallback runs on its own defaultModel. An explicit model goes only to the first provider.
    • It composes with retry: withFallback([withRetry(a), withRetry(b)]).
  • resilientProvider(provider, config) applies the LLMProviderConfig.maxRetries and timeout fields, which were declared but never read. timeout is a per-attempt limit (timeoutMs).
  • The mock provider's private abortableDelay moves to src/providers/abortableDelay.ts so the retry backoff can reuse it (fallow flagged the copy as a clone). The only change in behavior: it now rejects at once when the signal is already aborted.
  • Docs: a "Provider retries and fallback" section in docs/configuration.md with a type-checked snippet, plus a row in the Providers table in docs/api-overview.md. llms-full.txt is regenerated.

src/execution/retry.ts is not reused, and it is left untouched:

  • Its backoff sleep ignores the abort signal.
  • Its delay hint reads only the legacy RateLimitError, or a fixed 60s for any 429.
  • Its default isRetryableError retries any LLMProviderError without a status code, which includes compacted auth failures.

Reusing it would have meant changing its behavior for existing callers. Its removal is a separate ticket.

Why

LOU-V7 (Epic V, run loop) closes a parity gap with eve's AI Gateway fallbacks and open-harness's withRetry middleware. This is the provider-layer half. The agent-level createAgent({ retry, fallbackModels }) options and the typed retry stream event are LOU-V7.2.

Acceptance criteria

  • withRetry retries generate and stream on retryable errors using the SDK's provider-error classification
  • retryAfterMs honored (fake-timer test)
  • Non-retryable errors are not retried; aborts are never retried; abort during backoff stops retrying
  • withFallback switches providers, reports onFallback, and rethrows the last error when exhausted
  • name/defaultModel reflect the active provider; capabilities delegate to the first
  • Wrappers compose (withFallback([withRetry(a), withRetry(b)]), stream path)
  • LLMProviderConfig.maxRetries/timeout honored via resilientProvider()
  • Exported from src/providers/index.ts (and the root via export * from './providers')
  • Tests in src/providers/resilience.test.ts (17 tests)
  • Docs section with type-checked snippet; llms files regenerated
  • No edits to createAgent.ts, AgentExecutor.ts, agentEvents.ts, resume.ts, execution/retry.ts

How verified

  • npx tsc --noEmit: no errors
  • npm run lint: ✖ 433 problems (0 errors, 433 warnings), the same count as main
  • npm run test:types: Tests 19 passed (19), Type Errors no errors
  • npx vitest run: Test Files 133 passed (133), Tests 1969 passed | 3 skipped (1972)
  • npm run build && npm run build --workspace=packages/create-loushy-agent: both exit 0
  • npm run docs:verify-snippets -- --skip-build: all 85 snippet(s) type-check against src/; 6 also run cleanly
  • npm run docs:llms:check: passes
  • npm run test:coverage: passes (resilience.ts 99.54% lines). npm run fallow: ✓ No issues found, ✓ 0 above threshold. The remaining 30 duplicated lines are the existing OpenAI/Anthropic provider clone.

Note: the built-in providers still run the ai SDK's own internal retries (2 by default) inside each withRetry attempt. This is documented in the new section.

🤖 Generated with Claude Code

https://claude.ai/code/session_016bY29ibQgv3MDdjAbUT4YR


Generated by Claude Code

claude added 2 commits October 1, 2026 11:33
New src/providers/resilience.ts:
- withRetry(provider, options) retries generate()/stream() on transient
  failures with exponential backoff, classifying errors with the existing
  compactProviderError() (rate limit, timeout, network, 5xx retryable; auth,
  invalid request, context length not), honoring retryAfterMs, never
  retrying a cancellation, and stopping when the signal aborts.
- withFallback(providers, options) tries providers in order, reports
  onFallback, rethrows the last error; name/defaultModel follow the active
  provider.
- resilientProvider(provider, config) applies LLMProviderConfig.maxRetries
  and timeout.

The mock provider's abortable delay moves to a shared abortableDelay module
so the retry backoff reuses it. execution/retry.ts is left untouched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016bY29ibQgv3MDdjAbUT4YR
@LinuxDevil
LinuxDevil merged commit 9cc5772 into main Oct 1, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants