Repository navigation
[EPIC-V][LOU-V7.1] Provider retry policy and model fallback wrappers - #74
Merged
Merged
Conversation
New src/providers/resilience.ts: - withRetry(provider, options) retries generate()/stream() on transient failures with exponential backoff, classifying errors with the existing compactProviderError() (rate limit, timeout, network, 5xx retryable; auth, invalid request, context length not), honoring retryAfterMs, never retrying a cancellation, and stopping when the signal aborts. - withFallback(providers, options) tries providers in order, reports onFallback, rethrows the last error; name/defaultModel follow the active provider. - resilientProvider(provider, config) applies LLMProviderConfig.maxRetries and timeout. The mock provider's abortable delay moves to a shared abortableDelay module so the retry backoff reuses it. execution/retry.ts is left untouched. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016bY29ibQgv3MDdjAbUT4YR
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
New
src/providers/resilience.ts, exported fromsrc/providers/index.tsand so from the package root:withRetry(provider, { maxRetries = 2, backoff: { initialMs, maxMs, factor, jitter }, retryOn, onRetry, signal, timeoutMs })returns anLLMProviderwhosegenerate()andstream()retry transient failures with exponential backoff.compactProviderError()classification, or.compactedon aCompactedLLMProviderError, through the new exportedisRetryableProviderError(). Rate limits, timeouts, network errors and 5xx responses are retried. Auth failures, invalid requests and context-length errors are not.retryAfterMsreplaces the backoff delay.signalor the call'ssignalstops retrying, including during the backoff wait.stream()call is retried only when it rejects. An error inside a stream that was already returned is not retried.withFallback(providers, { fallbackOn, onFallback })tries the providers in order and rethrows the last error when all fail.name/defaultModelreport the provider that served the latest call.supportsTools/supportsStreaming/getModelsask the first provider.defaultModel. An explicitmodelgoes only to the first provider.withFallback([withRetry(a), withRetry(b)]).resilientProvider(provider, config)applies theLLMProviderConfig.maxRetriesandtimeoutfields, which were declared but never read.timeoutis a per-attempt limit (timeoutMs).abortableDelaymoves tosrc/providers/abortableDelay.tsso the retry backoff can reuse it (fallow flagged the copy as a clone). The only change in behavior: it now rejects at once when the signal is already aborted.docs/configuration.mdwith a type-checked snippet, plus a row in the Providers table indocs/api-overview.md.llms-full.txtis regenerated.src/execution/retry.tsis not reused, and it is left untouched:sleepignores the abort signal.RateLimitError, or a fixed 60s for any 429.isRetryableErrorretries anyLLMProviderErrorwithout a status code, which includes compacted auth failures.Reusing it would have meant changing its behavior for existing callers. Its removal is a separate ticket.
Why
LOU-V7 (Epic V, run loop) closes a parity gap with eve's AI Gateway fallbacks and open-harness's
withRetrymiddleware. This is the provider-layer half. The agent-levelcreateAgent({ retry, fallbackModels })options and the typedretrystream event are LOU-V7.2.Acceptance criteria
withRetryretriesgenerateandstreamon retryable errors using the SDK's provider-error classificationretryAfterMshonored (fake-timer test)withFallbackswitches providers, reportsonFallback, and rethrows the last error when exhaustedname/defaultModelreflect the active provider; capabilities delegate to the firstwithFallback([withRetry(a), withRetry(b)]), stream path)LLMProviderConfig.maxRetries/timeouthonored viaresilientProvider()src/providers/index.ts(and the root viaexport * from './providers')src/providers/resilience.test.ts(17 tests)createAgent.ts,AgentExecutor.ts,agentEvents.ts,resume.ts,execution/retry.tsHow verified
npx tsc --noEmit: no errorsnpm run lint:✖ 433 problems (0 errors, 433 warnings), the same count as mainnpm run test:types:Tests 19 passed (19),Type Errors no errorsnpx vitest run:Test Files 133 passed (133),Tests 1969 passed | 3 skipped (1972)npm run build && npm run build --workspace=packages/create-loushy-agent: both exit 0npm run docs:verify-snippets -- --skip-build:all 85 snippet(s) type-check against src/; 6 also run cleanlynpm run docs:llms:check: passesnpm run test:coverage: passes (resilience.ts99.54% lines).npm run fallow:✓ No issues found,✓ 0 above threshold. The remaining 30 duplicated lines are the existing OpenAI/Anthropic provider clone.Note: the built-in providers still run the
aiSDK's own internal retries (2 by default) inside eachwithRetryattempt. This is documented in the new section.🤖 Generated with Claude Code
https://claude.ai/code/session_016bY29ibQgv3MDdjAbUT4YR
Generated by Claude Code