fix(openai): retry a transient 429 instead of ending the run - #358
Open
JumpLink wants to merge 1 commit into
Open
fix(openai): retry a transient 429 instead of ending the run#358JumpLink wants to merge 1 commit into
JumpLink wants to merge 1 commit into
Conversation
JumpLink
force-pushed
the
fix/retry-transient-429
branch
from
July 31, 2026 12:29
e5df2e2 to
f25ccff
Compare
JumpLink
force-pushed
the
fix/retry-transient-429
branch
from
August 14, 2026 09:49
f25ccff to
217af02
Compare
LangChain refuses to retry any 429 whose text reads like a spent allowance:
`classifyRateLimitError` matches /insufficient[_ -]?quota/i, returns `stop`, and
its default failed-attempt handler throws before p-retry gets a second attempt.
For OpenAI that is right - `insufficient_quota` means the account is out of
credit.
Scaleway answers all three of its rate limits with that wording and nothing else
to go on:
HTTP/2 429
{"status":429,"error":"INSUFFICIENT QUOTA",
"message":"You exceeded your current limit of concurrent requests."}
The tokens-per-minute and requests-per-minute limits differ only in `message`,
which is exactly what the OpenAI SDK drops - `APIError` keeps `body.error`
alone - so the classifier only ever sees `429 "INSUFFICIENT QUOTA"` and reads a
transient limit as a billing wall. There are no rate-limit or retry-after
headers on the 429 either, although successful responses carry the former.
Measured against api.scaleway.ai on 2026-07-30 through ChatOpenAI itself, with
the new probe script:
150 concurrent tiny requests, stock policy: 123/150 in 2.5s
150 concurrent tiny requests, this handler: 150/150 in 3.3s
4 x 30k-token requests, stock policy: 2/4 in 4.0s
4 x 30k-token requests, this handler: 4/4 in 58.3s
The 58s is the token bucket refilling, and it is the point: a slow turn beats a
dead one. A rejected request is not billed, so the retries cost nothing beyond
the wait.
Terminal cases stay terminal - a structured `insufficient_quota`, wording that
names billing or credit, the never-retry statuses, cancelled requests, and a
single request too large for the per-minute allowance, which is handed to
overflow recovery instead. A caller-supplied `onFailedAttempt` still wins.
Applied to ChatOpenAI (so also Moonshot and OpenRouter), AzureChatOpenAI,
ChatDeepSeek and ChatXAI.
Signed-off-by: JumpLink <pascal@artandcode.studio>
JumpLink
force-pushed
the
fix/retry-transient-429
branch
from
August 14, 2026 09:51
217af02 to
6ffe7ff
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #357.
The problem
Any 429 whose text reads like a spent allowance is never retried.
@langchain/core'sclassifyRateLimitErrormatches/insufficient[_ -]?quota/i, returns{ action: 'stop' }, anddefaultFailedAttemptHandlerthrows before p-retry gets a second attempt. For OpenAI that is right -insufficient_quotameans out of credit.Scaleway answers all three of its rate limits with that wording and nothing else to go on:
Tokens-per-minute and requests-per-minute differ only in
message, which the OpenAI SDK drops -APIErrorkeepsbody.erroralone - so the classifier only ever sees429 "INSUFFICIENT QUOTA"and reads a limit that clears in milliseconds as a billing wall. Nox-ratelimit-*and noretry-afteron the 429 either, although successful responses carry the former.The fix
An
onFailedAttempthandler on the OpenAI-compatible constructors that retries a 429 unless something structured says the allowance is gone:error.code === 'insufficient_quota'- OpenAI's billing wall - stopThe discriminator between the two quota meanings is the underscore:
insufficient_quotais OpenAI's error code,INSUFFICIENT QUOTAwith a space is Scaleway's rate-limit label. A caller-suppliedonFailedAttemptstill wins, so nothing that already sets one changes behaviour.Applied to
ChatOpenAI(hence Moonshot and OpenRouter),AzureChatOpenAI,ChatDeepSeekandChatXAI.Measured
Against api.scaleway.ai on 2026-07-30, driven through
ChatOpenAIfrom this package with the added probe script (npm run probe:ratelimit):The 58s is the token bucket refilling, and that is the trade this makes: a slow turn instead of a dead one. A rejected request is not billed, so the retries cost waiting and nothing else.
Tests
22 unit tests built on real
APIErrorinstances generated by the OpenAI SDK rather than hand-written objects, so the assertions run against the shape production sees - including a wiring test that the handler reachesmodel.caller.onFailedAttemptfor all six classes, and that an explicit handler is left alone.src/utils+src/llm/openai: 300 tests green.The probe script follows
context-overflow-probe.ts: it exists so the retry policy stays grounded in what providers send instead of in phrases someone expected them to send.