Providers: wrong exception class escaping the lifecycle - #19
Merged
Conversation
leeovery
changed the title
A server closing the socket mid-body raises http.client.IncompleteRead out of response.read() — the routine failure mode for a 100MB podcast enclosure. IncompleteRead is an HTTPException, not an OSError, so every caller's `except OSError` connection guard missed it and a truncated download landed as `error` with a filed issue, frozen until the next engine release, where blocked-with-retries is the honest outcome. Normalizing the whole http.client family into ConnectionError at the two urllib seams (the driver transport and whisper-api's multipart POST) fixes all eight call sites at once, including the media stage and the enclosure download, and routes them through classify_connection. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two escapes from the same provider. HuggingFace raises HFValidationError from the cache lookup for repo-id shapes like `Systran/faster-whisper/ large-v3` that the name check waves through on its slash — and that probe runs on every CLI verb, so a typo in transcribe_model crashed status, run and transcribe outright. The cache check is now guarded: a name HF rejects reports one unavailable provider, an unreadable cache says so. And faster-whisper indexes a container's audio streams unguarded, so a video-only file arrives as a bare IndexError — an engine bug with a filed issue, where whisper-api already calls the same file manual. It maps to ProviderInputError like every other bad input. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
csv.Error escapes from the reader's iteration, not its construction, so the comprehension building the rows let it through: a CSV field over the stdlib's 128KB default — routine for an embedded JSON blob — landed as an engine bug with an issue filed. A big field is not a malformed file, so the bound is raised to 16MB rather than the failure merely being caught. It stays finite: one unterminated quote in a file that only looked like CSV makes the whole file a single field, and a bound turns that into a stated manual instead of eating the machine's memory. Anything the reader still refuses is ProviderInputError. The limit is process-global, so it is restored. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Whisper keeps a prompt's LAST ~224 tokens and discards the front, so an 800-char priming string (~270 tokens of jargon-dense text) threw away the title and show it exists to carry. The composed prompt is now budgeted to ~200 tokens: the head survives whole, the vocabulary is trimmed from its own tail, and whisper-api's continuity tail shares that budget instead of being appended past it. Second, a CDN error page cached as <hash>.mp3 decoded as garbage and parked the episode manual forever. An empty body, a body short of its declared Content-Length, or an HTML page where audio was expected is now blocked and never written under the audio name — nothing about the episode was learned. Bytes that are merely bad audio still reach the provider and its manual stands. The file driver's HTML-lead sniff moves to pipeline/detect, where the byte sniffing lives, so both callers read the same bytes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The capability report dropped an ok-with-caveat note for available and active providers, so "model not cached — the first transcription downloads it" never reached the surface built to explain it; the note now rides any state, and the renderer prints it on the active row too. Duplicate provider names in config passed through, probing the provider twice and printing it twice on the report — refused now, as loudly as an unknown name. The extract dispatch handled ProviderInputError and ScannedDocumentError but not ProviderUnavailableError, so a contract-honoring extract provider that failed at call time got error plus a filed issue where the transcribe path re-parks waiting. It re-parks too. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The multipart POST is the second urllib seam wrapped by the transport normalization; nothing asserted it, so a future edit could drop the guard and send truncated uploads back to the engine-bug path. The socket double now reads the whole request — headers and body — before answering: replying mid-upload closes the socket on unread bytes and the client sees a reset rather than the truncated read under test. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The head-first character budget was plain truncation wearing a docstring: the head already sat at the FRONT of the composed string, so `text[:600]` kept it either way, and the claim that truncating instead "would keep 600 characters of show notes and throw away the two names" was false. Worse, whisper counts its window in TOKENS. 600 characters is ~180 tokens of Cyrillic and ~690 of Chinese, and past 223 tokens whisper keeps only `previous_tokens[-(448 // 2 - 1):]` — it discards the FRONT. For every CJK or Cyrillic item the title was therefore the first thing lost, by the code written to save it, and whisper-api's chunk continuity compounded it. Two changes. The title/show now goes at the END of the prompt, behind the vocabulary: whatever overflows the window is show notes, by construction, in any script. And the budget is spent through a per-character estimate of what whisper's BPE charges, measured against openai/whisper-tiny's tokenizer and rounded up per script family — so a prompt does not overflow in the first place. The tokenizer itself is not loaded: that means a HuggingFace fetch inside a step that primes remote providers needing no local model, to buy an exactness the head's placement already made unnecessary. A mis-estimate now costs vocabulary, not names. A live-marked test checks the estimate against the real tokenizer in nine scripts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Three sites of one defect. The transport's error-body read already suppressed `http.client.HTTPException` and nothing pinned it: with the suppress narrowed to OSError the whole suite stayed green, while a real socket serving a truncated 404 turned the item's `dead` into `blocked` — the read failure escaping through `normalize_httplib_errors` as a connection error. That case is now a socket test. whisper-api's multipart POST, twenty lines from its sibling, had no such guard at all: a 400 whose body stopped short of its Content-Length raised out of `e.read()`, and `_transcribe_chunk`'s `except OSError` filed it as ProviderUnavailableError. The episode then waited forever on an endpoint that was answering fine, instead of parking manual under the escalation clock the bad audio had earned. The two seams are symmetric now, both pinned against a socket that hangs up mid-body. dex-inbox's two urllib sites had neither the guard nor a caller that could report what escaped: `main` states OSError, ValueError and RuntimeError, and an IncompleteRead is none of them, so a GitHub asset download cut mid-body reached the operator as a traceback. Both sites go through the seam's guard. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
leeovery
force-pushed
the
fix2/providers
branch
from
August 22, 2026 14:14
48f610a to
3da62d1
Compare
leeovery
marked this pull request as ready for review
August 22, 2026 14:14
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Seven confirmed provider-boundary defects from the sweep. The unifying theme: the wrong Python exception class escaped the intended lifecycle, landing as "engine bug, filed GitHub issue, frozen until the next release" where the design mandates a retryable block or a manual park.
http.client.IncompleteReadis not anOSError, and it is the normal failure mode for 100MB+ podcast enclosures. Normalised at the transport seam, so all eight callers are fixed at once and a truncated read isblocked, not an engine error.status,runandtranscribeall exited 1.IndexErrorout of whisper-local as an engine bug, while whisper-api mapped the same input correctly tomanual— same input, opposite lifecycle, decided by which provider resolved.csv.Errorescape on an oversized field, in exactly the fallback scenario it exists to serve. The field bound is now raised to 16MB with a stated refusal beyond it..mp3parked the episodemanualforever whereblockedis honest; empty, short-of-content-length and HTML-lead bodies are now blocked and never cached.ProviderUnavailableErroras a waiting re-park.Gates: 1087 tests, ruff and ty clean.
🤖 Generated with Claude Code