Repository navigation
fix(audio): keep audio downloads in the app's cache - #141
Conversation
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: d90aa927-b2a5-4119-a007-73fba23f2359) |
|
Local pipeline on d0bf53c (
Gated, same head, Fork tests on da4cd79 ( Cursor Bugbot has reported "usage limit reached" since 2026-10-06 and did not review this PR. |
Six speech-to-text families (voxtral, cohere, canary, wav2vec2/mms, lasr, moonshine) resolved their model a second time inside the loader without the cache AudioEngine passes, and on the text-to-speech side Chatterbox and OmniVoice did the same for their whole model, and the Llama and Qwen3 voices and Echo for the codec they fetch from a second repo, so those downloads landed in ~/.cache/huggingface, and since #134 inside the generation lock (#135). The controlled fork now passes the cache through those loaders (the STT half is upstream issue Blaizzy/mlx-audio-swift#279) and carries upstream's pending PR Blaizzy/mlx-audio-swift#257, without which a snapshot fetched by the first resolve would have counted as complete for a loader that asks for more patterns, such as cohere's tokenizer.model or SenseVoice's am.mvn. That PR recorded the patterns a load asked for; the Hub client serves the cached snapshot offline, so it could certify files never downloaded. The fork certifies by the Hub's listing instead, fetches one by one only the listed files not on disk at the revision the snapshot came from (or main when the app's Hub cache does not record that revision or the Hub no longer has the commit), refuses a listed path that cannot be written under the model directory, bounds the listing in time, backs off a repo whose listing ran out of time or took seconds to fail, a 404 included (a failure that comes back at once is asked again next time, and neither a cancelled caller nor a fetch with nothing to certify books one), certifies a fresh fetch from what landed on disk, drops a manifest found beside a snapshot it could not use, and records nothing offline. The pin moves from d138532 to da4cd79. Upstream's v0.1.5 builds against mlx-swift-lm 3.32.3 but has neither fix, so the fork stays for the cache fixes alone. A gated smoke loads a model through AudioEngine with HF_HUB_CACHE pointed at an empty directory and checks that it stays empty, that the files a loader's own patterns name land in the app cache, and that the weights the prefetch wrote are not replaced; it loads a speech model too, and checks the codec it fetches from a second repo lands in the app cache as well.
d0bf53c to
6f2451a
Compare
|
Rebased onto main at 0396f69 (after #140), which also folded in the last review's wording fixes (the CHANGELOG, both READMEs, the Package.swift and model-library comments). Local pipeline on 6f2451a (
Gated, same head, Fork tests on da4cd79 ( Cursor Bugbot reported "usage limit reached" on this PR too (it has since 2026-10-06) and did not review it. |
1 participant
Closes #135.
AudioEnginehands the upstream loaders aHubCacherooted at~/.mac-mlx/audio-models, and since #134 fetches the snapshot there before taking the generation lock. Six speech-to-text families (voxtral, cohere, canary, wav2vec2/mms, lasr, moonshine) then resolved the model a second time inside theirfromPretrained, without the cache they were given, so the weights they actually read came from a second full download into~/.cache/huggingface: twice the disk, a copy the model library does not manage, and, once #134 moved the load under the lock, a first use that stalled every other request for the length of that download. On the text-to-speech side, Chatterbox and OmniVoice did the same for their whole model (Chatterbox's S3 tokenizer, 495 MB, with it), and the Llama and Qwen3 voices (their SNAC codec, 79 MB) and Echo (its Fish codec, 1.87 GB) for the codec they fetch from a second repo. Upstream issue Blaizzy/mlx-audio-swift#279 covers the six STT loaders, has had no answer, and upstream's loaders still drop the cache (v0.1.5, released on 2026-10-05, builds against mlx-swift-lm 3.32.3 but has neither this fix nor the completeness rework below), so the fix goes into the controlled fork; rebasing the fork onto v0.1.5 is a follow-up.The fork (branch
v0.1.3-macmlx, d138532 → da4cd79):resolveOrDownloadModelcounted a cached snapshot as complete as soon as it held weights and a validconfig.json, so the snapshot the first resolve creates with the default patterns satisfied a loader that asks for*.modelor*.mvn, and the loader found those files missing. SenseVoice in v0.9.0 loaded exactly that way, withoutam.mvnand its tokenizer. The PR records the patterns a snapshot was fetched with in.mlx-audio-patternsand re-downloads when a loader asks for more.*.model; no Whisper repo ships one), so every cold Whisper load would have listed the repo again, under the lock. The Hub's listing is now what certifies a pattern: a resolve that finds a usable snapshot missing some patterns asks the Hub once, or twice when the cached commit is gone (ten seconds each at most; a repo whose listing ran out of time, or took two seconds or more to fail, a 404 included, is not asked again for ten minutes, so a network that drops packets or a proxy that answers late costs one attempt per repo every ten minutes instead of that much per resolve under the lock, while a failure that comes back at once, no network, a refused connection or a 404, is asked again at the next load; a fetch with nothing to certify does not list, and a cancelled caller is not booked as the Hub's failure), fetches one by one only the listed files not yet on disk, at the revision the snapshot came from (ormainwhen the app's Hub cache does not record that revision, or after one more listing when the Hub no longer has the commit), so the weights are neither downloaded nor copied again, refusing a listed path that cannot be written under the model directory, and records the patterns as complete, including those the repo has no file for. A fresh fetch records a pattern only when its listed files landed on disk, since the snapshot download serves the client's own cached snapshot when it cannot list, and drops a manifest left beside a snapshot it could not use (weights with no config) before it fetches. A name with no glob characters whose file is on disk is complete by itself, with no listing. Without a listing the snapshot is served as it is and nothing is recorded, so a later online load still fetches what is missing. The token clients, ModelUtils' and Whisper's, followHF_ENDPOINTlike the plain one. Twenty-six fork tests, run serialized, all but the path-rule table against a stub Hub (one that answers the listing, answers 404 for a revision at once or late, fails at once or after seconds, serves the files a test allows, can hang, and counts every request; or one that refuses every request at once): among them the fetch at the cached commit, the cached commit the Hub no longer has completed frommain, a repository it no longer has served as it is and asked again, the refused path and the path rule as a table, the fresh fetch that lists once, a fresh fetch with a pattern certified once its file landed, a fetch served from the client's own cache certifying nothing it lacks, a quick failure booking no backoff, a failure that takes seconds booking one, a 404 pair that answers late booking one, a manifest beside an unusable snapshot dropped before a fresh fetch (and still gone when that fetch fails) and not trusted by the next resolve, the timeout backoff through a Hub that never answers, and the cancellation that leaves no backoff.cache:threaded through the six STT loaders and the dispatcher (all 16resolveOrDownloadModelcalls underMLXAudioSTTpass it), and on the TTS side through Chatterbox (its model and S3 tokenizer), OmniVoice, and the codec loads of the Llama and Qwen3 voices and Echo. Still on the default cache: the tokenizer or codec of a model loaded from a local directory rather than by repo id (that source carries no cache), and Kokoro's and Kitten's grapheme-to-phoneme loader (Kokoro English G2P ignores the HubCache supplied to TTS.loadModel Blaizzy/mlx-audio-swift#242).Here: the pin and both lockfiles, the CHANGELOG, both READMEs' queued line, the Package.swift comment and the CLI pin test's comment (which rest the pin on the cache fixes, since upstream v0.1.5 builds against 3.32.3 on its own), the AudioEngine note, the library's doc comment on completeness, and a gated smoke that loads a model through
AudioEnginewithHF_HUB_CACHEpointed at an empty directory and asserts the directory stays empty, the app cache holds the snapshot and the files the loader's own patterns name, and the weights the prefetch wrote keep their file identity across the load (nothing re-copied them). Run on this machine, each with the default cache empty before and after: moonshine-ai/moonshine-tiny (5.3 s; the loader'stokenizer.*was already on disk, one listing, nothing fetched), Mediform/canary-1b-v2-mlx-q8 (run at a50c44b, an earlier pin, with an earlier version of this test that already checked the weights' identity; the listing rule on that path is unchanged since:*.modelandtokens.txtname nothing in that repo, one listing, nothing fetched, the patterns recorded so the next cold load makes no request), mlx-community/SenseVoiceSmall (22.9 s including 936 MB; the loader's*.mvn,*.model,tokenizer*fetched exactly two files,am.mvnandchn_jpn_yue_eng_ko_spectok.bpe.model, found in the app snapshot afterwards with the weights file untouched), and a speech model, mlx-community/VyvoTTS-EN-Beta-4bit, a Qwen3 voice (30.4 s: the model, then its SNAC codec from a second repo, both into the app cache, the codec's snapshot checked). The Llama and Echo codec paths are the same one-line change as the Qwen3 one and are covered by inspection, not by a run. The first version of this test, built against the old pin (d138532), fails: the moonshine loader fetched a second copy into the default cache (the mlx-audio copy plus the Hub cache's blobs, 221 MB) and the assertion listed every file.Upgrade notes, also in the CHANGELOG: the copies v0.9.0 left under
~/.cache/huggingface/hubare not removed; a snapshot cached by v0.9.0 is completed on its first load with a listing (two when the Hub no longer has the cached commit) and at most a few small files, and cannot be completed offline until then (cohere's app copy lackstokenizer.modeluntil one online load).