stt-api

An OpenAI-compatible speech-to-text server on ONNX Runtime. It serves NVIDIA's Parakeet TDT 0.6B v3 (25 European languages) and v2 (English), and OpenAI's Whisper, on CPU or GPU. Point an OpenAI client at it and call /v1/audio/transcriptions.

Parakeet's Token-and-Duration Transducer (TDT) architecture lets it transcribe many times faster than real time on a consumer CPU, and faster still on a GPU; see Performance. The project started as a Parakeet v3 server, which is why its settings are named PARAKEET_*.

Upgrading from 1.5.0? Read UPGRADING.md first: every request must now name its model.

Quick start

Docker

docker run -d --name parakeet-cpu -p 5092:5092 -v parakeet-models:/app/models \
    -e PARAKEET_PRELOAD_MODELS=parakeet-v3 ghcr.io/scagood/stt-api:latest-cpu

The first start downloads parakeet-v3 before the server reports ready. For the GPU image, docker compose and the image tags, see DOCKER.md.

From source

You need Python 3.14 and FFmpeg. In a virtual environment (venv or conda):

git clone https://github.com/scagood/stt-api
cd stt-api
pip install -r requirements.txt
python server.py   # GPU, on :5092

requirements.txt installs onnxruntime-gpu, which has no macOS build. For a CPU-only install, swap it for onnxruntime at the same version, as Dockerfile.cpu does, and turn the GPU off:

sed 's/^onnxruntime-gpu\[[a-z,]*\]==/onnxruntime==/' requirements.txt > requirements.cpu.txt
pip install -r requirements.cpu.txt
PARAKEET_USE_GPU=false python server.py

On Linux, install the CPU build of PyTorch first (silero-vad depends on it), or pip pulls the multi-GB CUDA wheels; Dockerfile.cpu shows how.

First request

curl http://localhost:5092/v1/audio/transcriptions \
  -F file=@audio.mp3 -F model=parakeet-v3

{"text":"The quick brown fox jumps over the lazy dog."}

With the OpenAI Python SDK (pip install openai; the server doesn't need it):

from openai import OpenAI

client = OpenAI(base_url="http://localhost:5092/v1", api_key="sk-no-key-required")

with open("audio.mp3", "rb") as f:
    transcript = client.audio.transcriptions.create(model="parakeet-v3", file=f)
print(transcript.text)

Swagger UI at http://localhost:5092/docs lets you try every request from the browser.

Choosing a model

Every request names its model; there is no default, and a request without one is a 422. Add a precision after a colon (parakeet-v3:fp16), or send it as the quantization field; if you send both, they must agree. With no precision you get fp32. An unknown model or precision is a 400 that lists the valid ones.

Model	Languages
parakeet-v3	25 European	The main model
parakeet-v2	English
whisper-tiny, whisper-base, whisper-small, whisper-medium, whisper-large-v3, whisper-large-v3-turbo	99	Word times only with an aligner
whisper-tiny.en, whisper-base.en, whisper-small.en, whisper-medium.en	English	Word times only with an aligner

Every model comes in fp32, fp16 and int8. Parakeet v3 transcribes Bulgarian, Croatian, Czech, Danish, Dutch, English, Estonian, Finnish, French, German, Greek, Hungarian, Italian, Latvian, Lithuanian, Maltese, Polish, Portuguese, Romanian, Russian, Slovak, Slovenian, Spanish, Swedish and Ukrainian, without being told which it is hearing.

curl http://localhost:5092/v1/audio/transcriptions \
  -F file=@audio.mp3 -F model=parakeet-v3:fp16

Choosing a precision. On a GPU, fp16 roughly halves VRAM. On a CPU, keep fp32: ONNX Runtime upcasts fp16 there, which is slower. int8 is the fastest on CPU, but pick it deliberately: the 1.x int8 export dropped words after silences and was ~4 WER points worse than fp32 on Spanish. The current one matched fp32 on English (below) but has no multilingual numbers yet.

Which export. parakeet-v3 runs Olicorne's re-export of NVIDIA's .nemo checkpoint in all three precisions. On a 648 s English audiobook chapter it scored 1.13% (fp32), 1.07% (fp16) and 1.20% (int8) WER, against 1.20%, 1.20% and 1.83% for the istupakov/grikdotnet exports it replaced, with int8 ~25% faster. It was chosen on CPU and has not been tested on a GPU or outside English; the parakeet-v3 entry in parakeet_service/models.yaml lists what to revert to if CUDA gives trouble.

Loading. Models named in PARAKEET_PRELOAD_MODELS load at startup; the rest load on first request (PARAKEET_MODEL_CACHE_SIZE caps how many stay loaded). A model or aligner, preloaded or not, that no request has used for 6 hours is unloaded to free its memory, and the next request that names it loads it again, without a warm-up; set PARAKEET_MODEL_IDLE_TIMEOUT_SEC to another number of seconds, or to -1 to keep everything loaded. A model that can't be loaded (a failed download, missing from the cache under PARAKEET_HF_OFFLINE=true, refused by ONNX Runtime) answers 503 with a detail naming it and the cause; the next request tries again.

GET /v1/models lists every model; GET /v1/models/parakeet-v3 returns one:

{"id":"parakeet-v3","object":"model","created":1785888000,"owned_by":"nvidia","language":["bg","hr","cs","da","nl","en","et","fi","fr","de","el","hu","it","lv","lt","mt","pl","pt","ro","ru","sk","sl","es","sv","uk"],"quantizations":["fp32","fp16","int8"],"task":"automatic-speech-recognition"}

Response formats

response_format is json (the default), text, srt, vtt or verbose_json. verbose_json always has segments, and has words when you send timestamp_granularities[]=word (words is null otherwise). timestamp_granularities[] takes word and segment; anything else is a 400.

curl http://localhost:5092/v1/audio/transcriptions \
  -F file=@audio.mp3 -F model=parakeet-v3 \
  -F response_format=verbose_json -F 'timestamp_granularities[]=word'

{
  "task": "transcribe",
  "language": "auto",
  "duration": 2.3473125,
  "text": "The quick brown fox jumps over the lazy dog.",
  "segments": [
    {"id": 0, "seek": 0, "start": 0.0, "end": 2.3473125,
     "text": "The quick brown fox jumps over the lazy dog.",
     "tokens": [], "temperature": 0.0, "avg_logprob": 0.0,
     "compression_ratio": 0.0, "no_speech_prob": 0.0}
  ],
  "words": [
    {"start": 0.0, "end": 0.16, "word": "The"},
    {"start": 0.16, "end": 0.4, "word": "quick"},
    {"start": 0.4, "end": 0.72, "word": "brown"},
    ...
    {"start": 2.0, "end": 2.3473125, "word": "dog."}
  ]
}

A segment's tokens, temperature, avg_logprob, compression_ratio and no_speech_prob are always empty or zero; they are there for OpenAI clients that expect them. language echoes the request's, or auto.

Word timestamps

Parakeet's own word times sit on 80 ms frames, and each word's end is estimated: in the example above, every word ends exactly where the next one starts. Name an aligner and a forced aligner retimes the words from the audio instead, on 20 ms frames, WhisperX-style but on ONNX Runtime with no PyTorch. Parakeet still decides the words.

Parakeet's times also slip into pauses: the word before a pause can start after the speech has stopped, and the word after it before the speech has started. In a 41-minute LibriVox chapter, 249 words lay entirely inside a pause (ffmpeg silencedetect, -30 dB, 0.3 s or longer), each a real word timed into the pause beside it. With aligner=wav2vec2-base-960h, none did. If you match words against text or silence, as an audiobook aligner does, name an aligner, or send retime_words=true.

retime_words=true finds the pauses by loudness and moves only the words touching one to its edge: a word that starts in a pause starts at its end, one that ends in it ends at its start, and one wholly inside goes before it if it ends a sentence or clause, after it if not. Every other word keeps Parakeet's time. It costs about 0.1 s per hour of audio, against the aligner's 3.4x the transcription time on a CPU, but it is not as good: the aligner times every word from the audio. Against the aligner's times on three LibriVox chapters, it cut the words lying inside a pause from 249, 68 and 79 to 15, 14 and 43, and the words starting over 200 ms from the aligner's start from 9.4%, 9.3% and 17.8% to 3.1%, 3.8% and 11.4%.

A pause holds one word at each edge, the last before it and the first after it. When more than one word wholly inside a pause would go to the same side, or one going after it comes before one going before it, it is no pause: it is speech the gate missed, or times off by more than a word, and those words keep Parakeet's times rather than pile up at its edge. A quiet stretch of 3 s or more is heard again at its own level, as when finding pauses in long audio, so a quieter speaker's turn gets pauses of its own instead of being one. In a 25-word turn 20 dB quieter than the rest, retime_words used to move 24 words' starts over 200 ms from the aligner's (aligning the turn on its own) and squeeze 23 into 40 ms; now 3 and 1, against Parakeet's own 5 and 0. On three LibriVox chapters it moves 23 of 5,967 words differently than before, 15 of them closer to the aligner and 8 farther: most are runs Parakeet timed more than a word early or late, which no edge fixes.

With an aligner named, it only re-times chunks the aligner could not. PARAKEET_RETIME_WORDS=true turns it on for requests that don't say.

curl http://localhost:5092/v1/audio/transcriptions \
  -F file=@audio.mp3 -F model=parakeet-v3 \
  -F response_format=verbose_json -F 'timestamp_granularities[]=word' \
  -F language=en -F aligner=mms-300m-forced-aligner

transcript = client.audio.transcriptions.create(
  model="parakeet-v3",
  file=f,
  response_format="verbose_json",
  timestamp_granularities=["word"],
  language="en",
  extra_body={"aligner": "mms-300m-forced-aligner"},  # non-commercial licence
)

The same clip, before and after (seconds):

Word	Parakeet	mms-300m-forced-aligner
The	0.00–0.16	0.04–0.10
quick	0.16–0.40	0.18–0.36
brown	0.40–0.72	0.42–0.66
fox	0.72–1.04	0.72–0.94
jumps	1.04–1.36	1.12–1.36

Name the aligner as you name a model, with its precision after a colon (aligner=mms-300m-forced-aligner:fp32); without one you get int8. There is no default aligner: a request that names none gets Parakeet's times. GET /v1/aligners lists them:

Aligner	Languages	Error, start / end (English TTS)	License
mms-300m-forced-aligner	en	37 / 106 ms	CC-BY-NC-4.0: non-commercial only
wav2vec2-large-xlsr-53-english	en	48 / 104 ms	Apache-2.0 (the model it exports)
omnilingual-ctc-300m	en and Parakeet v3's other 24	45 / 117 ms	Apache-2.0
wav2vec2-base-960h	en	57 / 131 ms	Apache-2.0

For English, use mms-300m-forced-aligner: it is the most accurate, and on real audiobook narration it sounds clearly the best. Its licence is non-commercial; for commercial use, wav2vec2-large-xlsr-53-english is the next best. wav2vec2-base-960h is the smallest and fastest, but the least accurate. For the other languages, use omnilingual-ctc-300m:

curl http://localhost:5092/v1/audio/transcriptions \
  -F file=@entretien.mp3 -F model=parakeet-v3 \
  -F response_format=verbose_json -F 'timestamp_granularities[]=word' \
  -F language=fr -F aligner=omnilingual-ctc-300m

Each aligner comes in int8 (the default) and fp32, which is about as accurate and 4x the download. All but omnilingual-ctc-300m also come in fp16, half the download of fp32, for GPUs: on a CPU, keep fp32 or int8, since ONNX Runtime upcasts fp16 there, which is slower than fp32.

On a GPU server, fp16 and fp32 aligners run on the GPU, as the models do. int8 always runs on the CPU: ONNX Runtime has no CUDA kernels for its 8-bit operations. The default stays int8 whatever the hardware, so to align on the GPU, ask for it: aligner=mms-300m-forced-aligner:fp16, or omnilingual-ctc-300m:fp32. This has not been tested on a GPU yet; PARAKEET_ALIGN_GPU=false keeps every aligner on the CPU, as before.

Which aligners exist, and the languages each aligns, is set in the model catalog. An unknown aligner or precision, or a language the aligner doesn't align, is a 400 that lists what is available.

Language. language is a bare ISO 639-1 code (en, fr), or empty or auto; anything else (en-US, EN, English) is a 400. Nothing detects the language for alignment: a request without one is aligned as PARAKEET_ALIGN_DEFAULT_LANGUAGE, English unless you change it, so send language for anything else. A chunk mostly in another alphabet (Cyrillic, Greek, ...) keeps Parakeet's times, but other Latin-script languages would be aligned as English. English-only models (parakeet-v2, whisper-*.en) are aligned as English whatever language says.

Whisper has no word times of its own, so it returns words only when the request names an aligner, and a multilingual Whisper model only when the request also sends language. If a chunk can't be aligned at all, its words are null, never guessed; a word the aligner can't place sits between its aligned neighbours.

Numbers and symbols are aligned as they are said, by the English aligners: 42 as "forty two", $5m as "five million dollars", 20°C, 5 May as "the fifth of May", R&D as "ar and dee". Only the timing uses this; the text is never changed. A count in year range (1500) is heard as a year, so if it was said "one thousand five hundred" it starts a little late. omnilingual-ctc-300m skips numbers in every language, English included: a number keeps Parakeet's times, and the words around it are aligned as usual.

Transcript mistakes. On clean speech, a word Parakeet missed or got wrong doesn't drag its neighbours' times; in heavy noise it occasionally still does. An invented word in a pause takes the pause, but a long one invented in the middle of continuous speech pushes its neighbours aside.

Cost. The aligner runs one request at a time, on its own CPU threads (int8, or any aligner without a GPU) or on the GPU (fp16 and fp32 on a GPU server), so it never holds up other requests' audio decoding. Each aligner downloads on the first request that names it (int8: ~320 MB, or ~95 MB for wav2vec2-base-960h) and adds roughly 3 s per 30 s of audio on a 4-core machine (2 s for wav2vec2-base-960h). If a download fails, words keep Parakeet's times and the load is retried every 5 minutes; /health lists the aligners that failed to load under failed.

Spoken numbers

Parakeet writes numbers its own way, and not consistently: "twenty-five pounds" may come back as £25 or 25 lb, "ten thirty" as 1030. With spoken_numbers=true, English transcripts (text, segments, words, every response format, and the batch endpoint) say numbers, money and units in words, the way they were said:

Parakeet wrote	Transcript says
$5, cost$25	five dollars, cost twenty-five dollars
£25, 25 lb	twenty-five pounds
$5 million, $5m	five million dollars
20°C, 50%, 21st	twenty degrees Celsius, fifty percent, twenty-first
5 May, 90 mph	the fifth of May, ninety miles per hour
MP3, COVID-19, 5m, 12C	unchanged: names, or ambiguous

curl http://localhost:5092/v1/audio/transcriptions \
  -F file=@audio.mp3 -F model=parakeet-v3 \
  -F spoken_numbers=true -F language=en

From the OpenAI SDK, send extra_body={"spoken_numbers": True}. PARAKEET_SPOKEN_NUMBERS=true turns it on for requests that don't say; they can still send spoken_numbers=false.

How it chooses. Different speech often comes out as the same text: £2.10 is "two pounds ten" or "two ten", 911 is "nine one one" or "nine eleven". So an aligner scores each number's readings against the audio and keeps the one that was said: the request's aligner if it names one, else PARAKEET_SPOKEN_NUMBERS_ALIGNER (wav2vec2-base-960h), which only listens and leaves the word times alone. Each number is heard in its own stretch of audio, a second either side, so one said quieter than the rest of its chunk is heard as clearly. It also scores likely mishearings of an amount (£1.10 for "two pounds ten", €3 for "thirty euros"), and replaces Parakeet's number only when the audio clearly prefers one. With PARAKEET_SPOKEN_NUMBERS_ALIGNER empty and no aligner in the request, each number gets a fixed reading, usually the most common one, but a code or a time is read as an amount (911 as "nine hundred eleven", 1030 as "one thousand thirty"). The choice was tuned with wav2vec2-base-960h; the other aligners haven't been measured for it yet.

Cost. Most chunks with a number need the aligner (97% in our test corpus), whether or not the request names one: it downloads on first use (~95 MB) and stays loaded, and each such chunk queues with word requests on the aligner's single worker. Each number is then heard once more, in its own stretch of audio. On a 44 s chunk with 14 numbers (a 4-core cloud CPU), saying the numbers took 3.5 s with fixed readings and 22-25 s heard; a clip with one number went from 0.6 s to 1.2 s. Requests whose numbers have nothing to decide (6pm) aren't held up. Without Hugging Face access, set PARAKEET_SPOKEN_NUMBERS_ALIGNER= (or fill the cache beforehand): otherwise each failed download falls back to fixed readings and is retried every 5 minutes.

Language. English only, decided as for word timestamps: a request without language is taken as PARAKEET_ALIGN_DEFAULT_LANGUAGE. A chunk mostly in another alphabet is left as written, but Latin-script languages sent without language are rewritten as if English, so send language, or set the default empty if you serve them.

Silence

Given no speech, Parakeet makes some up: two seconds of digital silence come back as Thank you., a pause cut out of an audiobook as Yeah. or Okay.. Audio longer than the model's chunk_max_sec (75 s for parakeet-v3) is cut at pauses, and stretches with no speech never reach the model. Shorter audio goes to the model whole. With vad_filter=true, Silero-VAD first checks it for speech, whatever PARAKEET_VAD says: audio without any comes back empty (no text, segments or words), and audio with speech is transcribed whole, as before, never trimmed.

curl http://localhost:5092/v1/audio/transcriptions \
  -F file=@clip.wav -F model=parakeet-v3 -F vad_filter=true

From the OpenAI SDK, send extra_body={"vad_filter": True}. PARAKEET_VAD_FILTER=true turns it on for requests that don't say; they can still send vad_filter=false. Without silero-vad installed, the filter falls back to volume, which drops digital silence but lets most noise through.

What it catches, and what it costs. It can drop real speech, which is why it is off by default. Measured with Silero on CPU, with Parakeet's output inferred from #64 rather than run here. The speech is 300 one-word recordings (spoken digits, 6 speakers, median 0.42 s, median peak -11 dBFS):

Clips	Kept (sent to the model)
the words cut tight, starting at the first sample	300 of 300
the words padded to 1.5 s	299 of 300
the same, with background noise at 5 / 0 dB SNR	299 / 278 of 300
the words 10 dB quieter, tight / padded	295 / 297 of 300
the words 20 dB quieter, tight / padded	235 / 245 of 300
non-speech sounds (ESC-50: rain, engines, dogs, ...), up to 5 s	19 of 176
coughs, laughs, cries, sneezes, snores, breathing (ESC-50)	56 of 120
digital silence, dither, white and pink noise at -80 to -20 dBFS, hum, clicks	none

Silero misses a word at a clip's very first sample (16 of the tight words above), so it hears each clip with 0.25 s of silence either side. It only counts speech it hears for at least PARAKEET_VAD_FILTER_MIN_SPEECH_MS (default 0: any 32 ms window). Its own default, 250 ms, drops short words it is sure of, such as "up", "go" or "no": at 250 ms, 281 of the padded words were kept, 249 at 0 dB SNR and 200 at 20 dB quieter, for 2 non-speech sounds and 27 coughs and the like. PARAKEET_VAD_THRESHOLD applies too: at 0.2, 289 of the padded words 20 dB quieter were kept, and 294 at 0 dB SNR, but 80 non-speech sounds and 93 coughs and the like.

It hears the first 2 s first, so audio with speech there costs about 25 ms of one CPU core; only audio without pays for the rest (about 12 ms per second of audio), and it is then spared the model.

Batch transcription

POST /v1/audio/transcriptions/batch takes several files in one request, with the same model, quantization, aligner, spoken_numbers and vad_filter fields as the single-file endpoint, but no language or response_format. It returns text only:

curl http://localhost:5092/v1/audio/transcriptions/batch \
  -F files=@fox.wav -F files=@fox.aiff -F model=parakeet-v3

{"results":[{"filename":"fox.wav","text":"The quick brown fox jumps over the lazy dog.","duration":2.3473125},{"filename":"fox.aiff","text":"The quick brown fox jumps over the lazy dog.","duration":2.3473125}],"batch_size":2}

A batch is limited to PARAKEET_MAX_BATCH_FILES (16) files and PARAKEET_MAX_BATCH_BYTES (512 MiB); see request limits.

Forced alignment

When you already have the text (lyrics, a script, a book's text), POST /v1/audio/alignments times its words from the audio without transcribing it, so nothing is misheard. Send the audio, the text and an aligner, plus language for anything but English:

curl http://localhost:5092/v1/audio/alignments \
  -F file=@quilter.wav -F aligner=mms-300m-forced-aligner \
  -F 'text=Mister Quilter is the apostle of the upper classes.'

{"aligner":"mms-300m-forced-aligner:int8","language":"en","duration":5.855,"words":[{"word":"Mister","start":0.56,"end":0.8,"score":0.344},{"word":"Quilter","start":0.84,"end":1.2,"score":1.0},{"word":"is","start":1.28,"end":1.36,"score":1.0},{"word":"the","start":1.42,"end":1.5,"score":1.0},{"word":"apostle","start":1.56,"end":2.08,"score":1.0},{"word":"of","start":2.16,"end":2.24,"score":1.0},{"word":"the","start":2.28,"end":2.36,"score":1.0},{"word":"upper","start":2.42,"end":2.62,"score":0.005},{"word":"classes.","start":2.64,"end":3.16,"score":1.0}]}

(Rounded here. The reader said "middle classes", and goes on past the text.) Each run of characters between spaces is one word, punctuation and all, in the order sent. aligner and language work as they do for word timestamps, except that the aligner has to be named. A word with no letters the aligner knows (—, or a number for omnilingual-ctc-300m) has null times and score.

Score. A forced aligner places every word somewhere, said or not. score, 0 to 1, says how closely the word matches what the aligner hears on the frames it was given: per 20 ms frame, the probability of the letter (or gap) forced there over that of the one the aligner hears best, averaged geometrically. A word that was said scores near 1; one that wasn't, near 0. On 73 LibriSpeech clips (8 minutes; 1,150 words), with one word inserted into each text and one replaced (146 words not said), the share of each that scored under a threshold (int8):

Aligner	Under 0.3: said / not said	Under 0.5: said / not said
mms-300m-forced-aligner	1.0% / 97.3%	3.0% / 99.3%
wav2vec2-large-xlsr-53-english	2.4% / 100%	3.3% / 100%
omnilingual-ctc-300m	0.8% / 97.3%	1.5% / 100%
wav2vec2-base-960h	5.9% / 100%	8.3% / 100%

Rejecting words under 0.5 is a reasonable start. wav2vec2-base-960h mishears more, so it rejects more words that were said. Two things a score can't tell you: a word that wasn't said where the text puts it, but was said nearby ("the", "and"), can score high, since the aligner finds it there; and a word that wasn't said can pull its neighbours' scores down with it, as it takes their frames. A text that's wrong throughout shows as many low scores.

Limits. The whole clip is aligned in one pass, whose memory grows with the audio's length times its text's, so audio is limited to PARAKEET_ALIGN_MAX_AUDIO_SECONDS (60 s); longer is a 413, so cut long audio at pauses and align each piece. Text the aligner can't spell, or with more letters than the audio can hold, is a 400; an aligner that can't load is a 503, retried after 5 minutes. Alignments run one at a time on the aligner's worker, queued with word-timestamp requests.

Comparing models and aligners by ear

With PARAKEET_COMPARE_UI=true, GET /compare serves a page for choosing between them by listening. Pick an audio file and cut it to a clip, then add rows: a model at a quantization, with or without an aligner (parakeet-v2:int8; parakeet-v2:int8 + mms-300m-forced-aligner:int8; ...). Compare sends the clip through /v1/audio/transcriptions once per row and lines up each row's words under the clip's waveform. Click a word to hear the span that row gave it; the same word is picked out in every row, and in a table of starts and ends. The arrow keys step through words and rows, Enter replays, and playback can be slowed to ½×. A file of exact word times, as synthetic speech has (a verbose_json response or its words, or SRT/VTT with one cue per word), adds a dashed reference row and each row's average error against it.

The browser decodes the file and sends only the clip, as a 16 kHz WAV, so the page and the server hear the same samples. The page is off by default: each row is a full transcription, and loads any model or aligner it names, which then stays loaded until it goes unused for PARAKEET_MODEL_IDLE_TIMEOUT_SEC.

Transcribing in the browser

With PARAKEET_TRANSCRIBE_UI=true, GET / and GET /transcribe serve a page for getting the transcript of one file without the API, with links to /docs and, when it is on, /compare. Pick a file (or drop it on the page, or record one from the microphone), choose a model (the first in PARAKEET_PRELOAD_MODELS to begin with), an optional aligner, a language and the spoken_numbers, vad_filter and retime_words toggles, which start at the server's own defaults. Transcribe sends the file as it is to /v1/audio/transcriptions, so the server decodes it and its limits apply; the page warns before sending a file near PARAKEET_MAX_UPLOAD_BYTES or PARAKEET_MAX_AUDIO_SECONDS, and shows the upload's progress and the time spent waiting. The transcript is listed by segment: click a time to hear the segment, or a word to hear the word. While the audio plays, the player stays at the top of the window and the playing word is marked, and the page scrolls to keep it in view unless you scroll away. Copy the text, or download it as text, SRT, VTT or the verbose_json response.

Browsers allow the microphone only over HTTPS or on localhost; on a plain HTTP host the page says so and still takes files. The page is off by default, and turned on separately from /compare: it loads any model or aligner it names, as a request would.

Open WebUI

This server works as Open WebUI's speech-to-text engine. Start it (see Quick start), then in Open WebUI Settings → Audio:

• STT Engine: OpenAI
• OpenAI Base URL: http://127.0.0.1:5092/v1
• OpenAI API Key: sk-no-key-required
• STT Model: parakeet-v3 (required: the server has no default model)

Configuration

Every setting is an environment variable, and all are optional. The Docker images already set the CPU or GPU ones (PARAKEET_USE_GPU, PARAKEET_BATCHED, ...).

Server

Variable	Default
PARAKEET_HOST	0.0.0.0	bind address
PARAKEET_PORT	5092	bind port
PARAKEET_UVICORN_WORKERS	1	uvicorn worker processes; each loads its own copy of every model
PARAKEET_COMPARE_UI	false	serve the /compare page
PARAKEET_TRANSCRIBE_UI	false	serve the transcription page at / and /transcribe

Models

Variable	Default
HF_HUB_CACHE	~/.cache/huggingface/hub (/app/models in the images)	model cache; models load straight from it
PARAKEET_MODEL_CATALOG	the built-in models.yaml	a catalog file that replaces it; see below
PARAKEET_PRELOAD_MODELS	empty	comma-separated model (fp32) or model:quantization to load and warm up before ready; requests must still name model
PARAKEET_MODEL_CACHE_SIZE	0	keep at most N loaded models, and separately N loaded aligners, evicting the least recently used; 0 is no limit
PARAKEET_MODEL_IDLE_TIMEOUT_SEC	21600 (6 h)	unload a model or aligner, preloaded or not, that no request has used for this many seconds; the next request that names it loads it again, without a warm-up. -1 keeps everything loaded
PARAKEET_HF_OFFLINE	false	never contact Hugging Face; every file must already be in the cache

Startup

Variable	Default
PARAKEET_WARMUP	true	run one synthetic chunk through each preloaded model before ready
PARAKEET_WARMUP_SEC	5	length of that chunk; 0 skips it
PARAKEET_WARMUP_TIMEOUT_SEC	120	a warm-up that fails or takes longer fails startup

Hardware and threads

Variable	Default
PARAKEET_USE_GPU	true	true requires CUDA, auto uses it when present, false runs on CPU
PARAKEET_GPU_DEVICE_ID	0	CUDA device
PARAKEET_GPU_MEMORY_LIMIT_MB	0	cap on ONNX Runtime's CUDA memory arena, in MiB; 0 is no cap
PARAKEET_GPU_CUDNN_ALGO_SEARCH	exhaustive	how cuDNN picks convolution kernels: exhaustive, heuristic or default; heuristic starts faster and uses less memory
PARAKEET_GPU_CUDNN_MAX_WORKSPACE	true	let cuDNN use its largest workspace; false saves VRAM, maybe at some speed
PARAKEET_GPU_ARENA_EXTEND_STRATEGY	next_power_of_two	how the CUDA arena grows; same_as_requested grows only by what is needed
PARAKEET_BATCHED	on, unless PARAKEET_USE_GPU=false	micro-batch requests together (GPU); off runs parallel single requests (CPU)
PARAKEET_MAX_BATCH_SIZE	4	largest micro-batch
PARAKEET_MAX_BATCH_AUDIO_SECONDS	300	most padded audio in one micro-batch: a batch pads every clip to its longest, so long chunks batch fewer at a time
PARAKEET_BATCH_WINDOW_MS	4	how long to wait to fill a micro-batch
PARAKEET_ORT_INTRA_THREADS	1 on GPU; physical cores on CPU	ONNX Runtime threads per inference
PARAKEET_ORT_INTER_THREADS	1	ONNX Runtime inter-op threads
PARAKEET_INFER_WORKERS	logical CPUs ÷ intra-op threads, at most 4	parallel inferences on CPU (PARAKEET_BATCHED off)
PARAKEET_AUDIO_WORKERS	physical cores, at most 8	audio decoding and chunking threads
PARAKEET_ALIGN_THREADS	physical cores, at most 4	word aligner threads on the CPU (int8, or any aligner without a GPU)
PARAKEET_ALIGN_GPU	true	run fp16 and fp32 word aligners on the GPU when the models use it; false keeps them all on the CPU

Core counts respect the affinity mask and the cgroup CPU quota; see Running under an orchestrator.

/health reports what each loaded model and aligner actually runs on under runtime, keyed name:quantization: its kind (model or aligner), backend (cuda or cpu), the providers each ONNX session bound to, and a fallback_reason when CUDA was asked for (PARAKEET_USE_GPU=auto) but ONNX Runtime used the CPU.

Chunking (long audio is cut at pauses; each model's chunk length is set in the catalog)

Variable	Default
PARAKEET_CHUNK_MIN_SEC	20	shortest chunk before neighbours are merged, unless merging would pass the model's chunk_max_sec less PARAKEET_CHUNK_CONTEXT_SEC on both sides (20 s for parakeet-v2) and either the next phrase fits a chunk of its own or cutting at the pause adds no chunk, as merging would cut inside speech
PARAKEET_CHUNK_TRIM_SILENCE_SEC	3	cut silences at least this long out of a chunk; the first and last chunks keep up to this much before and after the speech
PARAKEET_CHUNK_CONTEXT_SEC	5	Parakeet only: each chunk also decodes this much of its neighbours' audio either side, from and to a pause, and keeps only the words that start in its own range (see below); at most a quarter of the model's chunk_max_sec; 0 turns it off
PARAKEET_VAD	volume	how pauses are found: volume, frames quieter than a gate, or silero, a speech model, 30x slower or more (see below)
PARAKEET_VAD_GATE_DB	each file's own	volume's gate in dBFS, such as -45; unset, 0.4× the file's average 20 ms frame level (about 8 dB below it), never under -60 dBFS, and a long quiet stretch is heard again at its own level (see below)
PARAKEET_VAD_THRESHOLD	0.5	silero's speech probability
PARAKEET_VAD_MIN_SILENCE_MS	400	shortest pause to cut at
PARAKEET_VAD_SPEECH_PAD_MS	120	padding kept around speech
PARAKEET_VAD_FILTER	false	vad_filter for requests that don't send it: audio short enough to go to the model whole goes only if Silero hears speech in it
PARAKEET_VAD_FILTER_MIN_SPEECH_MS	0	the shortest speech vad_filter counts; longer drops more short words and fewer noise clips (see Silence)

Context at the cuts. Parakeet makes up a word (and, the, I) when its input ends shortly after speech. Chunks cut in a pause with no audio past the cut gained one at about one cut in five. So each chunk now also decodes some of its neighbours' audio, and keeps only the words that start in its own range. Each chunk times a word on its own 80 ms grid, so one starting at a cut could start before it in one chunk and after it in the other; near a cut the two chunks' words are matched up instead, and each word both heard is kept by one of them, so once. Both ends of what it decodes go in a pause, because a chunk that starts or ends inside speech can make Parakeet skip or drop tens of seconds of words: the nearest pause at least PARAKEET_CHUNK_CONTEXT_SEC from the cut, else the farthest that fits, or, with none, PARAKEET_CHUNK_CONTEXT_SEC from the cut. All of it fits inside the model's chunk_max_sec, so a chunk's own range is shorter by twice the context: parakeet-v3 still cuts at about 60 s (at most 65 s), parakeet-v2 at 20 s instead of 25 s. That decodes up to 1.25× the audio on parakeet-v3 and 1.5× on parakeet-v2. Long silences are still cut out, with no context across them. Whisper gets no context: it returns no word times to trim it back by.

Skipped speech. Parakeet can still skip a stretch of speech, which one depending on where its input starts and ends: a narrated opening over a music bed was lost in most clips of 70 s or more, single chunks among them, and heard in every clip of 68 s or less tried. Where a chunk has 3 s or more of speech (to VAD) with no word, a chunk with context is first decoded again as just its own range, without context, kept if it hears two or more words in what was skipped and no fewer in all. Then each stretch still skipped, in any chunk or a clip short enough to be one, is decoded again on its own, with 2 s either side (stretches that close as one decode), and the words heard where the chunk heard none are put in among its own, if two or more are in what it skipped: to VAD, music or noise can be speech, and one word may be made up. The chunk's words and the redo's are matched up word by word, as at a cut, and the redo's are put in only between words both heard either side of what was skipped, never where the chunk heard the speech as another word or split it differently, and never the redo's last word, which it may have made up as its input ended. Every word the chunk heard stays. Each such stretch costs one more decode, at most 4 s longer than it. On a clip short enough to be one chunk, VAD runs only once it has 3 s or more with no word. A redo that fails keeps what was heard.

Made-up last words. Where audio ends in a pause, Parakeet can add a word that was never said: a clip ending half a second after "he muttered" came back as "he muttered, and" (#110). PARAKEET_CHECK_ENDINGS=true turns on a check that drops such words; it is off by default, because it can still lose a real word now and then (below). Where the input ends decides it, so the same audio a little shorter, or with its pause a little longer, hears no such word, while a word that was said is heard either way. VAD can't tell them apart: the model times a made-up word inside the speech before the pause as often as after it. So the end of a clip, and of each chunk that ends at the end of the audio or at a long silence cut out, is decoded twice more (its last 8 s, or from 1 s before the word ahead of the checked ones): 0.2 s shorter, and with its last 0.3 s played again backwards. Words starting in the last 0.75 s that neither hears, after the last word both do, are dropped. Both must hear the word before them, as the same word, one like it ("Lori" for "Lorry"), split in two, joined to the word before ("dot org." as "LibriVox.org."), or any number for a number ("three" for "III"): Parakeet sometimes returns nothing at all for a clip, depending on where it ends, and that is no sign. A word stays, too, where either decode hears another word starting with it (from 0.1 s before to 0.15 s after): that is speech heard another way ("bit" as "bed."), not a made-up word. Audio that ends inside speech, its last 100 ms within 15 dB of its speech, is not checked: cut in its last word, both decodes lose that word too. A word starting more than 0.75 s before the end is never dropped: every made-up word seen started in the decoder's last 8 frames (0.65 s), and real words both decodes missed started earlier ("¿Sí?", "Abschnitt 1."). Nor is a word said on its own: where, after the word before it, the audio has a pause of 0.25 s (25 dB under its speech) and then 60 ms within 15 dB of it, the words stay ("patrão. 4.", "vriendje, Gij."). A made-up word has no sound of its own. A clip that ends within about 0.1 s of the next word's onset can lose that word ("Pretty, madam, said" becomes "Pretty, madam,"), which the model guessed from its first sound.

Of 1,697 clips of 3.5-12 s from LibriVox's Alice and Sherlock Holmes, each ending between 0.2 s before and 1.5 s after the speech before a pause, the check dropped 44 words from 38 clips (and, but, that, for, once "and the wrong way."), each one never said, and no word that was. Main had made up a word at the end of 13 more of them, which the check keeps, and in one more it keeps words main heard over speech it misheard. On 1,092 clips from six other chapters (English plays, Dutch, Italian, Portuguese, German), a version without the said-alone rule dropped 23 made-up words and 4 real ones, each a short word said alone after a pause, which the rule now keeps; that rule has not yet been tried on audio it was not tuned on. It costs two decodes of up to 8 s per clip, or per such chunk, and none where the last word starts more than 0.75 s before the end or the audio ends in speech. On 4 CPU cores with parakeet-v3:int8, 200 of those clips took a median 0.82 s each instead of 0.55 s; whole chapters took as long as before. If the decodes fail, every word stays.

Finding pauses. By default a pause is any stretch of 20 ms frames quieter than the gate, which takes under 0.3 ms of CPU per second of audio. PARAKEET_VAD=silero asks Silero-VAD instead, at about 10 ms per second, all of it before the first chunk reaches the model: some 10 minutes for a 17-hour audiobook. Measured on LibriVox narration with parakeet-v3:int8, as word errors against the book's text:

Audio	silero	volume	volume, gate -45
The Adventures of Sherlock Holmes, 15 min (2,769 words)	107	102	103
Alice's Adventures in Wonderland, chapter 3 (1,758)	144	139	135
The War of the Worlds, chapter 1 (2,292)	167	140	143
the Holmes clip under pink noise at -40 dBFS	107	109	119
the Holmes clip under pink noise at -35 dBFS	111	116	116

Silero's run of The War of the Worlds lost a sentence that volume's kept; where chunks fall decides that, so take it as luck rather than an advantage. Volume needs pauses quieter than the speech. Under a noise bed the file's own gate rises with the noise: at -40 dBFS it still found every pause, at -35 dBFS (5 dB under the speech) it missed some and 10 chunks were cut mid-speech. A fixed gate below the noise finds no pause at all, and the whole file is cut mid-speech, into the fewest chunks that fit. Music hasn't been measured. A DC offset is taken out before a frame's level is measured, so it sets no floor under the pauses: The Adventures of Sherlock Holmes, chapter 3, decodes at +0.094 (-20.5 dBFS), and counting it, volume found one span of speech 45 minutes long. The offset is the median frame mean over about 1 s, so it follows a slow drift or a step partway through. It is none under -60 dBFS, and no frame reads louder for it, so a file without one is measured nearly as before. Such a file is now cut as it would be without the offset, so a quiet word alone in a long pause is lost as it is on a clean file, where before the whole file was decoded and cut by length. Known differences: on content under about 2 Hz, such as an unfiltered warped disc, pauses can be found differently than before; where the offset steps just before a quiet passage, the step reads as a sound and the quiet passage can be lost; and very quiet speech (around -55 dBFS) with an offset of about the same size can lose a word. The model still hears the audio as it is.

Speech with no pause. A stretch of speech too long for one chunk, with no pause in it as long as PARAKEET_VAD_MIN_SILENCE_MS, is still cut into the fewest chunks that fit, as evenly as before, but a cut that falls inside a word moves, within 3 s (and an eighth of the chunk), to a quieter point: one 3 dB under where it was, over 200 ms. A cut that already looks like a gap stays where it is: one 3 dB under the speech around it, or one in a stretch of 40 ms or more that is 1.5 dB under the loudest speech on each side (within 1.2 s), and moves from there only to a pause far deeper (13 dB under that gap, 17 dB under the speech). That second test catches a short gap the cut sits at the edge of, and a gap beside a quieter word or a fricative. Loudness can't always tell a gap from a quieter word or a fricative, so a cut that moves can still land in one. A split chunk may give the pause it shares with the chunk beside it to that chunk, so a cut can reach a gap, as long as that chunk still needs no more pieces. With no context (Whisper's bounds, or PARAKEET_CHUNK_CONTEXT_SEC=0) a quiet word VAD missed in that pause can then be split where the moved end lands. The first chunk's lead and the last one's tail, which nothing else decodes, keep all of their margin, as before.

How much this helps varies by recording, and by the room the cuts have. scripts/quiet_cuts_real.py measures it on a recording against word times (for one stretch cut only by length, set PARAKEET_VAD_GATE_DB under a noise bed); scripts/quiet_cuts_fuzz.py measures it on synthetic speech. Where gaps between words are no quieter than a quieter word or a fricative, loudness can't find them, and cuts land in words about as often as an even split's.

Quieter speakers. The file's own gate follows its average, so a speaker far quieter than the rest (a remote guest, a phone leg, a question from the audience) can sit under it for a whole turn, which would then be cut out as a long silence and never decoded. So a quiet stretch at least PARAKEET_CHUNK_TRIM_SILENCE_SEC long is heard again at its own level: 100 ms of it louder than 0.4× its own average, and 10 dB over its quietest tenth, is loud. The 100 ms is a median, so no click or knock passes, and room tone stays within a few dB of its floor. A loud sound (joined across dips under 400 ms, or under PARAKEET_VAD_MIN_SILENCE_MS where that is longer) is speech if half a second of it is loud, and so is any shorter one reaching within PARAKEET_CHUNK_TRIM_SILENCE_SEC of such a sound, kept whole up to half a second past that: a quiet speaker's short words, which would otherwise leave a stretch to cut out between two phrases. Breaths alone in a pause, further apart than those dips, never add up to speech, however many; breaths closer together (panting), footsteps or typing can join into a sound long enough. On LibriVox narration a long pause's breaths and rustles were 0.1-0.34 s (decoded alone, one made Parakeet say "yeah"); 26 dB down, a quieter reader's phrases were 0.58-1.18 s and the words between them 0.12-0.32 s. With a 15 s turn (25 words) of one LibriVox reader between two minutes of another:

Quiet turn	words heard before	now
14 dB quieter	25	25
20 dB quieter	6, and an "Oh." not said	25
26 dB quieter	0	25

On other recordings of the three chapters above (not the table's), and the two noise beds, the chunks, and so the transcripts, are the same as before. Still taken for a pause: a sound under half a second with no longer one near it (a quiet "Yes." alone in a long pause, or a halting quiet speaker's words when each comes alone), a quiet stretch that never rises 10 dB over its own floor (a steady tone, or speech under noise within 10 dB of it), and all under a fixed PARAKEET_VAD_GATE_DB, which is never heard again.

Words and numbers

Variable	Default
PARAKEET_ALIGN_DEFAULT_LANGUAGE	en	language assumed for word timestamps and spoken numbers when a request sends none; empty uses them only when language is sent
PARAKEET_SPOKEN_NUMBERS	false	spoken numbers for requests that don't send spoken_numbers
PARAKEET_SPOKEN_NUMBERS_ALIGNER	wav2vec2-base-960h	the aligner, name or name:quantization, that hears how each number was said when a request names none; empty gives fixed readings without loading one
PARAKEET_RETIME_WORDS	false	retime_words for requests that don't send it: move Parakeet's word times out of pauses
PARAKEET_CHECK_ENDINGS	false	decode the end of each clip twice more to drop words Parakeet makes up where the audio ends in a pause; it can still lose a real word now and then (see Made-up last words above)

Request limits

A request over a limit is rejected with 413.

Variable	Default
PARAKEET_MAX_UPLOAD_BYTES	256 MiB	one uploaded file
PARAKEET_MAX_AUDIO_SECONDS	7200 (2 h)	one file's decoded duration
PARAKEET_MAX_REQUEST_CHUNKS	512	chunks one request may produce
PARAKEET_ALIGN_MAX_AUDIO_SECONDS	60	one file's decoded duration for forced alignment
PARAKEET_MAX_BATCH_FILES	16	files in one batch request
PARAKEET_MAX_BATCH_BYTES	512 MiB	total bytes in one batch request
PARAKEET_FFMPEG_TIMEOUT_SEC	180	per-request FFmpeg decode time (not a 413)

Your own model catalog

The models and aligners are defined in parakeet_service/models.yaml. Each model has its family, languages and chunk lengths, and per precision a Hugging Face repo, a pinned commit and the files that differ from the defaults:

models:
  parakeet-v2:
    family: parakeet
    onnx_asr_type: nemo-conformer-tdt
    languages: *english            # from the file's `languages` section
    chunk_target_sec: 25.0
    chunk_max_sec: 30.0
    quantizations:
      int8:
        repo: istupakov/parakeet-tdt-0.6b-v2-onnx
        revision: "0bbb45a3365852604aef28b538a8f066f4ccaa85"
        files:
          encoder-model.onnx: encoder-model.int8.onnx
          decoder_joint-model.onnx: decoder_joint-model.int8.onnx

The aligners section lists the word aligners the same way, plus the languages each aligns and how to spell a transcript for it (aligners: {} serves none). A quantization marked cpu_only: true stays on the CPU on a GPU server; the built-in catalog marks the int8 ones. To serve a different set without rebuilding the image, point PARAKEET_MODEL_CATALOG at another file of the same shape. It replaces the built-in catalog, so copy the built-in file and edit it. The service checks it at startup and refuses to start on a mistake, naming it; it is read only then, so restart after changing it.

On Kubernetes, keep it in a ConfigMap:

kubectl create configmap parakeet-models --from-file=models.yaml=parakeet_service/models.yaml

# in the Deployment's pod spec
containers:
  - name: parakeet
    env:
      - name: PARAKEET_MODEL_CATALOG
        value: /config/models.yaml
    volumeMounts:
      - name: model-catalog
        mountPath: /config
volumes:
  - name: model-catalog
    configMap:
      name: parakeet-models

Running under an orchestrator

CPU limits are quotas, not cpusets. A Kubernetes resources.limits.cpu is invisible to sched_getaffinity() and psutil, which report the node's full core count, so without care a 4-core pod on a 64-core node would start 64 ONNX Runtime threads. Thread pools are sized from the cgroup quota instead, read from the process's own cgroup and its ancestors, so it is found under systemd CPUQuota= and --cgroupns=host as well. /health reports cgroup_quota next to the detected core counts under cpu; set PARAKEET_ORT_INTRA_THREADS to override.

Cold start. Nothing loads at startup unless it is listed in PARAKEET_PRELOAD_MODELS. Each listed model is then warmed up before /healthz answers 200, so the first real request doesn't pay ONNX Runtime's setup. A warm-up that fails or exceeds PARAKEET_WARMUP_TIMEOUT_SEC fails startup rather than reporting a replica ready that can't run inference. A preloaded model unloaded after PARAKEET_MODEL_IDLE_TIMEOUT_SEC unused is loaded again by the next request that names it, which pays for the load and the setup the warm-up hid; /healthz stays 200 meanwhile. The healthcheck start_period in the Dockerfiles and docker-compose.yml allows for model load plus that timeout; raise both together. With a pre-seeded cache, set PARAKEET_HF_OFFLINE=true to skip the Hugging Face revision check each start makes.

Performance

Workload	CPU: i7-12700KF, int8	GPU: RTX 3090, fp32
One 300 s file	10.41 s (27.2× real time)	1.37 s (205.9×)
16 × 10 s files at once	39.3× throughput	200.3× throughput

Measured in 1.x, with the parakeet-v3 exports 2.0 replaced. The speed comes from cutting long audio at pauses with Silero-VAD and running the chunks in parallel (micro-batched on GPU), decoding each upload once (16 kHz PCM WAV without FFmpeg at all), and sizing every thread pool to the CPUs actually available. OPTIMIZATION.md has the method, what didn't work, the GPU sweeps, and accuracy benchmarks against Whisper.

Acknowledgments

This project stands on the shoulders of giants and wouldn't be possible without:

• Shadowfita - For the original FastAPI implementation that served as the foundation for this project
• NVIDIA - For developing and open-sourcing the exceptional Parakeet TDT model family
• groxaxo - The mastermind behind this project, bringing together ONNX optimization, multilingual support, and seamless OpenAI API compatibility

Thank you to all contributors and the open-source community for making high-performance, local speech recognition accessible to everyone!