README.md
11.7 KB · 365 lines · markdown Raw
1 ---
2 license: cc-by-4.0
3 base_model: moondream/parakeet-ultra
4 base_model_relation: quantized
5 library_name: transcribe.cpp
6 pipeline_tag: automatic-speech-recognition
7 language:
8 - en
9 - de
10 - fr
11 - es
12 - it
13 - pt
14 - ru
15 - uk
16 - hr
17 - sl
18 - lv
19 - lt
20 - et
21 - fi
22 - sv
23 - da
24 - nl
25 - pl
26 - cs
27 - sk
28 - hu
29 - ro
30 - bg
31 - el
32 - mt
33 tags:
34 - gguf
35 - transcribe.cpp
36 - asr
37 - speech-to-text
38 - parakeet
39 - conformer
40 - tdt
41 - multilingual
42 - long-form
43 transcribe_cpp:
44 schema_version: 2
45 wer_librispeech_test_clean:
46 f32: 1.8
47 f16: 1.8
48 q8_0: 1.8
49 q6_k: 1.81
50 q5_k_m: 1.81
51 q4_k_m: 1.82
52 wer_fleurs_en:
53 q8_0: 4.26
54 wer_fleurs_de:
55 q8_0: 4.58
56 wer_fleurs_fr:
57 q8_0: 4.66
58 wer_fleurs_es:
59 q8_0: 3.06
60 wer_fleurs_it:
61 q8_0: 2.34
62 wer_fleurs_pt:
63 q8_0: 4.16
64 wer_fleurs_ru:
65 q8_0: 5.63
66 wer_fleurs_uk:
67 q8_0: 5.71
68 wer_fleurs_hr:
69 q8_0: 11.3
70 wer_fleurs_sl:
71 q8_0: 18.98
72 wer_fleurs_lv:
73 q8_0: 18.04
74 wer_fleurs_lt:
75 q8_0: 17.06
76 wer_fleurs_et:
77 q8_0: 14.44
78 wer_fleurs_fi:
79 q8_0: 11.14
80 wer_fleurs_sv:
81 q8_0: 12.83
82 wer_fleurs_da:
83 q8_0: 15.98
84 wer_fleurs_nl:
85 q8_0: 6.7
86 wer_fleurs_pl:
87 q8_0: 6.16
88 wer_fleurs_cs:
89 q8_0: 11.04
90 wer_fleurs_sk:
91 q8_0: 7.83
92 wer_fleurs_hu:
93 q8_0: 13.09
94 wer_fleurs_ro:
95 q8_0: 10.12
96 wer_fleurs_bg:
97 q8_0: 11.04
98 wer_fleurs_el:
99 q8_0: 32.25
100 wer_fleurs_mt:
101 q8_0: 17.01
102 rtf_m4_max:
103 cpu: 35.03
104 metal: 189.12
105 rtf_ryzen_4750u:
106 cpu: 13.29
107 vulkan: 23.0
108 streaming: false
109 translate: false
110 lang_detect: true
111 timestamps: token
112 ---
113
114 # parakeet-ultra: transcribe.cpp GGUF
115
116 GGUF conversions of [moondream/parakeet-ultra](https://huggingface.co/moondream/parakeet-ultra) for use
117 with [transcribe.cpp](https://github.com/handy-computer/transcribe.cpp).
118
119 Ported from upstream commit
120 [73175eb](https://huggingface.co/moondream/parakeet-ultra/commit/73175eb),
121 pinned 2026-10-04.
122 Validated against the kestrel 0.9.1 (Photon) reference at transcribe.cpp commit
123 [08f9e6f0](https://github.com/handy-computer/transcribe.cpp/tree/08f9e6f0)
124 on 2026-10-05.
125
126 Moondream's post-trained parakeet-tdt-0.6b-v3. Offline multilingual
127 speech-to-text over the same 25 European languages, with lower error rates
128 than v3 on English and on every FLEURS language we measured. A FastConformer
129 encoder with a TDT transducer decoder, plus a small voice-activity head that
130 the model uses to cut long recordings at pauses into segments of at most
131 30 seconds. Takes 16 kHz mono WAV and produces a punctuated, cased
132 transcript with optional token-level timestamps. A drop-in replacement for
133 v3: same size, languages and API. Not a streaming model and does not
134 translate.
135
136
137 ## Downloads
138
139 | Quantization | Download | Size | WER (LibriSpeech test-clean) |
140 | --- | --- | ---: | ---: |
141 | F32 | [parakeet-ultra-F32.gguf](https://huggingface.co/handy-computer/parakeet-ultra-gguf/resolve/main/parakeet-ultra-F32.gguf) | 2.51 GB | 1.80% |
142 | F16 | [parakeet-ultra-F16.gguf](https://huggingface.co/handy-computer/parakeet-ultra-gguf/resolve/main/parakeet-ultra-F16.gguf) | 1.26 GB | 1.80% |
143 | Q8_0 | [parakeet-ultra-Q8_0.gguf](https://huggingface.co/handy-computer/parakeet-ultra-gguf/resolve/main/parakeet-ultra-Q8_0.gguf) | 740 MB | 1.80% |
144 | Q6_K | [parakeet-ultra-Q6_K.gguf](https://huggingface.co/handy-computer/parakeet-ultra-gguf/resolve/main/parakeet-ultra-Q6_K.gguf) | 611 MB | 1.81% |
145 | Q5_K_M | [parakeet-ultra-Q5_K_M.gguf](https://huggingface.co/handy-computer/parakeet-ultra-gguf/resolve/main/parakeet-ultra-Q5_K_M.gguf) | 550 MB | 1.81% |
146 | Q4_K_M | [parakeet-ultra-Q4_K_M.gguf](https://huggingface.co/handy-computer/parakeet-ultra-gguf/resolve/main/parakeet-ultra-Q4_K_M.gguf) | 486 MB | 1.82% |
147
148 WER on the full LibriSpeech test-clean split (2,620 utterances), batch size 8, timestamps none, language hint `en`, decoded on cuda. Measured at transcribe.cpp `021aa17d`.
149
150 Greedy transducer decoding, no external LM. The reference is Moondream's
151 own runtime (kestrel 0.9.1, the engine behind Photon), not transformers.
152 On this manifest kestrel scores 1.80% and the C++ F32 build matches it.
153
154 Moondream's card reports 1.41% on the same split, scored with the Open
155 ASR Leaderboard normalizer.
156
157 Long-form: on 11 full TED-LIUM 3 talks, segmented by the model's
158 voice-activity head exactly as kestrel does it, F32 scores 2.22%, identical
159 to kestrel, and every quant stays within 0.05pp.
160
161
162 ## Usage
163
164 Build transcribe.cpp from source:
165
166 ```bash
167 git clone git@github.com:handy-computer/transcribe.cpp.git
168 cd transcribe.cpp
169 cmake -B build && cmake --build build
170 ```
171
172 Run on a 16 kHz mono WAV:
173
174 ```bash
175 build/bin/transcribe-cli \
176 -m parakeet-ultra-Q8_0.gguf \
177 input.wav
178 ```
179
180 If your audio isn't already 16 kHz mono WAV, convert it first:
181
182 ```bash
183 ffmpeg -i input.mp3 -ar 16000 -ac 1 output.wav
184 ```
185
186 See the [transcribe.cpp model page](https://github.com/handy-computer/transcribe.cpp/blob/main/docs/models/parakeet-ultra.md) for performance
187 numbers, numerical validation, and reproduction steps.
188
189 ## License
190
191 Inherited from the base model: **CC-BY-4.0**. See the
192 [upstream model card](https://huggingface.co/moondream/parakeet-ultra) for full terms.
193 This repository includes the license text
194 ([`LICENSE`](https://huggingface.co/handy-computer/parakeet-ultra-gguf/blob/main/LICENSE)) and a
195 [`NOTICE`](https://huggingface.co/handy-computer/parakeet-ultra-gguf/blob/main/NOTICE) recording
196 how these files were derived from the original.
197
198 ---
199
200 ## Original Model Card
201
202 > The section below is reproduced from
203 > [moondream/parakeet-ultra](https://huggingface.co/moondream/parakeet-ultra) at commit
204 > `73175eb` for offline reference. The upstream card is the
205 > authoritative source.
206
207 # Moondream Parakeet Ultra
208
209 A post-trained version of [parakeet-tdt-0.6b-v3](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3). Same
210 architecture, same tokenizer, same 0.6B parameters in full precision. It is better than the original on every
211 benchmark group below, by a wide margin on the 25-language FLEURS set, in background noise and on long-form audio,
212 and it runs faster than the original does in NeMo.
213
214 | | parakeet-tdt-0.6b-v3 | parakeet-ultra |
215 |---|---|---|
216 | Open ASR Leaderboard, 7 English sets (WER %) | 6.26 | **5.80** |
217 | FLEURS, 25 languages (WER %) | 11.62 | **9.55** |
218 | Business speech, AA-WER style (WER %) | 6.15 | **5.79** |
219 | Background noise, 9 MUSAN conditions (WER %) | 6.72 | **5.82** |
220 | TED-LIUM long-form (WER %) | 2.71 | **1.94** |
221
222 Its sibling [Parakeet Redux](https://huggingface.co/moondream/parakeet-redux) is the ternary version of the same
223 architecture, 178 MB, built for CPUs and Apple silicon. Read the
224 [release post](https://moondream.ai/blog/introducing-parakeet-redux-and-ultra) for the story behind both models.
225
226 ## Usage
227
228 Run it with [Photon](https://moondream.ai/photon). Every accuracy number on this page is Photon on an NVIDIA GPU.
229
230 ```python
231 # pip install moondream
232 import moondream as md
233
234 with md.photon("moondream/parakeet-ultra") as speech:
235 result = speech.transcribe(audio="speech.wav")
236 print(result["text"])
237
238 # timestamps="segment": one entry per sentence, with its start
239 # and end in seconds; "word" adds the start and end of every word
240 result = speech.transcribe(audio="speech.wav", timestamps="word")
241 for segment in result["segments"]:
242 print(segment["start"], segment["end"], segment["text"])
243 for word in segment["words"]:
244 print(word["start"], word["end"], word["word"])
245 ```
246
247 ## Performance
248
249 Real-time factor: seconds of audio transcribed per second of wall clock, higher is faster. One NVIDIA B200, 128
250 requests in flight, the same files on both sides; NeMo 3.0 runs the original checkpoint with its own transcribe
251 call at batch 128, Photon runs parakeet-ultra.
252
253 | | NeMo, parakeet-tdt-0.6b-v3 | Photon, parakeet-ultra |
254 |---|---|---|
255 | LibriSpeech test-clean, 2,620 utterances, 5.4 hours | 6,005× | **9,743×** |
256 | AMI test, 12,643 utterances, 8.7 hours | 4,394× | **6,688×** |
257
258 ## Benchmarks
259
260 Word error rate in percent, lower is better. Both models are scored on the same files with the Open ASR
261 Leaderboard's own pipeline as of September 2026 (its normalizers and compound-merging alignment, with the FLEURS
262 references prepared as the leaderboard's text column is). parakeet-ultra runs in Photon on an NVIDIA GPU, the
263 original in NeMo in bf16.
264
265 ### Open ASR Leaderboard
266
267 The seven English test sets of the Hugging Face Open ASR Leaderboard: audiobooks (LibriSpeech), meetings (AMI),
268 earnings calls (Earnings-22), podcasts and YouTube (GigaSpeech), financial calls (SPGISpeech) and parliament
269 (VoxPopuli).
270
271 | set | parakeet-tdt-0.6b-v3 | parakeet-ultra |
272 |---|---|---|
273 | LibriSpeech test-clean | 1.52 | 1.41 |
274 | LibriSpeech test-other | 3.13 | 2.98 |
275 | AMI | 10.86 | 9.77 |
276 | Earnings-22 | 10.75 | 9.76 |
277 | GigaSpeech | 8.05 | 7.71 |
278 | SPGISpeech | 3.63 | 3.34 |
279 | VoxPopuli | 5.88 | 5.65 |
280 | **average** | **6.26** | **5.80** |
281
282 ### FLEURS
283
284 The FLEURS test split: read Wikipedia sentences, a few hundred per language, for all 25 languages the model
285 supports.
286
287 | language | parakeet-tdt-0.6b-v3 | parakeet-ultra |
288 |---|---|---|
289 | Bulgarian | 11.90 | 10.09 |
290 | Croatian | 10.93 | 9.65 |
291 | Czech | 10.85 | 9.97 |
292 | Danish | 16.78 | 14.31 |
293 | Dutch | 6.18 | 5.46 |
294 | English | 4.25 | 3.55 |
295 | Estonian | 13.23 | 9.69 |
296 | Finnish | 11.05 | 9.19 |
297 | French | 4.81 | 4.32 |
298 | German | 4.13 | 3.61 |
299 | Greek | 35.71 | 32.25 |
300 | Hungarian | 13.65 | 10.76 |
301 | Italian | 2.61 | 2.00 |
302 | Latvian | 21.38 | 15.95 |
303 | Lithuanian | 21.09 | 16.36 |
304 | Maltese | 19.13 | 14.92 |
305 | Polish | 6.70 | 5.54 |
306 | Portuguese | 4.65 | 3.96 |
307 | Romanian | 11.54 | 9.18 |
308 | Russian | 5.91 | 5.21 |
309 | Slovak | 9.46 | 7.03 |
310 | Slovene | 21.76 | 16.64 |
311 | Spanish | 3.12 | 2.72 |
312 | Swedish | 13.75 | 11.57 |
313 | Ukrainian | 5.94 | 4.75 |
314 | **average** | **11.62** | **9.55** |
315
316 ### Business speech (AA-WER style)
317
318 The AMI and VoxPopuli test sets with the Artificial Analysis cleaning applied, and Earnings-22 scored in 30-second
319 chunks joined per call, the way the AA-WER benchmark does it.
320
321 | set | parakeet-tdt-0.6b-v3 | parakeet-ultra |
322 |---|---|---|
323 | AMI (cleaned) | 9.52 | 8.48 |
324 | VoxPopuli (cleaned) | 3.02 | 3.10 |
325 | Earnings-22, 30-second chunks | 5.90 | 5.78 |
326 | **average** | **6.15** | **5.79** |
327
328 ### Background noise
329
330 The clean sets with MUSAN noise (the half of the corpus not used for training) mixed in at a fixed signal-to-noise
331 ratio; 0 dB means the noise is as loud as the speech.
332
333 | set | parakeet-tdt-0.6b-v3 | parakeet-ultra |
334 |---|---|---|
335 | LibriSpeech test-other, 10 dB | 4.12 | 3.87 |
336 | LibriSpeech test-other, 5 dB | 5.49 | 5.10 |
337 | LibriSpeech test-other, 0 dB | 9.06 | 8.31 |
338 | FLEURS German, 10 dB | 5.78 | 4.98 |
339 | FLEURS German, 5 dB | 8.07 | 6.91 |
340 | FLEURS German, 0 dB | 14.45 | 12.08 |
341 | FLEURS Spanish, 10 dB | 3.99 | 3.13 |
342 | FLEURS Spanish, 5 dB | 4.12 | 3.38 |
343 | FLEURS Spanish, 0 dB | 5.44 | 4.64 |
344 | **average** | **6.72** | **5.82** |
345
346 ### Long-form
347
348 Eleven complete TED-LIUM 3 talks of 10-20 minutes each. parakeet-ultra transcribes them through Photon, whose
349 segmenter cuts each talk at pauses found by the model's VAD head into segments of at most 30 seconds; the original
350 runs NeMo's own long-audio path.
351
352 | set | parakeet-tdt-0.6b-v3 | parakeet-ultra |
353 |---|---|---|
354 | TED-LIUM 3, 11 full talks of 10-20 minutes | 2.71 | 1.94 |
355
356 ## Notes
357
358 - Based on [parakeet-tdt-0.6b-v3](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3) by NVIDIA. Languages,
359 tokenizer and output conventions (punctuation, casing, numerals) are the original's.
360 - Runs with [Photon](https://moondream.ai/photon).
361 - Long audio is segmented by the model itself: the weights carry a small voice-activity head on the encoder's
362 subsampler, and Photon uses it to cut recordings at pauses into segments of at most 30 seconds. No external VAD
363 model is needed.
364 - License is CC-BY-4.0, same as the original.
365