README.md
61.5 KB · 1600 lines · markdown Raw
1 ---
2 license: apache-2.0
3 base_model: Qwen/Qwen3-ASR-1.7B
4 base_model_relation: quantized
5 library_name: transcribe.cpp
6 pipeline_tag: automatic-speech-recognition
7 language:
8 - zh
9 - en
10 - yue
11 - ar
12 - de
13 - fr
14 - es
15 - pt
16 - id
17 - it
18 - ko
19 - ru
20 - th
21 - vi
22 - ja
23 - tr
24 - hi
25 - ms
26 - nl
27 - sv
28 - da
29 - fi
30 - pl
31 - cs
32 - fil
33 - fa
34 - el
35 - ro
36 - hu
37 - mk
38 tags:
39 - gguf
40 - transcribe.cpp
41 - asr
42 - speech-to-text
43 - qwen3
44 - qwen3-asr
45 - multilingual
46 transcribe_cpp:
47 schema_version: 2
48 wer_fleurs_ar:
49 q8_0: 14.91
50 wer_fleurs_cs:
51 q8_0: 22.97
52 wer_fleurs_da:
53 q8_0: 21.23
54 wer_fleurs_de:
55 q8_0: 4.25
56 wer_fleurs_el:
57 q8_0: 29.22
58 wer_fleurs_en:
59 q8_0: 3.23
60 wer_fleurs_es:
61 q8_0: 3.31
62 wer_fleurs_fa:
63 q8_0: 28.29
64 wer_fleurs_fi:
65 q8_0: 25.48
66 wer_fleurs_fil:
67 q8_0: 24.29
68 wer_fleurs_fr:
69 q8_0: 4.52
70 wer_fleurs_hi:
71 q8_0: 7.84
72 wer_fleurs_hu:
73 q8_0: 32.84
74 wer_fleurs_id:
75 q8_0: 5.37
76 wer_fleurs_it:
77 q8_0: 2.68
78 cer_fleurs_ja:
79 q8_0: 5.29
80 cer_fleurs_ko:
81 q8_0: 4.6
82 wer_fleurs_mk:
83 q8_0: 18.22
84 wer_fleurs_ms:
85 q8_0: 10.42
86 wer_fleurs_nl:
87 q8_0: 7.43
88 wer_fleurs_pl:
89 q8_0: 12.5
90 wer_fleurs_pt:
91 q8_0: 4.37
92 wer_fleurs_ro:
93 q8_0: 20.46
94 wer_fleurs_ru:
95 q8_0: 6.25
96 wer_fleurs_sv:
97 q8_0: 19.68
98 cer_fleurs_th:
99 q8_0: 6.89
100 wer_fleurs_tr:
101 q8_0: 9.46
102 wer_fleurs_vi:
103 q8_0: 6.15
104 cer_fleurs_yue:
105 q8_0: 6.13
106 cer_fleurs_zh:
107 q8_0: 7.14
108 wer_librispeech_test_clean:
109 bf16: 1.62
110 f16: 1.62
111 q8_0: 1.62
112 q6_k: 1.65
113 q5_k_m: 1.65
114 q4_k_m: 1.81
115 rtf_m4_max:
116 cpu: 9.59
117 metal: 41.08
118 rtf_ryzen_4750u:
119 cpu: 2.47
120 vulkan: 4.08
121 streaming: false
122 translate: false
123 lang_detect: true
124 timestamps: none
125 ---
126
127 # Qwen3-ASR-1.7B: transcribe.cpp GGUF
128
129 GGUF conversions of [Qwen/Qwen3-ASR-1.7B](https://huggingface.co/Qwen/Qwen3-ASR-1.7B) for use
130 with [transcribe.cpp](https://github.com/handy-computer/transcribe.cpp).
131
132 Ported from upstream commit
133 [7278e1e](https://huggingface.co/Qwen/Qwen3-ASR-1.7B/commit/7278e1e),
134 pinned 2026-04-19.
135 Validated against the qwen_asr 0.0.6 reference at transcribe.cpp commit
136 [3f61df7](https://github.com/handy-computer/transcribe.cpp/tree/3f61df7)
137 on 2026-04-20.
138
139 Offline multilingual speech-to-text. Same audio-LLM architecture as the
140 0.6B variant (bidirectional audio encoder feeding a Qwen3 causal LM with
141 audio-token injection), wider: encoder `d_model=1024` (16 heads), LM
142 `hidden_size=2048`, `intermediate_size=6144`. Auto-detects the audio's
143 language across 30 languages and emits the transcript in that language.
144 Takes a 16 kHz mono WAV and produces a transcript.
145
146
147 ## Downloads
148
149 | Quantization | Download | Size | WER (LibriSpeech test-clean) |
150 | --- | --- | ---: | ---: |
151 | BF16 | [Qwen3-ASR-1.7B-BF16.gguf](https://huggingface.co/handy-computer/Qwen3-ASR-1.7B-gguf/resolve/main/Qwen3-ASR-1.7B-BF16.gguf) | 4.08 GB | 1.62% |
152 | F16 | [Qwen3-ASR-1.7B-F16.gguf](https://huggingface.co/handy-computer/Qwen3-ASR-1.7B-gguf/resolve/main/Qwen3-ASR-1.7B-F16.gguf) | 4.09 GB | 1.62% |
153 | Q8_0 | [Qwen3-ASR-1.7B-Q8_0.gguf](https://huggingface.co/handy-computer/Qwen3-ASR-1.7B-gguf/resolve/main/Qwen3-ASR-1.7B-Q8_0.gguf) | 2.19 GB | 1.62% |
154 | Q6_K | [Qwen3-ASR-1.7B-Q6_K.gguf](https://huggingface.co/handy-computer/Qwen3-ASR-1.7B-gguf/resolve/main/Qwen3-ASR-1.7B-Q6_K.gguf) | 1.69 GB | 1.65% |
155 | Q5_K_M | [Qwen3-ASR-1.7B-Q5_K_M.gguf](https://huggingface.co/handy-computer/Qwen3-ASR-1.7B-gguf/resolve/main/Qwen3-ASR-1.7B-Q5_K_M.gguf) | 1.52 GB | 1.65% |
156 | Q4_K_M | [Qwen3-ASR-1.7B-Q4_K_M.gguf](https://huggingface.co/handy-computer/Qwen3-ASR-1.7B-gguf/resolve/main/Qwen3-ASR-1.7B-Q4_K_M.gguf) | 1.32 GB | 1.81% |
157
158 WER on the full LibriSpeech test-clean split (2,620 utterances), batch size 1, timestamps none. Figures without a commit were published before provenance was recorded.
159
160 Scored with the Whisper-style English text normalizer and jiwer 3.x on an Apple M4.
161 Qwen3-ASR is a multilingual model — this number characterizes the English case only.
162 The larger decoder gives 1.7B more quantization headroom than the 0.6B; BF16 / F16 /
163 Q8_0 / Q6_K / Q5_K_M are all within bootstrap CI of each other, and Q4_K_M regresses
164 only ~0.2 WER points. Reproduce with `scripts/wer/run.py` + `scripts/wer/score.py`.
165
166
167 ## Usage
168
169 Build transcribe.cpp from source:
170
171 ```bash
172 git clone git@github.com:handy-computer/transcribe.cpp.git
173 cd transcribe.cpp
174 cmake -B build && cmake --build build
175 ```
176
177 Run on a 16 kHz mono WAV:
178
179 ```bash
180 build/bin/transcribe-cli \
181 -m Qwen3-ASR-1.7B-Q8_0.gguf \
182 input.wav
183 ```
184
185 If your audio isn't already 16 kHz mono WAV, convert it first:
186
187 ```bash
188 ffmpeg -i input.mp3 -ar 16000 -ac 1 output.wav
189 ```
190
191 See the [transcribe.cpp model page](https://github.com/handy-computer/transcribe.cpp/blob/main/docs/models/qwen3-asr-1.7b.md) for performance
192 numbers, numerical validation, and reproduction steps.
193
194 ## License
195
196 Inherited from the base model: **Apache-2.0**. See the
197 [upstream model card](https://huggingface.co/Qwen/Qwen3-ASR-1.7B) for full terms.
198 This repository includes the license text
199 ([`LICENSE`](https://huggingface.co/handy-computer/Qwen3-ASR-1.7B-gguf/blob/main/LICENSE)) and a
200 [`NOTICE`](https://huggingface.co/handy-computer/Qwen3-ASR-1.7B-gguf/blob/main/NOTICE) recording
201 how these files were derived from the original.
202
203 ---
204
205 ## Original Model Card
206
207 > The section below is reproduced from
208 > [Qwen/Qwen3-ASR-1.7B](https://huggingface.co/Qwen/Qwen3-ASR-1.7B) at commit
209 > `7278e1e` for offline reference. The upstream card is the
210 > authoritative source.
211
212 # Qwen3-ASR
213
214 ## Overview
215
216 ### Introduction
217
218 <p align="center">
219 <img src="https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/qwen3_asr_introduction.png" width="90%"/>
220 <p>
221
222 The Qwen3-ASR family includes Qwen3-ASR-1.7B and Qwen3-ASR-0.6B, which support language identification and ASR for 52 languages and dialects. Both leverage large-scale speech training data and the strong audio understanding capability of their foundation model, Qwen3-Omni. Experiments show that the 1.7B version achieves state-of-the-art performance among open-source ASR models and is competitive with the strongest proprietary commercial APIs. Here are the main features:
223
224 * **All-in-one**: Qwen3-ASR-1.7B and Qwen3-ASR-0.6B support language identification and speech recognition for 30 languages and 22 Chinese dialects, so as to English accents from multiple countries and regions.
225
226 * **Excellent and Fast**: The Qwen3-ASR family ASR models maintains high-quality and robust recognition under complex acoustic environments and challenging text patterns. Qwen3-ASR-1.7B achieves strong performance on both open-sourced and internal benchmarks. While the 0.6B version achieves accuracy-efficient trade-off, it reaches 2000 times throughput at a concurrency of 128. They both achieve streaming / offline unified inference with single model and support transcribe long audio.
227
228 * **Novel and strong forced alignment Solution**: We introduce Qwen3-ForcedAligner-0.6B, which supports timestamp prediction for arbitrary units within up to 5 minutes of speech in 11 languages. Evaluations show its timestamp accuracy surpasses E2E based forced-alignment models.
229
230 * **Comprehensive inference toolkit**: In addition to open-sourcing the architectures and weights of the Qwen3-ASR series, we also release a powerful, full-featured inference framework that supports vLLM-based batch inference, asynchronous serving, streaming inference, timestamp prediction, and more.
231
232 ### Model Architecture
233
234 <p align="center">
235 <img src="https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/overview.jpg" width="100%"/>
236 <p>
237
238
239 ### Released Models Description and Download
240
241 Below is an introduction and download information for the Qwen3-ASR models. Please select and download the model that fits your needs.
242
243 | Model | Supported Languages | Supported Dialects | Inference Mode | Audio Types |
244 |---|---|---|---|---|
245 | Qwen3-ASR-1.7B & Qwen3-ASR-0.6B | Chinese (zh), English (en), Cantonese (yue), Arabic (ar), German (de), French (fr), Spanish (es), Portuguese (pt), Indonesian (id), Italian (it), Korean (ko), Russian (ru), Thai (th), Vietnamese (vi), Japanese (ja), Turkish (tr), Hindi (hi), Malay (ms), Dutch (nl), Swedish (sv), Danish (da), Finnish (fi), Polish (pl), Czech (cs), Filipino (fil), Persian (fa), Greek (el), Hungarian (hu), Macedonian (mk), Romanian (ro) | Anhui, Dongbei, Fujian, Gansu, Guizhou, Hebei, Henan, Hubei, Hunan, Jiangxi, Ningxia, Shandong, Shaanxi, Shanxi, Sichuan, Tianjin, Yunnan, Zhejiang, Cantonese (Hong Kong accent), Cantonese (Guangdong accent), Wu language, Minnan language. | Offline / Streaming | Speech, Singing Voice, Songs with BGM |
246 | Qwen3-ForcedAligner-0.6B | Chinese, English, Cantonese, French, German, Italian, Japanese, Korean, Portuguese, Russian, Spanish | -- | NAR | Speech |
247
248 During model loading in the `qwen-asr` package or vLLM, model weights will be downloaded automatically based on the model name. However, if your runtime environment does not allow downloading weights during execution, you can use the following commands to manually download the model weights to a local directory:
249
250 ```bash
251 # Download through ModelScope (recommended for users in Mainland China)
252 pip install -U modelscope
253 modelscope download --model Qwen/Qwen3-ASR-1.7B --local_dir ./Qwen3-ASR-1.7B
254 modelscope download --model Qwen/Qwen3-ASR-0.6B --local_dir ./Qwen3-ASR-0.6B
255 modelscope download --model Qwen/Qwen3-ForcedAligner-0.6B --local_dir ./Qwen3-ForcedAligner-0.6B
256 # Download through Hugging Face
257 pip install -U "huggingface_hub[cli]"
258 huggingface-cli download Qwen/Qwen3-ASR-1.7B --local-dir ./Qwen3-ASR-1.7B
259 huggingface-cli download Qwen/Qwen3-ASR-0.6B --local-dir ./Qwen3-ASR-0.6B
260 huggingface-cli download Qwen/Qwen3-ForcedAligner-0.6B --local-dir ./Qwen3-ForcedAligner-0.6B
261 ```
262
263
264 ## Quickstart
265
266 ### Environment Setup
267
268 The easiest way to use Qwen3-ASR is to install the `qwen-asr` Python package from PyPI. This will pull in the required runtime dependencies and allow you to load any released Qwen3-ASR model. If you’d like to simplify environment setup further, you can also use our official [Docker image](#docker). The `qwen-asr` package provides two backends: the transformers backend and the vLLM backend. For usage instructions for different backends, please refer to [Python Package Usage](#python-package-usage). We recommend using a **fresh, isolated environment** to avoid dependency conflicts with existing packages. You can create a clean Python 3.12 environment like this:
269
270 ```bash
271 conda create -n qwen3-asr python=3.12 -y
272 conda activate qwen3-asr
273 ```
274
275 Run the following command to get the minimal installation with transformers-backend support:
276
277 ```bash
278 pip install -U qwen-asr
279 ```
280
281 To enable the vLLM backend for faster inference and streaming support, run:
282
283 ```bash
284 pip install -U qwen-asr[vllm]
285 ```
286
287 If you want to develop or modify the code locally, install from source in editable mode:
288
289 ```bash
290 git clone https://github.com/QwenLM/Qwen3-ASR.git
291 cd Qwen3-ASR
292 pip install -e .
293 # support vLLM backend
294 # pip install -e ".[vllm]"
295 ```
296
297 Additionally, we recommend using FlashAttention 2 to reduce GPU memory usage and accelerate inference speed, especially for long inputs and large batch sizes.
298
299 ```bash
300 pip install -U flash-attn --no-build-isolation
301 ```
302
303 If your machine has less than 96GB of RAM and lots of CPU cores, run:
304
305 ```bash
306 MAX_JOBS=4 pip install -U flash-attn --no-build-isolation
307 ```
308
309 Also, you should have hardware that is compatible with FlashAttention 2. Read more about it in the official documentation of the [FlashAttention repository](https://github.com/Dao-AILab/flash-attention). FlashAttention 2 can only be used when a model is loaded in `torch.float16` or `torch.bfloat16`.
310
311 ### Python Package Usage
312
313 #### Quick Inference
314
315 The `qwen-asr` package provides two backends: **transformers backend** and **vLLM backend**. You can pass audio inputs as a local path, a URL, base64 data, or a `(np.ndarray, sr)` tuple, and run batch inference. To quickly try Qwen3-ASR, you can use `Qwen3ASRModel.from_pretrained(...)` for the transformers backend with the following code:
316
317 ```python
318 import torch
319 from qwen_asr import Qwen3ASRModel
320
321 model = Qwen3ASRModel.from_pretrained(
322 "Qwen/Qwen3-ASR-1.7B",
323 dtype=torch.bfloat16,
324 device_map="cuda:0",
325 # attn_implementation="flash_attention_2",
326 max_inference_batch_size=32, # Batch size limit for inference. -1 means unlimited. Smaller values can help avoid OOM.
327 max_new_tokens=256, # Maximum number of tokens to generate. Set a larger value for long audio input.
328 )
329
330 results = model.transcribe(
331 audio="https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_en.wav",
332 language=None, # set "English" to force the language
333 )
334
335 print(results[0].language)
336 print(results[0].text)
337 ```
338
339 If you want to return timestamps, pass `forced_aligner` and its init kwargs. Here is an example of batch inference with timestamps output:
340
341 ```python
342 import torch
343 from qwen_asr import Qwen3ASRModel
344
345 model = Qwen3ASRModel.from_pretrained(
346 "Qwen/Qwen3-ASR-1.7B",
347 dtype=torch.bfloat16,
348 device_map="cuda:0",
349 # attn_implementation="flash_attention_2",
350 max_inference_batch_size=32, # Batch size limit for inference. -1 means unlimited. Smaller values can help avoid OOM.
351 max_new_tokens=256, # Maximum number of tokens to generate. Set a larger value for long audio input.
352 forced_aligner="Qwen/Qwen3-ForcedAligner-0.6B",
353 forced_aligner_kwargs=dict(
354 dtype=torch.bfloat16,
355 device_map="cuda:0",
356 # attn_implementation="flash_attention_2",
357 ),
358 )
359
360 results = model.transcribe(
361 audio=[
362 "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_zh.wav",
363 "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_en.wav",
364 ],
365 language=["Chinese", "English"], # can also be set to None for automatic language detection
366 return_time_stamps=True,
367 )
368
369 for r in results:
370 print(r.language, r.text, r.time_stamps[0])
371 ```
372
373 For more detailed usage examples, please refer to the [example code](https://github.com/QwenLM/Qwen3-ASR/blob/main/examples/example_qwen3_asr_transformers.py) for the transformers backend.
374
375 #### vLLM Backend
376
377 If you want the fastest inference speed with Qwen3-ASR, we strongly recommend using the vLLM backend by initializing the model with `Qwen3ASRModel.LLM(...)`. Example code is provided below. Note that you must install it via `pip install -U qwen-asr[vllm]`. If you want the model to output timestamps, it’s best to install FlashAttention via `pip install -U flash-attn --no-build-isolation` to speed up inference for the forced aligner model. Remember to wrap your code under `if __name__ == '__main__':` to avoid the `spawn` error described in [vLLM Troubleshooting](https://docs.vllm.ai/en/latest/usage/troubleshooting/#python-multiprocessing).
378
379 ```python
380 import torch
381 from qwen_asr import Qwen3ASRModel
382
383 if __name__ == '__main__':
384 model = Qwen3ASRModel.LLM(
385 model="Qwen/Qwen3-ASR-1.7B",
386 gpu_memory_utilization=0.7,
387 max_inference_batch_size=128, # Batch size limit for inference. -1 means unlimited. Smaller values can help avoid OOM.
388 max_new_tokens=4096, # Maximum number of tokens to generate. Set a larger value for long audio input.
389 forced_aligner="Qwen/Qwen3-ForcedAligner-0.6B",
390 forced_aligner_kwargs=dict(
391 dtype=torch.bfloat16,
392 device_map="cuda:0",
393 # attn_implementation="flash_attention_2",
394 ),
395 )
396
397 results = model.transcribe(
398 audio=[
399 "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_zh.wav",
400 "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_en.wav",
401 ],
402 language=["Chinese", "English"], # can also be set to None for automatic language detection
403 return_time_stamps=True,
404 )
405
406 for r in results:
407 print(r.language, r.text, r.time_stamps[0])
408 ```
409
410 For more detailed usage examples, please refer to the [example code](https://github.com/QwenLM/Qwen3-ASR/blob/main/examples/example_qwen3_asr_vllm.py) for the vLLM backend. In addition, you can start a vLLM server via the `qwen-asr-serve` command, which is a wrapper around `vllm serve`. You can pass any arguments supported by `vllm serve`, for example:
411
412 ```bash
413 qwen-asr-serve Qwen/Qwen3-ASR-1.7B --gpu-memory-utilization 0.8 --host 0.0.0.0 --port 8000
414 ```
415
416 And send requests to the server via:
417
418 ```python
419 import requests
420
421 url = "http://localhost:8000/v1/chat/completions"
422 headers = {"Content-Type": "application/json"}
423
424 data = {
425 "messages": [
426 {
427 "role": "user",
428 "content": [
429 {
430 "type": "audio_url",
431 "audio_url": {
432 "url": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_en.wav"
433 },
434 }
435 ],
436 }
437 ]
438 }
439
440 response = requests.post(url, headers=headers, json=data, timeout=300)
441 response.raise_for_status()
442 content = response.json()['choices'][0]['message']['content']
443 print(content)
444
445 # parse ASR output if you want
446 from qwen_asr import parse_asr_output
447 language, text = parse_asr_output(content)
448 print(language)
449 print(text)
450 ```
451
452 #### Streaming Inference
453
454 Qwen3-ASR fully supports streaming inference. Currently, streaming inference is only available with the vLLM backend. Note that streaming inference does not support batch inference or returning timestamps. Please refer to the [example code](https://github.com/QwenLM/Qwen3-ASR/blob/main/examples/example_qwen3_asr_vllm_streaming.py) for details. You can also launch a streaming web demo through the [guide](#streaming-demo) to experience Qwen3-ASR’s streaming transcription capabilities.
455
456 #### ForcedAligner Usage
457
458 `Qwen3-ForcedAligner-0.6B` can align text–speech pairs and return word or character level timestamps. Here is an example of using the forced aligner directly:
459
460 ```python
461 import torch
462 from qwen_asr import Qwen3ForcedAligner
463
464 model = Qwen3ForcedAligner.from_pretrained(
465 "Qwen/Qwen3-ForcedAligner-0.6B",
466 dtype=torch.bfloat16,
467 device_map="cuda:0",
468 # attn_implementation="flash_attention_2",
469 )
470
471 results = model.align(
472 audio="https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_zh.wav",
473 text="甚至出现交易几乎停滞的情况。",
474 language="Chinese",
475 )
476
477 print(results[0])
478 print(results[0][0].text, results[0][0].start_time, results[0][0].end_time)
479 ```
480
481 In addition, the forced aligner supports local paths / URLs / base64 data / `(np.ndarray, sr)` inputs and batch inference. Please refer to the [example code](https://github.com/QwenLM/Qwen3-ASR/blob/main/examples/example_qwen3_forced_aligner.py) for details.
482
483 ### DashScope API Usage
484
485 To further explore Qwen3-ASR, we encourage you to try our DashScope API for a faster and more efficient experience. For detailed API information and documentation, please refer to the following:
486
487 | API Description | API Documentation (Mainland China) | API Documentation (International) |
488 |------------------|-----------------------------------|------------------------------------|
489 | Real-time API for Qwen3-ASR. | [https://help.aliyun.com/zh/model-studio/qwen-real-time-speech-recognition](https://help.aliyun.com/zh/model-studio/qwen-real-time-speech-recognition) | [https://www.alibabacloud.com/help/en/model-studio/qwen-real-time-speech-recognition](https://www.alibabacloud.com/help/en/model-studio/qwen-real-time-speech-recognition) |
490 | FileTrans API for Qwen3-ASR. | [https://help.aliyun.com/zh/model-studio/qwen-speech-recognition](https://help.aliyun.com/zh/model-studio/qwen-speech-recognition) | [https://www.alibabacloud.com/help/en/model-studio/qwen-speech-recognition](https://www.alibabacloud.com/help/en/model-studio/qwen-speech-recognition) |
491
492
493 ## Launch Local Web UI Demo
494
495 ### Gradio Demo
496
497 To launch the Qwen3-ASR web UI gradio demo, install the `qwen-asr` package and run `qwen-asr-demo`. Use the command below for help:
498
499 ```bash
500 qwen-asr-demo --help
501 ```
502
503 To launch the demo, you can use the following commands:
504
505 ```bash
506 # Transformers backend
507 qwen-asr-demo \
508 --asr-checkpoint Qwen/Qwen3-ASR-1.7B \
509 --backend transformers \
510 --cuda-visible-devices 0 \
511 --ip 0.0.0.0 --port 8000
512
513 # Transformers backend + Forced Aligner (enable timestamps)
514 qwen-asr-demo \
515 --asr-checkpoint Qwen/Qwen3-ASR-1.7B \
516 --aligner-checkpoint Qwen/Qwen3-ForcedAligner-0.6B \
517 --backend transformers \
518 --cuda-visible-devices 0 \
519 --backend-kwargs '{"device_map":"cuda:0","dtype":"bfloat16","max_inference_batch_size":8,"max_new_tokens":256}' \
520 --aligner-kwargs '{"device_map":"cuda:0","dtype":"bfloat16"}' \
521 --ip 0.0.0.0 --port 8000
522
523 # vLLM backend + Forced Aligner (enable timestamps)
524 qwen-asr-demo \
525 --asr-checkpoint Qwen/Qwen3-ASR-1.7B \
526 --aligner-checkpoint Qwen/Qwen3-ForcedAligner-0.6B \
527 --backend vllm \
528 --cuda-visible-devices 0 \
529 --backend-kwargs '{"gpu_memory_utilization":0.7,"max_inference_batch_size":8,"max_new_tokens":2048}' \
530 --aligner-kwargs '{"device_map":"cuda:0","dtype":"bfloat16"}' \
531 --ip 0.0.0.0 --port 8000
532 ```
533
534 Then open `http://<your-ip>:8000`, or access it via port forwarding in tools like VS Code.
535
536 #### Backend Notes
537
538 This demo supports two backends: transformers and vLLM. All backend-specific initialization parameters should be passed via `--backend-kwargs` as a JSON dict. If not provided, the demo will use sensible defaults.
539
540 ```bash
541 # Example: override transformers init args without flash attention
542 --backend-kwargs '{"device_map":"cuda:0","dtype":"bfloat16"}'
543
544 # Example: override vLLM init args with 65% GPU memory
545 --backend-kwargs '{"gpu_memory_utilization":0.65}'
546 ```
547
548 #### CUDA Device Notes
549
550 Because vLLM does not follow `cuda:0` style device selection, this demo selects GPUs by setting `CUDA_VISIBLE_DEVICES` via `--cuda-visible-devices`.
551
552 ```bash
553 # Use GPU 0
554 --cuda-visible-devices 0
555
556 # Use GPU 1
557 --cuda-visible-devices 1
558 ```
559
560 #### Timestamps Notes
561
562 Timestamps are only available when `--aligner-checkpoint` is provided. If you launch the demo without a forced aligner, the timestamps UI will be hidden automatically.
563
564 ```bash
565 # No forced aligner
566 qwen-asr-demo --asr-checkpoint Qwen/Qwen3-ASR-1.7B
567
568 # With forced aligner
569 qwen-asr-demo \
570 --asr-checkpoint Qwen/Qwen3-ASR-1.7B \
571 --aligner-checkpoint Qwen/Qwen3-ForcedAligner-0.6B
572 ```
573
574 #### HTTPS Notes
575
576 To avoid browser microphone permission issues after deploying the server, it is recommended/required to run the gradio service over HTTPS (especially when accessed remotely or behind modern browsers/gateways). Use `--ssl-certfile` and `--ssl-keyfile` to enable HTTPS. First, generate a private key and a self-signed certificate (valid for 365 days):
577
578 ```bash
579 openssl req -x509 -newkey rsa:2048 \
580 -keyout key.pem -out cert.pem \
581 -days 365 -nodes \
582 -subj "/CN=localhost"
583 ```
584
585 Then run the demo with HTTPS:
586
587 ```bash
588 qwen-asr-demo \
589 --asr-checkpoint Qwen/Qwen3-ASR-1.7B \
590 --backend transformers \
591 --cuda-visible-devices 0 \
592 --ip 0.0.0.0 --port 8000 \
593 --ssl-certfile cert.pem \
594 --ssl-keyfile key.pem \
595 --no-ssl-verify
596 ```
597
598 Then open `https://<your-ip>:8000` to use it. If your browser shows a warning, that’s expected for self-signed certificates. For production, use a real certificate.
599
600 ### Streaming Demo
601
602 To experience Qwen3-ASR’s streaming transcription capability in a web UI, we provide a minimal Flask-based streaming demo. The demo captures microphone audio in the browser, resamples it to 16,000 Hz, and continuously pushes PCM chunks to the model. Run the demo with the following command:
603
604 ```bash
605 qwen-asr-demo-streaming \
606 --asr-model-path Qwen/Qwen3-ASR-1.7B \
607 --host 0.0.0.0 \
608 --port 8000 \
609 --gpu-memory-utilization 0.9
610 ```
611
612 Then open `http://<your-ip>:8000`, or access it via port forwarding in tools like VS Code.
613
614 ## Deployment with vLLM
615
616 vLLM officially provides day-0 model support for Qwen3-ASR for efficient inference.
617
618 ### Installation
619 You can run Qwen3-ASR with vLLM nightly wheel or docker image. To install the nightly version of vLLM, we recommend using `uv` as the environment manager
620 ```bash
621 uv venv
622 source .venv/bin/activate
623 uv pip install -U vllm --pre \
624 --extra-index-url https://wheels.vllm.ai/nightly/cu129 \
625 --extra-index-url https://download.pytorch.org/whl/cu129 \
626 --index-strategy unsafe-best-match
627 uv pip install "vllm[audio]" # For additional audio dependencies
628 ```
629
630 ### Online Serving
631 You can easily deploy Qwen3-ASR with vLLM by running the following command
632 ```bash
633 vllm serve Qwen/Qwen3-ASR-1.7B
634 ```
635 After the model server is successfully deployed, you can interact with it in multiple ways.
636
637 #### Using OpenAI SDK
638 ```python
639 import base64
640 import httpx
641 from openai import OpenAI
642
643 # Initialize client
644 client = OpenAI(
645 base_url="http://localhost:8000/v1",
646 api_key="EMPTY"
647 )
648
649 # Create multimodal chat completion request
650 response = client.chat.completions.create(
651 model="Qwen/Qwen3-ASR-1.7B",
652 messages=[
653 {
654 "role": "user",
655 "content": [
656 {
657 "type": "audio_url",
658 "audio_url": {
659 {"url": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_en.wav"}
660 }
661 }
662 ]
663 }
664 ],
665 )
666
667 print(response.choices[0].message.content)
668 ```
669 This model is also supported on vLLM with OpenAI transcription API.
670 ```python
671 import httpx
672 from openai import OpenAI
673
674 # Initialize client
675 client = OpenAI(
676 base_url="http://localhost:8000/v1",
677 api_key="EMPTY"
678 )
679 audio_url = "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_en.wav"
680 audio_file = httpx.get(audio_url).content
681
682 transcription = client.audio.transcriptions.create(
683 model="Qwen/Qwen3-ASR-1.7B",
684 file=audio_file,
685 )
686
687 print(transcription.text)
688 ```
689
690 #### Using cURL
691 ```bash
692 curl http://localhost:8000/v1/chat/completions \
693 -H "Content-Type: application/json" \
694 -d '{
695 "messages": [
696 {"role": "user", "content": [
697 {"type": "audio_url", "audio_url": {"url": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_en.wav"}}
698 ]}
699 ]
700 }'
701 ```
702
703 ### Offline Inference
704 See the following example on using vLLM to run offline inference with Qwen3-ASR
705 ```python
706 from vllm import LLM, SamplingParams
707 from vllm.assets.audio import AudioAsset
708 import base64
709 import requests
710
711 # Initialize the LLM
712 llm = LLM(
713 model="Qwen/Qwen3-ASR-1.7B"
714 )
715
716 # Load audio
717 audio_asset = AudioAsset("winning_call")
718
719 # Create conversation with audio content
720 conversation = [
721 {
722 "role": "user",
723 "content": [
724 {
725 "type": "audio_url",
726 "audio_url": {"url": audio_asset.url}
727 }
728 ]
729 }
730 ]
731
732 sampling_params = SamplingParams(temperature=0.01, max_tokens=256)
733
734 # Run inference using .chat()
735 outputs = llm.chat(conversation, sampling_params=sampling_params)
736 print(outputs[0].outputs[0].text)
737 ```
738
739
740 ## Docker
741
742 To make it easier to use our `qwen-asr` Python package, we provide a pre-built Docker image: [qwenllm/qwen3-asr](https://hub.docker.com/r/qwenllm/qwen3-asr). You only need to install the GPU driver and download the model files to run the code. Please follow the [NVIDIA Container Toolkit installation guide](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) to ensure Docker can access your GPU. If you are in Mainland China and have trouble reaching Docker Hub, you may use a registry mirror to accelerate image pulls.
743
744 First, pull the image and start a container:
745
746 ```bash
747 LOCAL_WORKDIR=/path/to/your/workspace
748 HOST_PORT=8000
749 CONTAINER_PORT=80
750 docker run --gpus all --name qwen3-asr \
751 -v /var/run/docker.sock:/var/run/docker.sock -p $HOST_PORT:$CONTAINER_PORT \
752 --mount type=bind,source=$LOCAL_WORKDIR,target=/data/shared/Qwen3-ASR \
753 --shm-size=4gb \
754 -it qwenllm/qwen3-asr:latest
755 ```
756
757 After running the command, you will enter the container’s bash shell. Your local workspace (**replace** `/path/to/your/workspace` **with the actual path**) will be mounted inside the container at `/data/shared/Qwen3-ASR`. Port `8000` on the host is mapped to port `80` in the container, so you can access services running in the container via `http://<host-ip>:8000`. Note that services inside the container must bind to `0.0.0.0` (not `127.0.0.1`) for port forwarding to work.
758
759 If you exit the container, you can start it again and re-enter it with:
760
761 ```bash
762 docker start qwen3-asr
763 docker exec -it qwen3-asr bash
764 ```
765
766 To remove the container completely, run:
767
768 ```bash
769 docker rm -f qwen3-asr
770 ```
771
772
773 ## Evaluation
774
775 During evaluation, we ran inference for all models with `dtype=torch.bfloat16` and set `max_new_tokens=1024` using vLLM. Greedy search was used for all decoding, and none of the tests specified a language parameter. The detailed evaluation results are shown below.
776
777 <details>
778 <summary>ASR Benchmarks on Public Datasets (WER ↓)</summary>
779
780 <table>
781 <thead>
782 <tr>
783 <th colspan="2" style="text-align: left;"></th>
784 <th style="text-align: center;">GPT-4o<br>-Transcribe</th>
785 <th style="text-align: center;">Gemini-2.5<br>-Pro</th>
786 <th style="text-align: center;">Doubao-ASR</th>
787 <th style="text-align: center;">Whisper<br>-large-v3</th>
788 <th style="text-align: center;">Fun-ASR<br>-MLT-Nano</th>
789 <th style="text-align: center;">Qwen3-ASR<br>-0.6B</th>
790 <th style="text-align: center;">Qwen3-ASR<br>-1.7B</th>
791 </tr>
792 </thead>
793 <tbody>
794 <tr>
795 <td colspan="9" style="text-align: left; font-style: italic; border-top: 1px solid #ddd; border-bottom: 1px solid #ddd;">English (en)</td>
796 </tr>
797 <tr>
798 <td colspan="2" style="text-align: left;">Librispeech<br>clean | other</td>
799 <td style="text-align: center;"><strong>1.39</strong> | 3.75</td>
800 <td style="text-align: center;">2.89 | 3.56</td>
801 <td style="text-align: center;">2.78 | 5.70</td>
802 <td style="text-align: center;">1.51 | 3.97</td>
803 <td style="text-align: center;">1.68 | 4.03</td>
804 <td style="text-align: center;">2.11 | 4.55</td>
805 <td style="text-align: center;">1.63 | <strong>3.38</strong></td>
806 </tr>
807 <tr>
808 <td colspan="2" style="text-align: left;">GigaSpeech</td>
809 <td style="text-align: center;">25.50</td>
810 <td style="text-align: center;">9.37</td>
811 <td style="text-align: center;">9.55</td>
812 <td style="text-align: center;">9.76</td>
813 <td style="text-align: center;">-</td>
814 <td style="text-align: center;">8.88</td>
815 <td style="text-align: center;"><strong>8.45</strong></td>
816 </tr>
817 <tr>
818 <td colspan="2" style="text-align: left;">CV-en</td>
819 <td style="text-align: center;">9.08</td>
820 <td style="text-align: center;">14.49</td>
821 <td style="text-align: center;">13.78</td>
822 <td style="text-align: center;">9.90</td>
823 <td style="text-align: center;">9.90</td>
824 <td style="text-align: center;">9.92</td>
825 <td style="text-align: center;"><strong>7.39</strong></td>
826 </tr>
827 <tr>
828 <td colspan="2" style="text-align: left;">Fleurs-en</td>
829 <td style="text-align: center;"><strong>2.40</strong></td>
830 <td style="text-align: center;">2.94</td>
831 <td style="text-align: center;">6.31</td>
832 <td style="text-align: center;">4.08</td>
833 <td style="text-align: center;">5.49</td>
834 <td style="text-align: center;">4.39</td>
835 <td style="text-align: center;">3.35</td>
836 </tr>
837 <tr>
838 <td colspan="2" style="text-align: left;">MLS-en</td>
839 <td style="text-align: center;">5.12</td>
840 <td style="text-align: center;"><strong>3.68</strong></td>
841 <td style="text-align: center;">7.09</td>
842 <td style="text-align: center;">4.87</td>
843 <td style="text-align: center;">-</td>
844 <td style="text-align: center;">6.00</td>
845 <td style="text-align: center;">4.58</td>
846 </tr>
847 <tr>
848 <td colspan="2" style="text-align: left;">Tedlium</td>
849 <td style="text-align: center;">7.69</td>
850 <td style="text-align: center;">6.15</td>
851 <td style="text-align: center;">4.91</td>
852 <td style="text-align: center;">6.84</td>
853 <td style="text-align: center;">-</td>
854 <td style="text-align: center;"><strong>3.85<strong></td>
855 <td style="text-align: center;"><strong>4.50</strong></td>
856 </tr>
857 <tr>
858 <td colspan="2" style="text-align: left;">VoxPopuli</td>
859 <td style="text-align: center;">10.29</td>
860 <td style="text-align: center;">11.36</td>
861 <td style="text-align: center;">12.12</td>
862 <td style="text-align: center;">12.05</td>
863 <td style="text-align: center;">-</td>
864 <td style="text-align: center;"><strong>9.96<strong></td>
865 <td style="text-align: center;"><strong>9.15</strong></td>
866 </tr>
867 <tr>
868 <td colspan="9" style="text-align: left; font-style: italic; border-top: 1px solid #ddd; border-bottom: 1px solid #ddd;">Chinese (zh)</td>
869 </tr>
870 <tr>
871 <td colspan="2" style="text-align: left;">WenetSpeech<br>net | meeting</td>
872 <td style="text-align: center;">15.30 | 32.27</td>
873 <td style="text-align: center;">14.43 | 13.47</td>
874 <td style="text-align: center;">N/A</td>
875 <td style="text-align: center;">9.86 | 19.11</td>
876 <td style="text-align: center;">6.35 | -</td>
877 <td style="text-align: center;">5.97 | 6.88</td>
878 <td style="text-align: center;"><strong>4.97</strong> | <strong>5.88</strong></td>
879 </tr>
880 <tr>
881 <td colspan="2" style="text-align: left;">AISHELL-2-test</td>
882 <td style="text-align: center;">4.24</td>
883 <td style="text-align: center;">11.62</td>
884 <td style="text-align: center;">2.85</td>
885 <td style="text-align: center;">5.06</td>
886 <td style="text-align: center;">-</td>
887 <td style="text-align: center;">3.15</td>
888 <td style="text-align: center;"><strong>2.71</strong></td>
889 </tr>
890 <tr>
891 <td colspan="2" style="text-align: left;">SpeechIO</td>
892 <td style="text-align: center;">12.86</td>
893 <td style="text-align: center;">5.30</td>
894 <td style="text-align: center;">2.93</td>
895 <td style="text-align: center;">7.56</td>
896 <td style="text-align: center;">-</td>
897 <td style="text-align: center;">3.44</td>
898 <td style="text-align: center;"><strong>2.88</strong></td>
899 </tr>
900 <tr>
901 <td colspan="2" style="text-align: left;">Fleurs-zh</td>
902 <td style="text-align: center;">2.44</td>
903 <td style="text-align: center;">2.71</td>
904 <td style="text-align: center;">2.69</td>
905 <td style="text-align: center;">4.09</td>
906 <td style="text-align: center;">3.51</td>
907 <td style="text-align: center;">2.88</td>
908 <td style="text-align: center;"><strong>2.41</strong></td>
909 </tr>
910 <tr>
911 <td colspan="2" style="text-align: left;">CV-zh</td>
912 <td style="text-align: center;">6.32</td>
913 <td style="text-align: center;">7.70</td>
914 <td style="text-align: center;">5.95</td>
915 <td style="text-align: center;">12.91</td>
916 <td style="text-align: center;">6.20</td>
917 <td style="text-align: center;">6.89</td>
918 <td style="text-align: center;"><strong>5.35</strong></td>
919 </tr>
920 <tr>
921 <td colspan="9" style="text-align: left; font-style: italic; border-top: 1px solid #ddd; border-bottom: 1px solid #ddd;">Chinese Dialect</td>
922 </tr>
923 <tr>
924 <td colspan="2" style="text-align: left;">KeSpeech</td>
925 <td style="text-align: center;">26.87</td>
926 <td style="text-align: center;">24.71</td>
927 <td style="text-align: center;">5.27</td>
928 <td style="text-align: center;">28.79</td>
929 <td style="text-align: center;">-</td>
930 <td style="text-align: center;">7.08</td>
931 <td style="text-align: center;"><strong>5.10</strong></td>
932 </tr>
933 <tr>
934 <td colspan="2" style="text-align: left;">Fleurs-yue</td>
935 <td style="text-align: center;">4.98</td>
936 <td style="text-align: center;">9.43</td>
937 <td style="text-align: center;">4.98</td>
938 <td style="text-align: center;">9.18</td>
939 <td style="text-align: center;">-</td>
940 <td style="text-align: center;">5.79</td>
941 <td style="text-align: center;"><strong>3.98</strong></td>
942 </tr>
943 <tr>
944 <td colspan="2" style="text-align: left;">CV-yue</td>
945 <td style="text-align: center;">11.36</td>
946 <td style="text-align: center;">18.76</td>
947 <td style="text-align: center;">13.20</td>
948 <td style="text-align: center;">16.23</td>
949 <td style="text-align: center;">-</td>
950 <td style="text-align: center;">9.50</td>
951 <td style="text-align: center;"><strong>7.57</strong></td>
952 </tr>
953 <tr>
954 <td colspan="2" style="text-align: left;">CV-zh-tw</td>
955 <td style="text-align: center;">6.32</td>
956 <td style="text-align: center;">7.31</td>
957 <td style="text-align: center;">4.06</td>
958 <td style="text-align: center;">7.84</td>
959 <td style="text-align: center;">-</td>
960 <td style="text-align: center;">5.59</td>
961 <td style="text-align: center;"><strong>3.77</strong></td>
962 </tr>
963 <tr>
964 <td colspan="2" style="text-align: left;">WenetSpeech-Yue<br>short | long</td>
965 <td style="text-align: center;">15.62 | 25.29</td>
966 <td style="text-align: center;">25.19 | 11.23</td>
967 <td style="text-align: center;">9.74 | 11.40</td>
968 <td style="text-align: center;">32.26 | 46.64</td>
969 <td style="text-align: center;">- | -</td>
970 <td style="text-align: center;">7.54 | 9.92</td>
971 <td style="text-align: center;"><strong>5.82</strong> | <strong>8.85</strong></td>
972 </tr>
973 <tr>
974 <td colspan="2" style="text-align: left;">WenetSpeech-Chuan<br>easy | hard</td>
975 <td style="text-align: center;">34.81 | 53.98</td>
976 <td style="text-align: center;">43.79 | 67.30</td>
977 <td style="text-align: center;"><strong>11.40<strong> | <strong>20.20</strong></td>
978 <td style="text-align: center;">14.35 | 26.80</td>
979 <td style="text-align: center;">- | -</td>
980 <td style="text-align: center;">13.92 | 24.45</td>
981 <td style="text-align: center;">11.99 | 21.63</td>
982 </tr>
983 </tbody>
984 </table>
985
986 </details>
987
988 <details>
989 <summary>ASR Benchmarks on Internal Datasets (WER ↓)</summary>
990
991 <table>
992 <thead>
993 <tr>
994 <th style="text-align: left;"></th>
995 <th style="text-align: center;">GPT-4o<br>-Transcribe</th>
996 <th style="text-align: center;">Gemini-2.5<br>-Pro</th>
997 <th style="text-align: center;">Doubao-ASR</th>
998 <th style="text-align: center;">Whisper<br>-large-v3</th>
999 <th style="text-align: center;">Fun-ASR<br>-MLT-Nano</th>
1000 <th style="text-align: center;">Qwen3-ASR<br>-0.6B</th>
1001 <th style="text-align: center;">Qwen3-ASR<br>-1.7B</th>
1002 </tr>
1003 </thead>
1004 <tbody>
1005 <tr>
1006 <td colspan="8" style="text-align: left; font-style: italic; border-top: 1px solid #ddd; border-bottom: 1px solid #ddd;">Accented English</td>
1007 </tr>
1008 <tr>
1009 <td style="text-align: left;">Dialog-Accented English</td>
1010 <td style="text-align: center;">28.56</td>
1011 <td style="text-align: center;">23.85</td>
1012 <td style="text-align: center;">20.41</td>
1013 <td style="text-align: center;">21.30</td>
1014 <td style="text-align: center;">19.96</td>
1015 <td style="text-align: center;"><strong>16.62<strong></td>
1016 <td style="text-align: center;"><strong>16.07</strong></td>
1017 </tr>
1018 <tr>
1019 <td colspan="8" style="text-align: left; font-style: italic; border-top: 1px solid #ddd; border-bottom: 1px solid #ddd;">Chinese Mandarin</td>
1020 </tr>
1021 <tr>
1022 <td style="text-align: left;">Elders&Kids</td>
1023 <td style="text-align: center;">14.27</td>
1024 <td style="text-align: center;">36.93</td>
1025 <td style="text-align: center;">4.17</td>
1026 <td style="text-align: center;">10.61</td>
1027 <td style="text-align: center;">4.54</td>
1028 <td style="text-align: center;">4.48</td>
1029 <td style="text-align: center;"><strong>3.81</strong></td>
1030 </tr>
1031 <tr>
1032 <td style="text-align: left;">ExtremeNoise</td>
1033 <td style="text-align: center;">36.11</td>
1034 <td style="text-align: center;">29.06</td>
1035 <td style="text-align: center;">17.04</td>
1036 <td style="text-align: center;">63.17</td>
1037 <td style="text-align: center;">36.55</td>
1038 <td style="text-align: center;">17.88</td>
1039 <td style="text-align: center;"><strong>16.17</strong></td>
1040 </tr>
1041 <tr>
1042 <td style="text-align: left;">TongueTwister</td>
1043 <td style="text-align: center;">20.87</td>
1044 <td style="text-align: center;">4.97</td>
1045 <td style="text-align: center;">3.47</td>
1046 <td style="text-align: center;">16.63</td>
1047 <td style="text-align: center;">9.02</td>
1048 <td style="text-align: center;">4.06</td>
1049 <td style="text-align: center;"><strong>2.44</strong></td>
1050 </tr>
1051 <tr>
1052 <td style="text-align: left;">Dialog-Mandarin</td>
1053 <td style="text-align: center;">20.73</td>
1054 <td style="text-align: center;">12.50</td>
1055 <td style="text-align: center;">6.61</td>
1056 <td style="text-align: center;">14.01</td>
1057 <td style="text-align: center;">7.32</td>
1058 <td style="text-align: center;">7.06</td>
1059 <td style="text-align: center;"><strong>6.54</strong></td>
1060 </tr>
1061 <tr>
1062 <td colspan="8" style="text-align: left; font-style: italic; border-top: 1px solid #ddd; border-bottom: 1px solid #ddd;">Chinese Dialect</td>
1063 </tr>
1064 <tr>
1065 <td style="text-align: left;">Dialog-Cantonese</td>
1066 <td style="text-align: center;">16.05</td>
1067 <td style="text-align: center;">14.98</td>
1068 <td style="text-align: center;">7.56</td>
1069 <td style="text-align: center;">31.04</td>
1070 <td style="text-align: center;">5.85</td>
1071 <td style="text-align: center;"><strong>4.80<strong></td>
1072 <td style="text-align: center;"><strong>4.12</strong></td>
1073 </tr>
1074 <tr>
1075 <td style="text-align: left;">Dialog-Chinese Dialects</td>
1076 <td style="text-align: center;">45.37</td>
1077 <td style="text-align: center;">47.70</td>
1078 <td style="text-align: center;">19.85</td>
1079 <td style="text-align: center;">44.55</td>
1080 <td style="text-align: center;">19.41</td>
1081 <td style="text-align: center;"><strong>18.24<strong></td>
1082 <td style="text-align: center;"><strong>15.94</strong></td>
1083 </tr>
1084 </tbody>
1085 </table>
1086 <p><strong>Dialect coverage:</strong> Results for <em>Dialog-Accented English</em> are averaged over 16 accents, and results for <em>Dialog-Chinese Dialects</em> are averaged over 22 Chinese dialects.</p>
1087
1088 </details>
1089
1090 <details>
1091 <summary>Multilingual ASR Benchmarks (WER ↓)</summary>
1092
1093 <table>
1094 <thead>
1095 <tr>
1096 <th style="text-align: left;"></th>
1097 <th style="text-align: center;">GLM-ASR<br>-Nano-2512</th>
1098 <th style="text-align: center;">Whisper<br>-large-v3</th>
1099 <th style="text-align: center;">Fun-ASR<br>-MLT-Nano</th>
1100 <th style="text-align: center;">Qwen3-ASR<br>-0.6B</th>
1101 <th style="text-align: center;">Qwen3-ASR<br>-1.7B</th>
1102 </tr>
1103 </thead>
1104 <tbody>
1105 <tr>
1106 <td colspan="6" style="text-align: left; font-style: italic; border-top: 1px solid #ddd; border-bottom: 1px solid #ddd;">Open-sourced Benchmarks</td>
1107 </tr>
1108 <tr>
1109 <td style="text-align: left;">MLS</td>
1110 <td style="text-align: center;">13.32</td>
1111 <td style="text-align: center;">8.62</td>
1112 <td style="text-align: center;">28.70</td>
1113 <td style="text-align: center;">13.19</td>
1114 <td style="text-align: center;"><strong>8.55</strong></td>
1115 </tr>
1116 <tr>
1117 <td style="text-align: left;">CommonVoice</td>
1118 <td style="text-align: center;">19.40</td>
1119 <td style="text-align: center;">10.77</td>
1120 <td style="text-align: center;">17.25</td>
1121 <td style="text-align: center;">12.75</td>
1122 <td style="text-align: center;"><strong>9.18</strong></td>
1123 </tr>
1124 <tr>
1125 <td style="text-align: left;">MLC-SLM</td>
1126 <td style="text-align: center;">34.93</td>
1127 <td style="text-align: center;">15.68</td>
1128 <td style="text-align: center;">29.94</td>
1129 <td style="text-align: center;">15.84</td>
1130 <td style="text-align: center;"><strong>12.74</strong></td>
1131 </tr>
1132 <tr>
1133 <td style="text-align: left;">Fleurs</td>
1134 <td style="text-align: center;">16.08</td>
1135 <td style="text-align: center;">5.27</td>
1136 <td style="text-align: center;">10.03</td>
1137 <td style="text-align: center;">7.57</td>
1138 <td style="text-align: center;"><strong>4.90</strong></td>
1139 </tr>
1140 <tr>
1141 <td style="text-align: left;">Fleurs<sup>†</sup></td>
1142 <td style="text-align: center;">20.05</td>
1143 <td style="text-align: center;">6.85</td>
1144 <td style="text-align: center;">31.89</td>
1145 <td style="text-align: center;">10.37</td>
1146 <td style="text-align: center;"><strong>6.62</strong></td>
1147 </tr>
1148 <tr>
1149 <td style="text-align: left;">Fleurs<sup>††</sup></td>
1150 <td style="text-align: center;">24.83</td>
1151 <td style="text-align: center;"><strong>8.16</strong></td>
1152 <td style="text-align: center;">47.84</td>
1153 <td style="text-align: center;">21.80</td>
1154 <td style="text-align: center;">12.60</td>
1155 </tr>
1156 <tr>
1157 <td colspan="6" style="text-align: left; font-style: italic; border-top: 1px solid #ddd; border-bottom: 1px solid #ddd;">Qwen-ASR Internal Benchmarks</td>
1158 </tr>
1159 <tr>
1160 <td style="text-align: left;">News-Multilingual</td>
1161 <td style="text-align: center;">49.40</td>
1162 <td style="text-align: center;">14.80</td>
1163 <td style="text-align: center;">65.07</td>
1164 <td style="text-align: center;">17.39</td>
1165 <td style="text-align: center;"><strong>12.80</strong></td>
1166 </tr>
1167 </tbody>
1168 </table>
1169 <p><strong>Language coverage:</strong> <em>MLS</em> includes 8 languages: {da, de, en, es, fr, it, pl, pt}.<br><em>CommonVoice</em> includes 13 languages: {en, zh, yue, zh_TW, ar, de, es, fr, it, ja, ko, pt, ru}.<br><em>MLC-SLM</em> includes 11 languages: {en, fr, de, it, pt, es, ja, ko, ru, th, vi}.<br><em>Fleurs</em> includes 12 languages: {en, zh, yue, ar, de, es, fr, it, ja, ko, pt, ru }.<br><em>Fleurs<sup>†</sup></em> includes 8 additional languages beyond Fleurs: {hi, id, ms, nl, pl, th, tr, vi}.<br><em>Fleurs<sup>††</sup></em> includes 10 additional languages beyond Fleurs<sup>†</sup>: {cs, da, el, fa, fi, fil, hu, mk, ro, sv}.<br><em>News-Multilingual</em> includes 15 languages: {ar, de, es, fr, hi, id, it, ja, ko, nl, pl, pt, ru, th, vi}.</p>
1170
1171 </details>
1172
1173 <details>
1174 <summary>Language Identification Accuracy (%) ↑</summary>
1175
1176 <table>
1177 <thead>
1178 <tr>
1179 <th style="text-align: left;"></th>
1180 <th style="text-align: center;">Whisper-large-v3</th>
1181 <th style="text-align: center;">Qwen3-ASR-0.6B</th>
1182 <th style="text-align: center;">Qwen3-ASR-1.7B</th>
1183 </tr>
1184 </thead>
1185 <tbody>
1186 <tr>
1187 <td style="text-align: left;">MLS</td>
1188 <td style="text-align: center;"><strong>99.9</strong></td>
1189 <td style="text-align: center;">99.3</td>
1190 <td style="text-align: center;"><strong>99.9</strong></td>
1191 </tr>
1192 <tr>
1193 <td style="text-align: left;">CommonVoice</td>
1194 <td style="text-align: center;">92.7</td>
1195 <td style="text-align: center;"><strong>98.2<strong></td>
1196 <td style="text-align: center;"><strong>98.7</strong></td>
1197 </tr>
1198 <tr>
1199 <td style="text-align: left;">MLC-SLM</td>
1200 <td style="text-align: center;">89.2</td>
1201 <td style="text-align: center;"><strong>92.7<strong></td>
1202 <td style="text-align: center;"><strong>94.1</strong></td>
1203 </tr>
1204 <tr>
1205 <td style="text-align: left;">Fleurs</td>
1206 <td style="text-align: center;">94.6</td>
1207 <td style="text-align: center;"><strong>97.1<strong></td>
1208 <td style="text-align: center;"><strong>98.7</strong></td>
1209 </tr>
1210 <tr style="border-top: 1px solid #ddd;">
1211 <td style="text-align: left;"><em>Avg.</em></td>
1212 <td style="text-align: center;">94.1</td>
1213 <td style="text-align: center;"><strong>96.8<strong></td>
1214 <td style="text-align: center;"><strong>97.9</strong></td>
1215 </tr>
1216 </tbody>
1217 </table>
1218 <p><strong>Language coverage:</strong> The language sets follow Multilingual ASR Benchmarks. Here, Fleurs corresponds to Fleurs<sup>††</sup> in Multilingual ASR Benchmarks and covers 30 languages.</p>
1219
1220 </details>
1221
1222 <details>
1223 <summary>Singing Voice & Song Transcription (WER ↓)</summary>
1224
1225 <table>
1226 <thead>
1227 <tr>
1228 <th style="text-align: left;"></th>
1229 <th style="text-align: center;">GPT-4o<br>-Transcribe</th>
1230 <th style="text-align: center;">Gemini-2.5<br>-Pro</th>
1231 <th style="text-align: center;">Doubao-ASR<br>-1.0</th>
1232 <th style="text-align: center;">Whisper<br>-large-v3</th>
1233 <th style="text-align: center;">Fun-ASR-MLT<br>-Nano</th>
1234 <th style="text-align: center;">Qwen3-ASR<br>-1.7B</th>
1235 </tr>
1236 </thead>
1237 <tbody>
1238 <tr>
1239 <td colspan="7" style="text-align: left; font-style: italic; border-top: 1px solid #ddd; border-bottom: 1px solid #ddd;">Singing</td>
1240 </tr>
1241 <tr>
1242 <td style="text-align: left;">M4Singer</td>
1243 <td style="text-align: center;">16.77</td>
1244 <td style="text-align: center;">20.88</td>
1245 <td style="text-align: center;">7.88</td>
1246 <td style="text-align: center;">13.58</td>
1247 <td style="text-align: center;">7.29</td>
1248 <td style="text-align: center;"><strong>5.98</strong></td>
1249 </tr>
1250 <tr>
1251 <td style="text-align: left;">MIR-1k-vocal</td>
1252 <td style="text-align: center;">11.87</td>
1253 <td style="text-align: center;">9.85</td>
1254 <td style="text-align: center;">6.56</td>
1255 <td style="text-align: center;">11.71</td>
1256 <td style="text-align: center;">8.17</td>
1257 <td style="text-align: center;"><strong>6.25</strong></td>
1258 </tr>
1259 <tr>
1260 <td style="text-align: left;">Opencpop</td>
1261 <td style="text-align: center;">7.93</td>
1262 <td style="text-align: center;">6.49</td>
1263 <td style="text-align: center;">3.80</td>
1264 <td style="text-align: center;">9.52</td>
1265 <td style="text-align: center;"><strong>2.98</strong></td>
1266 <td style="text-align: center;">3.08</td>
1267 </tr>
1268 <tr>
1269 <td style="text-align: left;">Popcs</td>
1270 <td style="text-align: center;">32.84</td>
1271 <td style="text-align: center;">15.13</td>
1272 <td style="text-align: center;">8.97</td>
1273 <td style="text-align: center;">13.77</td>
1274 <td style="text-align: center;">9.42</td>
1275 <td style="text-align: center;"><strong>8.52</strong></td>
1276 </tr>
1277 <tr>
1278 <td colspan="7" style="text-align: left; font-style: italic; border-top: 1px solid #ddd; border-bottom: 1px solid #ddd;">Songs with BGM</td>
1279 </tr>
1280 <tr>
1281 <td style="text-align: left;">EntireSongs-en</td>
1282 <td style="text-align: center;">30.71</td>
1283 <td style="text-align: center;"><strong>12.18</strong></td>
1284 <td style="text-align: center;">33.51</td>
1285 <td style="text-align: center;">N/A</td>
1286 <td style="text-align: center;">N/A</td>
1287 <td style="text-align: center;">14.60</td>
1288 </tr>
1289 <tr>
1290 <td style="text-align: left;">EntireSongs-zh</td>
1291 <td style="text-align: center;">34.86</td>
1292 <td style="text-align: center;">18.68</td>
1293 <td style="text-align: center;">23.99</td>
1294 <td style="text-align: center;">N/A</td>
1295 <td style="text-align: center;">N/A</td>
1296 <td style="text-align: center;"><strong>13.91</strong></td>
1297 </tr>
1298 </tbody>
1299 </table>
1300
1301 </details>
1302
1303 <details>
1304 <summary>ASR Inference Mode Performance (WER ↓)</summary>
1305
1306 <table>
1307 <thead>
1308 <tr>
1309 <th style="text-align: left;">Model</th>
1310 <th style="text-align: left;">Infer. Mode</th>
1311 <th style="text-align: center;">Librispeech</th>
1312 <th style="text-align: center;">Fleurs-en</th>
1313 <th style="text-align: center;">Fleurs-zh</th>
1314 <th style="text-align: center;">Avg.</th>
1315 </tr>
1316 </thead>
1317 <tbody>
1318 <tr>
1319 <td rowspan="2" style="text-align: left; vertical-align: middle;">Qwen3-ASR-1.7B</td>
1320 <td style="text-align: left;">Offline</td>
1321 <td style="text-align: center;">1.63 | 3.38</td>
1322 <td style="text-align: center;">3.35</td>
1323 <td style="text-align: center;">2.41</td>
1324 <td style="text-align: center;">2.69</td>
1325 </tr>
1326 <tr>
1327 <td style="text-align: left;">Streaming</td>
1328 <td style="text-align: center;">1.95 | 4.51</td>
1329 <td style="text-align: center;">4.02</td>
1330 <td style="text-align: center;">2.84</td>
1331 <td style="text-align: center;">3.33</td>
1332 </tr>
1333 <tr style="border-top: 1px solid #ddd;">
1334 <td rowspan="2" style="text-align: left; vertical-align: middle;">Qwen3-ASR-0.6B</td>
1335 <td style="text-align: left;">Offline</td>
1336 <td style="text-align: center;">2.11 | 4.55</td>
1337 <td style="text-align: center;">4.39</td>
1338 <td style="text-align: center;">2.88</td>
1339 <td style="text-align: center;">3.48</td>
1340 </tr>
1341 <tr>
1342 <td style="text-align: left;">Streaming</td>
1343 <td style="text-align: center;">2.54 | 6.27</td>
1344 <td style="text-align: center;">5.38</td>
1345 <td style="text-align: center;">3.40</td>
1346 <td style="text-align: center;">4.40</td>
1347 </tr>
1348 </tbody>
1349 </table>
1350
1351 </details>
1352
1353 <details>
1354 <summary>Forced Alignment Benchmarks (AAS ms ↓)</summary>
1355
1356 <table>
1357 <thead>
1358 <tr>
1359 <th style="text-align: left;"></th>
1360 <th style="text-align: center;">Monotonic-Aligner</th>
1361 <th style="text-align: center;">NFA</th>
1362 <th style="text-align: center;">WhisperX</th>
1363 <th style="text-align: center;">Qwen3-ForcedAligner-0.6B</th>
1364 </tr>
1365 </thead>
1366 <tbody>
1367 <tr>
1368 <td colspan="5" style="text-align: left; font-style: italic; border-top: 1px solid #ddd; border-bottom: 1px solid #ddd;">MFA-Labeled Raw</td>
1369 </tr>
1370 <tr>
1371 <td style="text-align: left;">Chinese</td>
1372 <td style="text-align: center;">161.1</td>
1373 <td style="text-align: center;">109.8</td>
1374 <td style="text-align: center;">-</td>
1375 <td style="text-align: center;"><strong>33.1</strong></td>
1376 </tr>
1377 <tr>
1378 <td style="text-align: left;">English</td>
1379 <td style="text-align: center;">-</td>
1380 <td style="text-align: center;">107.5</td>
1381 <td style="text-align: center;">92.1</td>
1382 <td style="text-align: center;"><strong>37.5</strong></td>
1383 </tr>
1384 <tr>
1385 <td style="text-align: left;">French</td>
1386 <td style="text-align: center;">-</td>
1387 <td style="text-align: center;">100.7</td>
1388 <td style="text-align: center;">145.3</td>
1389 <td style="text-align: center;"><strong>41.7</strong></td>
1390 </tr>
1391 <tr>
1392 <td style="text-align: left;">German</td>
1393 <td style="text-align: center;">-</td>
1394 <td style="text-align: center;">122.7</td>
1395 <td style="text-align: center;">165.1</td>
1396 <td style="text-align: center;"><strong>46.5</strong></td>
1397 </tr>
1398 <tr>
1399 <td style="text-align: left;">Italian</td>
1400 <td style="text-align: center;">-</td>
1401 <td style="text-align: center;">142.7</td>
1402 <td style="text-align: center;">155.5</td>
1403 <td style="text-align: center;"><strong>75.5</strong></td>
1404 </tr>
1405 <tr>
1406 <td style="text-align: left;">Japanese</td>
1407 <td style="text-align: center;">-</td>
1408 <td style="text-align: center;">-</td>
1409 <td style="text-align: center;">-</td>
1410 <td style="text-align: center;"><strong>42.2</strong></td>
1411 </tr>
1412 <tr>
1413 <td style="text-align: left;">Korean</td>
1414 <td style="text-align: center;">-</td>
1415 <td style="text-align: center;">-</td>
1416 <td style="text-align: center;">-</td>
1417 <td style="text-align: center;"><strong>37.2</strong></td>
1418 </tr>
1419 <tr>
1420 <td style="text-align: left;">Portuguese</td>
1421 <td style="text-align: center;">-</td>
1422 <td style="text-align: center;">-</td>
1423 <td style="text-align: center;">-</td>
1424 <td style="text-align: center;"><strong>38.4</strong></td>
1425 </tr>
1426 <tr>
1427 <td style="text-align: left;">Russian</td>
1428 <td style="text-align: center;">-</td>
1429 <td style="text-align: center;">200.7</td>
1430 <td style="text-align: center;">-</td>
1431 <td style="text-align: center;"><strong>40.2</strong></td>
1432 </tr>
1433 <tr>
1434 <td style="text-align: left;">Spanish</td>
1435 <td style="text-align: center;">-</td>
1436 <td style="text-align: center;">124.7</td>
1437 <td style="text-align: center;">108.0</td>
1438 <td style="text-align: center;"><strong>36.8</strong></td>
1439 </tr>
1440 <tr>
1441 <td style="text-align: left;"><em>Avg.</em></td>
1442 <td style="text-align: center;">161.1</td>
1443 <td style="text-align: center;">129.8</td>
1444 <td style="text-align: center;">133.2</td>
1445 <td style="text-align: center;"><strong>42.9</strong></td>
1446 </tr>
1447 <tr>
1448 <td colspan="5" style="text-align: left; font-style: italic; border-top: 1px solid #ddd; border-bottom: 1px solid #ddd;">MFA-Labeled Concat-300s</td>
1449 </tr>
1450 <tr>
1451 <td style="text-align: left;">Chinese</td>
1452 <td style="text-align: center;">1742.4</td>
1453 <td style="text-align: center;">235.0</td>
1454 <td style="text-align: center;">-</td>
1455 <td style="text-align: center;"><strong>36.5</strong></td>
1456 </tr>
1457 <tr>
1458 <td style="text-align: left;">English</td>
1459 <td style="text-align: center;">-</td>
1460 <td style="text-align: center;">226.7</td>
1461 <td style="text-align: center;">227.2</td>
1462 <td style="text-align: center;"><strong>58.6</strong></td>
1463 </tr>
1464 <tr>
1465 <td style="text-align: left;">French</td>
1466 <td style="text-align: center;">-</td>
1467 <td style="text-align: center;">230.6</td>
1468 <td style="text-align: center;">2052.2</td>
1469 <td style="text-align: center;"><strong>53.4</strong></td>
1470 </tr>
1471 <tr>
1472 <td style="text-align: left;">German</td>
1473 <td style="text-align: center;">-</td>
1474 <td style="text-align: center;">220.3</td>
1475 <td style="text-align: center;">993.4</td>
1476 <td style="text-align: center;"><strong>62.4</strong></td>
1477 </tr>
1478 <tr>
1479 <td style="text-align: left;">Italian</td>
1480 <td style="text-align: center;">-</td>
1481 <td style="text-align: center;">290.5</td>
1482 <td style="text-align: center;">5719.4</td>
1483 <td style="text-align: center;"><strong>81.6</strong></td>
1484 </tr>
1485 <tr>
1486 <td style="text-align: left;">Japanese</td>
1487 <td style="text-align: center;">-</td>
1488 <td style="text-align: center;">-</td>
1489 <td style="text-align: center;">-</td>
1490 <td style="text-align: center;"><strong>81.3</strong></td>
1491 </tr>
1492 <tr>
1493 <td style="text-align: left;">Korean</td>
1494 <td style="text-align: center;">-</td>
1495 <td style="text-align: center;">-</td>
1496 <td style="text-align: center;">-</td>
1497 <td style="text-align: center;"><strong>42.2</strong></td>
1498 </tr>
1499 <tr>
1500 <td style="text-align: left;">Portuguese</td>
1501 <td style="text-align: center;">-</td>
1502 <td style="text-align: center;">-</td>
1503 <td style="text-align: center;">-</td>
1504 <td style="text-align: center;"><strong>50.0</strong></td>
1505 </tr>
1506 <tr>
1507 <td style="text-align: left;">Russian</td>
1508 <td style="text-align: center;">-</td>
1509 <td style="text-align: center;">283.3</td>
1510 <td style="text-align: center;">-</td>
1511 <td style="text-align: center;"><strong>43.0</strong></td>
1512 </tr>
1513 <tr>
1514 <td style="text-align: left;">Spanish</td>
1515 <td style="text-align: center;">-</td>
1516 <td style="text-align: center;">240.2</td>
1517 <td style="text-align: center;">4549.9</td>
1518 <td style="text-align: center;"><strong>39.6</strong></td>
1519 </tr>
1520 <tr>
1521 <td style="text-align: left;">Cross-lingual</td>
1522 <td style="text-align: center;">-</td>
1523 <td style="text-align: center;">-</td>
1524 <td style="text-align: center;">-</td>
1525 <td style="text-align: center;"><strong>34.2</strong></td>
1526 </tr>
1527 <tr>
1528 <td style="text-align: left;"><em>Avg.</em></td>
1529 <td style="text-align: center;">1742.4</td>
1530 <td style="text-align: center;">246.7</td>
1531 <td style="text-align: center;">2708.4</td>
1532 <td style="text-align: center;"><strong>52.9</strong></td>
1533 </tr>
1534 <tr>
1535 <td colspan="5" style="text-align: left; font-style: italic; border-top: 1px solid #ddd; border-bottom: 1px solid #ddd;">Human-Labeled</td>
1536 </tr>
1537 <tr>
1538 <td style="text-align: left;">Raw</td>
1539 <td style="text-align: center;">49.9</td>
1540 <td style="text-align: center;">88.6</td>
1541 <td style="text-align: center;">-</td>
1542 <td style="text-align: center;"><strong>27.8</strong></td>
1543 </tr>
1544 <tr>
1545 <td style="text-align: left;">Raw-Noisy</td>
1546 <td style="text-align: center;">53.3</td>
1547 <td style="text-align: center;">89.5</td>
1548 <td style="text-align: center;">-</td>
1549 <td style="text-align: center;"><strong>41.8</strong></td>
1550 </tr>
1551 <tr>
1552 <td style="text-align: left;">Concat-60s</td>
1553 <td style="text-align: center;">51.1</td>
1554 <td style="text-align: center;">86.7</td>
1555 <td style="text-align: center;">-</td>
1556 <td style="text-align: center;"><strong>25.3</strong></td>
1557 </tr>
1558 <tr>
1559 <td style="text-align: left;">Concat-300s</td>
1560 <td style="text-align: center;">410.8</td>
1561 <td style="text-align: center;">140.0</td>
1562 <td style="text-align: center;">-</td>
1563 <td style="text-align: center;"><strong>24.8</strong></td>
1564 </tr>
1565 <tr>
1566 <td style="text-align: left;">Concat-Cross-lingual</td>
1567 <td style="text-align: center;">-</td>
1568 <td style="text-align: center;">-</td>
1569 <td style="text-align: center;">-</td>
1570 <td style="text-align: center;"><strong>42.5</strong></td>
1571 </tr>
1572 <tr>
1573 <td style="text-align: left;"><em>Avg.</em></td>
1574 <td style="text-align: center;">141.3</td>
1575 <td style="text-align: center;">101.2</td>
1576 <td style="text-align: center;">-</td>
1577 <td style="text-align: center;"><strong>32.4</strong></td>
1578 </tr>
1579 </tbody>
1580 </table>
1581
1582 </details>
1583
1584
1585 ## Citation
1586
1587 If you find our paper and code useful in your research, please consider giving a star :star: and citation :pencil: :)
1588
1589 ```BibTeX
1590 @article{Qwen3-ASR,
1591 title={Qwen3-ASR Technical Report},
1592 author={Xian Shi, Xiong Wang, Zhifang Guo, Yongqi Wang, Pei Zhang, Xinyu Zhang, Zishan Guo, Hongkun Hao, Yu Xi, Baosong Yang, Jin Xu, Jingren Zhou, Junyang Lin},
1593 journal={arXiv preprint arXiv:2601.21337},
1594 year={2026}
1595 }
1596 ```
1597
1598
1599 <br>
1600