README.md · canary-180m-flash-gguf

1

---

2

license: cc-by-4.0

3

base_model: nvidia/canary-180m-flash

4

base_model_relation: quantized

5

library_name: transcribe.cpp

6

pipeline_tag: automatic-speech-recognition

7

language:

8

- en

9

- de

10

- es

11

- fr

12

tags:

13

- gguf

14

- transcribe.cpp

15

- asr

16

- speech-to-text

17

- canary

18

- multitask-aed

19

- encoder-decoder

20

- translation

21

transcribe_cpp:

22

wer_librispeech_test_clean:

23

f32: 1.94

24

f16: 1.94

25

q8_0: 1.93

26

q6_k: 1.93

27

q5_k_m: 1.9

28

q4_k_m: 1.93

29

rtf_m4_max:

30

metal: 139.6

31

cpu: 74

32

rtf_ryzen_4750u:

33

vulkan: 31.9

34

cpu: 21.3

35

streaming: false

36

translate: true

37

lang_detect: false

38

timestamps: none

39

---

40

41

# canary-180m-flash: transcribe.cpp GGUF

42

43

GGUF conversions of [nvidia/canary-180m-flash](https://huggingface.co/nvidia/canary-180m-flash) for use

44

with [transcribe.cpp](https://github.com/handy-computer/transcribe.cpp).

45

46

Ported from upstream commit

47

[b12ab41](https://huggingface.co/nvidia/canary-180m-flash/commit/b12ab41),

48

pinned 2026-05-08.

49

Validated against the NeMo reference at transcribe.cpp commit

50

[db53eda](https://github.com/handy-computer/transcribe.cpp/tree/db53eda)

51

on 2026-05-08.

52

53

Offline multilingual speech-to-text and translation. A 182M-parameter

54

multitask AED with a 17-layer FastConformer encoder and a 4-layer

55

Transformer decoder. Supports automatic speech recognition in English,

56

German, Spanish, and French, and bidirectional EN↔{DE, ES, FR}

57

translation. Takes a

58

16 kHz mono WAV and produces a transcript. Not a streaming model;

59

word/segment timestamps are upstream-experimental and not exposed in

60

the v1 port.

61

62

63

## Downloads

64

65

66

| --- | --- | ---: | ---: |

67

| F32 | [canary-180m-flash-F32.gguf](https://huggingface.co/handy-computer/canary-180m-flash-gguf/resolve/main/canary-180m-flash-F32.gguf) | 721 MB | 1.94% |

68

| F16 | [canary-180m-flash-F16.gguf](https://huggingface.co/handy-computer/canary-180m-flash-gguf/resolve/main/canary-180m-flash-F16.gguf) | 364 MB | 1.94% |

69

| Q8_0 | [canary-180m-flash-Q8_0.gguf](https://huggingface.co/handy-computer/canary-180m-flash-gguf/resolve/main/canary-180m-flash-Q8_0.gguf) | 208 MB | 1.93% |

70

| Q6_K | [canary-180m-flash-Q6_K.gguf](https://huggingface.co/handy-computer/canary-180m-flash-gguf/resolve/main/canary-180m-flash-Q6_K.gguf) | 168 MB | 1.93% |

71

| Q5_K_M | [canary-180m-flash-Q5_K_M.gguf](https://huggingface.co/handy-computer/canary-180m-flash-gguf/resolve/main/canary-180m-flash-Q5_K_M.gguf) | 151 MB | 1.90% |

72

| Q4_K_M | [canary-180m-flash-Q4_K_M.gguf](https://huggingface.co/handy-computer/canary-180m-flash-gguf/resolve/main/canary-180m-flash-Q4_K_M.gguf) | 133 MB | 1.93% |

73

74

WER measured on the full LibriSpeech test-clean split (2620 utterances)

75

with greedy decoding and no external LM. F32 reference baseline: 1.94%.

76

On the same wavs, NeMo's reference run produces 1.93% (one substitution

77

difference out of ~27k reference words), so the F32 port matches the

78

reference framework at the noise floor. NVIDIA's self-reported number

79

on the upstream model card is 1.87%.

80

81

82

## Usage

83

84

Build transcribe.cpp from source:

85

86

```bash

87

git clone git@github.com:handy-computer/transcribe.cpp.git

88

cd transcribe.cpp

89

cmake -B build && cmake --build build

90

```

91

92

Run on a 16 kHz mono WAV:

93

94

```bash

95

build/bin/transcribe-cli \

96

-m canary-180m-flash-Q8_0.gguf \

97

input.wav

98

```

99

100

If your audio isn't already 16 kHz mono WAV, convert it first:

101

102

```bash

103

ffmpeg -i input.mp3 -ar 16000 -ac 1 output.wav

104

```

105

106

See the [transcribe.cpp model page](https://github.com/handy-computer/transcribe.cpp/blob/main/docs/models/canary-180m-flash.md) for performance

107

numbers, numerical validation, and reproduction steps.

108

109

## License

110

111

Inherited from the base model: **CC-BY-4.0**. See the

112

[upstream model card](https://huggingface.co/nvidia/canary-180m-flash) for full terms.

113

114

---

115

116

## Original Model Card

117

118

> The section below is reproduced from

119

> [nvidia/canary-180m-flash](https://huggingface.co/nvidia/canary-180m-flash) at commit

120

> `b12ab41` for offline reference. The upstream card is the

121

> authoritative source.

122

123

# Canary 180M Flash

124

125

<style>

126

img {

127

display: inline;

128

}

129

</style>

130

131

## Description:

132

NVIDIA NeMo Canary Flash [1] is a family of multilingual multi-tasking models based on Canary architecture [2] that achieves state-of-the art performance on multiple speech benchmarks. With 182 million parameters and an inference speed of more than 1200 RTFx (on open-asr-leaderboard sets), canary-180m-flash supports automatic speech-to-text recognition (ASR) in 4 languages (English, German, French, Spanish) and translation from English to German/French/Spanish and from German/French/Spanish to English with or without punctuation and capitalization (PnC).

133

Additionally, canary-180m-flash offers an experimental feature for word-level and segment-level timestamps in English, German, French, and Spanish.

134

This model is released under the permissive CC-BY-4.0 license and is available for commercial use.

135

136

137

## Model Architecture:

138

Canary is an encoder-decoder model with FastConformer [3] Encoder and Transformer Decoder [4]. With audio features extracted from the encoder, task tokens such as \<target language\>, \<task\>, \<toggle timestamps\> and \<toggle PnC\> are fed into the Transformer Decoder to trigger the text generation process. Canary uses a concatenated tokenizer [5] from individual SentencePiece [6] tokenizers of each language, which makes it easy to scale up to more languages. The canary-180m-flash model has 17 encoder layers and 4 decoder layers, leading to a total of 182M parameters. For more details about the architecture, please refer to [1].

139

140

## NVIDIA NeMo

141

142

To train, fine-tune or transcribe with canary-180m-flash, you will need to install [NVIDIA NeMo](https://github.com/NVIDIA/NeMo).

143

144

## How to Use this Model

145

146

The model is available for use in the NeMo framework [7], and can be used as a pre-trained checkpoint for inference or for fine-tuning on another dataset.

147

148

Please refer to [our tutorial](https://github.com/NVIDIA/NeMo/blob/main/tutorials/asr/Canary_Multitask_Speech_Model.ipynb) for more details.

149

150

A few inference examples listed below:

151

152

### Loading the Model

153

154

```python

155

from nemo.collections.asr.models import EncDecMultiTaskModel

156

# load model

157

canary_model = EncDecMultiTaskModel.from_pretrained('nvidia/canary-180m-flash')

158

# update decode params

159

decode_cfg = canary_model.cfg.decoding

160

decode_cfg.beam.beam_size = 1

161

canary_model.change_decoding_strategy(decode_cfg)

162

```

163

164

## Input:

165

**Input Type(s):** Audio 

166

**Input Format(s):** .wav or .flac files 

167

**Input Parameters(s):** 1D 

168

**Other Properties Related to Input:** 16000 Hz Mono-channel Audio, Pre-Processing Not Needed 

169

170

Input to canary-180m-flash can be either a list of paths to audio files or a jsonl manifest file.

171

172

### Inference with canary-180m-flash:

173

If the input is a list of paths, canary-180m-flash assumes that the audio is English and transcribes it. I.e., canary-180m-flash default behavior is English ASR.

174

```python

175

output = canary_model.transcribe(

176

['path1.wav', 'path2.wav'],

177

batch_size=16, # batch size to run the inference with

178

pnc='True', # generate output with Punctuation and Capitalization

179

)

180

181

predicted_text = output[0].text

182

183

```

184

185

canary-180m-flash can also predict word-level and segment-level timestamps

186

```python

187

output = canary_model.transcribe(

188

['filepath.wav'],

189

timestamps=True, # generate output with timestamps

190

)

191

192

predicted_text = output[0].text

193

word_level_timestamps = output[0].timestamp['word']

194

segment_level_timestamps = output[0].timestamp['segment']

195

196

```

197

To predict timestamps for audio files longer than 10 seconds, we recommend using the longform inference script (explained in the next section) with `chunk_len_in_secs=10.0`.

198

199

To use canary-180m-flash for transcribing other supported languages or perform Speech-to-Text translation or provide word-level timestamps, specify the input as jsonl manifest file, where each line in the file is a dictionary containing the following fields:

200

201

```yaml

202

# Example of a line in input_manifest.json

203

{

204

"audio_filepath": "/path/to/audio.wav", # path to the audio file

205

    "source_lang": "en",  # language of the audio input, set `source_lang`==`target_lang` for ASR, choices=['en','de','es','fr']

206

"target_lang": "en", # language of the text output, choices=['en','de','es','fr']

207

"pnc": "yes", # whether to have PnC output, choices=['yes', 'no']

208

"timestamp": "yes", # whether to output word-level timestamps, choices=['yes', 'no']

209

}

210

```

211

212

and then use:

213

```python

214

output = canary_model.transcribe(

215

"<path to input manifest file>",

216

batch_size=16, # batch size to run the inference with

217

)

218

```

219

220

### Longform inference with canary-180m-flash:

221

Canary models are designed to handle input audio smaller than 40 seconds. In order to handle longer audios, NeMo includes [speech_to_text_aed_chunked_infer.py](https://github.com/NVIDIA/NeMo/blob/main/examples/asr/asr_chunked_inference/aed/speech_to_text_aed_chunked_infer.py) script that handles chunking, performs inference on the chunked files, and stitches the transcripts.

222

223

The script will perform inference on all `.wav` files in `audio_dir`. Alternatively you can also pass a path to a manifest file as shown above. The decoded output will be saved at `output_json_path`.

224

225

```

226

python scripts/speech_to_text_aed_chunked_infer.py \

227

pretrained_name="nvidia/canary-180m-flash" \

228

audio_dir=$audio_dir \

229

output_filename=$output_json_path \

230

chunk_len_in_secs=40.0 \

231

batch_size=1 \

232

decoding.beam.beam_size=1 \

233

timestamps=False

234

```

235

236

**Note** that for longform inference with timestamps, it is recommended to use `chunk_len_in_secs` of 10 seconds.

237

238

239

## Output:

240

**Output Type(s):** Text 

241

**Output Format:** Text output as a string (w/ timestamps) depending on the task chosen for decoding 

242

**Output Parameters:** 1-Dimensional text string 

243

**Other Properties Related to Output:** May Need Inverse Text Normalization; Does Not Handle Special Characters 

244

245

246

## Software Integration:

247

**Runtime Engine(s):**

248

* NeMo - main 

249

250

**Supported Hardware Microarchitecture Compatibility:** 

251

* [NVIDIA Ampere] 

252

* [NVIDIA Blackwell] 

253

* [NVIDIA Jetson] 

254

* [NVIDIA Hopper] 

255

* [NVIDIA Lovelace] 

256

* [NVIDIA Pascal] 

257

* [NVIDIA Turing] 

258

* [NVIDIA Volta] 

259

260

**[Preferred/Supported] Operating System(s):** 

261

* [Linux] 

262

* [Linux 4 Tegra] 

263

* [Windows] 

264

265

## Model Version(s):

266

canary-180m-flash 

267

268

269

# Training and Evaluation Datasets:

270

271

## Training Dataset:

272

273

The canary-180m-flash model is trained on a total of 85K hrs of speech data. It consists of 31K hrs of public data, 20K hrs collected by [Suno](https://suno.ai/), and 34K hrs of in-house data.

274

The datasets below include conversations, videos from the web, and audiobook recordings.

275

276

**Data Collection Method:**

277

* Human 

278

279

**Labeling Method:**

280

* Hybrid: Human, Automated 

281

282

The constituents of public data are as follows.

283

284

#### English (25.5k hours)

285

- Librispeech 960 hours

286

- Fisher Corpus

287

- Switchboard-1 Dataset

288

- WSJ-0 and WSJ-1

289

- National Speech Corpus (Part 1, Part 6)

290

- VCTK

291

- VoxPopuli (EN)

292

- Europarl-ASR (EN)

293

- Multilingual Librispeech (MLS EN) - 2,000 hour subset

294

- Mozilla Common Voice (v7.0)

295

- People's Speech - 12,000 hour subset

296

- Mozilla Common Voice (v11.0) - 1,474 hour subset

297

298

#### German (2.5k hours)

299

- Mozilla Common Voice (v12.0) - 800 hour subset

300

- Multilingual Librispeech (MLS DE) - 1,500 hour subset

301

- VoxPopuli (DE) - 200 hr subset

302

303

#### Spanish (1.4k hours)

304

- Mozilla Common Voice (v12.0) - 395 hour subset

305

- Multilingual Librispeech (MLS ES) - 780 hour subset

306

- VoxPopuli (ES) - 108 hour subset

307

- Fisher - 141 hour subset

308

309

#### French (1.8k hours)

310

- Mozilla Common Voice (v12.0) - 708 hour subset

311

- Multilingual Librispeech (MLS FR) - 926 hour subset

312

- VoxPopuli (FR) - 165 hour subset

313

314

315

## Evaluation Dataset:

316

317

**Data Collection Method:** 

318

* Human 

319

320

**Labeling Method:** 

321

* Human 

322

323

Automatic Speech Recognition:

324

* [HuggingFace OpenASR Leaderboard evaluation sets](https://huggingface.co/spaces/hf-audio/open_asr_leaderboard)

325

* [MLS](https://huggingface.co/datasets/facebook/multilingual_librispeech)

326

* [MCV] (https://commonvoice.mozilla.org/en/datasets)

327

328

Automatic Speech Translation:

329

* [FLEURS](https://huggingface.co/datasets/google/fleurs)

330

* [COVOST-v2](https://github.com/facebookresearch/covost)

331

* [mExpresso](https://huggingface.co/facebook/seamless-expressive#mexpresso-multilingual-expresso)

332

333

Timestamp Prediction:

334

* [Librispeech](https://www.openslr.org/12)

335

336

Hallucination Robustness:

337

* [MUSAN](https://www.openslr.org/17/) 48 hrs eval set

338

339

Noise Robustness:

340

* [Librispeech](https://www.openslr.org/12)

341

342

Model Fairness:

343

* [Casual Conversations Dataset](https://arxiv.org/pdf/2104.02821)

344

345

## Training

346

347

canary-180m-flash is trained using the NVIDIA NeMo framework [7] for a total of 219K steps with 2D bucketing [1] and optimal batch sizes set using OOMptimizer [8]. The model is trained on 32 NVIDIA A100 80GB GPUs.

348

The model can be trained using this [example script](https://github.com/NVIDIA/NeMo/blob/main/examples/asr/speech_multitask/speech_to_text_aed.py) and [base config](https://github.com/NVIDIA/NeMo/blob/main/examples/asr/conf/speech_multitask/fast-conformer_aed.yaml).

349

350

The tokenizers for these models were built using the text transcripts of the train set with this [script](https://github.com/NVIDIA/NeMo/blob/main/scripts/tokenizers/process_asr_text_tokenizer.py).

351

352

## Inference:

353

**Engine:** NVIDIA NeMo 

354

**Test Hardware :** 

355

* A6000 

356

* A100 

357

* V100 

358

359

## Performance

360

361

For ASR and AST experiments, predictions were generated using greedy decoding. Note that utterances shorter than 1 second are symmetrically zero-padded upto 1 second during evaluation.

362

363

### English ASR Performance (w/o PnC)

364

365

The ASR performance is measured with word error rate (WER), and we process the groundtruth and predicted text with [whisper-normalizer](https://pypi.org/project/whisper-normalizer/).

366

367

WER on [HuggingFace OpenASR leaderboard](https://huggingface.co/spaces/hf-audio/open_asr_leaderboard) evaluated with a batch size of 128:

368

369

| **Version** | **Model**     | **RTFx**   | **AMI**   | **GigaSpeech**   | **LS Clean**   | **LS Other**   | **Earnings22**   | **SPGISpech**   | **Tedlium**   | **Voxpopuli**   |

370

|:---------:|:-----------:|:------:|:------:|:------:|:------:|:------:|:------:|:------:|:------:|:------:|

371

| main | canary-180m-flash | 1233 | 14.86 | 10.51 | 1.87 | 3.83 | 13.33 | 2.26 | 3.98 | 6.35 |

372

373

#### Inference speed on different systems

374

We profiled inference speed on the OpenASR benchmark using the [real-time factor](https://github.com/NVIDIA/DeepLearningExamples/blob/master/Kaldi/SpeechRecognition/README.md#metrics) (RTFx) to quantify throughput.

375

376

377

|:-----------:|:-------------:|:------------:|:----------:|

378

| main | canary-180m-flash | NVIDIA A100 | 1233 |

379

| main | canary-180m-flash | NVIDIA H100 | 2041 |

380

381

382

383

### Multilingual ASR Performance

384

WER on [MLS](https://huggingface.co/datasets/facebook/multilingual_librispeech) test set:

385

386

| **Version** | **Model** | **De** | **Es** | **Fr** |

387

|:---------:|:-----------:|:------:|:------:|:------:|

388

| main | canary-180m-flash | 4.81 | 3.17 | 4.75 |

389

390

391

WER on [MCV-16.1](https://commonvoice.mozilla.org/en/datasets) test set:

392

| **Version** | **Model** | **En** | **De** | **Es** | **Fr** |

393

|:---------:|:-----------:|:------:|:------:|:------:|:------:|

394

| main | canary-180m-flash | 9.53 | 5.94 | 4.90 | 8.19 |

395

396

397

More details on evaluation can be found at [HuggingFace ASR Leaderboard](https://huggingface.co/spaces/hf-audio/open_asr_leaderboard)

398

399

### AST Performance

400

401

We evaluate AST performance with [BLEU score](https://lightning.ai/docs/torchmetrics/stable/text/sacre_bleu_score.html), and use native annotations with punctuation and capitalization in the datasets.

402

403

[FLEURS](https://huggingface.co/datasets/google/fleurs) test set:

404

405

BLEU score:

406

407

408

|:-----------:|:---------:|:----------:|:----------:|:----------:|:----------:|:----------:|:----------:|

409

| main | canary-180m-flash | 28.18 | 20.47 | 36.66 | 32.08 | 20.09 | 29.75 |

410

411

COMET score:

412

413

414

|:-----------:|:---------:|:----------:|:----------:|:----------:|:----------:|:----------:|:----------:|

415

| main | canary-180m-flash | 77.56 | 78.10 | 78.53 | 83.03 | 81.48 | 82.28 |

416

417

[COVOST-v2](https://github.com/facebookresearch/covost) test set:

418

419

BLEU score:

420

421

422

|:-----------:|:---------:|:----------:|:----------:|:----------:|

423

| main | canary-180m-flash | 35.61 | 39.84 | 38.57 |

424

425

COMET score:

426

427

428

|:-----------:|:---------:|:----------:|:----------:|:----------:|

429

| main | canary-180m-flash | 80.94 | 84.54 | 82.50 |

430

431

[mExpresso](https://huggingface.co/facebook/seamless-expressive#mexpresso-multilingual-expresso) test set:

432

433

BLEU score:

434

435

436

|:-----------:|:---------:|:----------:|:----------:|:----------:|

437

| main | canary-180m-flash | 21.60 | 33.45 | 25.96 |

438

439

COMET score:

440

441

442

|:-----------:|:---------:|:----------:|:----------:|:----------:|

443

| main | canary-180m-flash | 77.71 | 80.87 | 77.82 |

444

445

446

### Timestamp Prediction

447

F1-score on [Librispeech Test sets](https://www.openslr.org/12) at collar value of 200ms

448

449

450

|:-----------:|:---------:|:----------:|:----------:|

451

| main | canary-180m-flash | 93.48 | 91.38 |

452

453

454

### Hallucination Robustness

455

Number of characters per minute on [MUSAN](https://www.openslr.org/17) 48 hrs eval set

456

457

| **Version** | **Model** | **# of character per minute** |

458

|:-----------:|:---------:|:----------:|

459

| main | canary-180m-flash | 91.52 |

460

461

### Noise Robustness

462

WER on [Librispeech Test Clean](https://www.openslr.org/12) at different SNR (signal to noise ratio) levels of additive white noise

463

464

| **Version** | **Model** | **SNR 10** | **SNR 5** | **SNR 0** | **SNR -5** |

465

|:-----------:|:---------:|:----------:|:----------:|:----------:|:----------:|

466

| main | canary-180m-flash | 3.23 | 5.34 | 12.21 | 34.03 |

467

468

## Model Fairness Evaluation

469

470

As outlined in the paper "Towards Measuring Fairness in AI: the Casual Conversations Dataset" [9], we assessed the canary-180m-flash model for fairness. The model was evaluated on the CausalConversations-v1 dataset, and the results are reported as follows:

471

472

### Gender Bias:

473

474

| Gender | Male | Female | N/A | Other |

475

| :--- | :--- | :--- | :--- | :--- |

476

| Num utterances | 19325 | 24532 | 926 | 33 |

477

| % WER | 16.92 | 14.01 | 20.01 | 25.04 |

478

479

### Age Bias:

480

481

| Age Group | (18-30) | (31-45) | (46-85) | (1-100) |

482

| :--- | :--- | :--- | :--- | :--- |

483

| Num utterances | 15956 | 14585 | 13349 | 43890 |

484

| % WER | 14.95 | 15.36 | 15.65 | 15.29 |

485

486

(Error rates for fairness evaluation are determined by normalizing both the reference and predicted text, similar to the methods used in the evaluations found at https://github.com/huggingface/open_asr_leaderboard.)

487

488

## License/Terms of Use:

489

canary-180m-flash is released under the CC-BY-4.0 license. By using this model, you are agreeing to the [terms and conditions](https://choosealicense.com/licenses/cc-by-4.0/) of the license. <br>

490

491

## References:

492

493

[1] [Training and Inference Efficiency of Encoder-Decoder Speech Models](https://arxiv.org/pdf/2503.05931)

494

495

[2] [Less is More: Accurate Speech Recognition & Translation without Web-Scale Data](https://www.isca-archive.org/interspeech_2024/puvvada24_interspeech.pdf) <br>

496

497

[3] [Fast Conformer with Linearly Scalable Attention for Efficient Speech Recognition](https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=10389701)

498

499

[4] [Attention is All You Need](https://arxiv.org/abs/1706.03762)

500

501

[5] [Unified Model for Code-Switching Speech Recognition and Language Identification Based on Concatenated Tokenizer](https://aclanthology.org/2023.calcs-1.7.pdf)

502

503

[6] [Google Sentencepiece Tokenizer](https://github.com/google/sentencepiece)

504

505

[7] [NVIDIA NeMo Framework](https://github.com/NVIDIA/NeMo)

506

507

[8] [EMMeTT: Efficient Multimodal Machine Translation Training](https://arxiv.org/abs/2409.13523)

508

509

[9] [Towards Measuring Fairness in AI: the Casual Conversations Dataset](https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=9634168)

510

511

512

## Ethical Considerations:

513

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications.  When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.

514

Please report security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).

515