README.md · canary-1b-v2-gguf

1

---

2

license: cc-by-4.0

3

base_model: nvidia/canary-1b-v2

4

base_model_relation: quantized

5

library_name: transcribe.cpp

6

pipeline_tag: automatic-speech-recognition

7

language:

8

- bg

9

- hr

10

- cs

11

- da

12

- nl

13

- en

14

- et

15

- fi

16

- fr

17

- de

18

- el

19

- hu

20

- it

21

- lv

22

- lt

23

- mt

24

- pl

25

- pt

26

- ro

27

- sk

28

- sl

29

- es

30

- sv

31

- ru

32

- uk

33

tags:

34

- gguf

35

- transcribe.cpp

36

- asr

37

- speech-to-text

38

- canary

39

- multitask-aed

40

- encoder-decoder

41

- multilingual

42

- translation

43

transcribe_cpp:

44

schema_version: 2

45

wer_fleurs_bg:

46

q8_0: 9.22

47

wer_fleurs_cs:

48

q8_0: 8.56

49

wer_fleurs_da:

50

q8_0: 11.34

51

wer_fleurs_de:

52

q8_0: 4.46

53

wer_fleurs_el:

54

q8_0: 26.02

55

wer_fleurs_en:

56

q8_0: 4.47

57

wer_fleurs_es:

58

q8_0: 3.1

59

wer_fleurs_et:

60

q8_0: 12.72

61

wer_fleurs_fi:

62

q8_0: 8.86

63

wer_fleurs_fr:

64

q8_0: 5.09

65

wer_fleurs_hr:

66

q8_0: 8.4

67

wer_fleurs_hu:

68

q8_0: 13.06

69

wer_fleurs_it:

70

q8_0: 3.1

71

wer_fleurs_lt:

72

q8_0: 13.45

73

wer_fleurs_lv:

74

q8_0: 10.41

75

wer_fleurs_mt:

76

q8_0: 19.75

77

wer_fleurs_nl:

78

q8_0: 6.28

79

wer_fleurs_pl:

80

q8_0: 6.88

81

wer_fleurs_pt:

82

q8_0: 4.5

83

wer_fleurs_ro:

84

q8_0: 6.87

85

wer_fleurs_ru:

86

q8_0: 7.83

87

wer_fleurs_sk:

88

q8_0: 6.84

89

wer_fleurs_sl:

90

q8_0: 12.76

91

wer_fleurs_sv:

92

q8_0: 9.74

93

wer_fleurs_uk:

94

q8_0: 10.58

95

wer_librispeech_test_clean:

96

f32: 1.92

97

f16: 1.92

98

q8_0: 1.91

99

q6_k: 1.94

100

q5_k_m: 1.93

101

q4_k_m: 1.91

102

rtf_m4_max:

103

cpu: 25.08

104

metal: 98.25

105

rtf_ryzen_4750u:

106

cpu: 8.32

107

vulkan: 14.13

108

streaming: false

109

translate: true

110

lang_detect: false

111

timestamps: none

112

---

113

114

# canary-1b-v2: transcribe.cpp GGUF

115

116

GGUF conversions of [nvidia/canary-1b-v2](https://huggingface.co/nvidia/canary-1b-v2) for use

117

with [transcribe.cpp](https://github.com/handy-computer/transcribe.cpp).

118

119

Ported from upstream commit

120

[87bc526](https://huggingface.co/nvidia/canary-1b-v2/commit/87bc526),

121

pinned 2026-05-08.

122

Validated against the NeMo reference at transcribe.cpp commit

123

[db53eda](https://github.com/handy-computer/transcribe.cpp/tree/db53eda)

124

on 2026-05-08.

125

126

Offline multilingual speech-to-text and translation across 25 European

127

languages. A multitask AED with a 32-layer FastConformer encoder and an

128

8-layer Transformer decoder. Supports automatic speech recognition for

129

any of the 25 supported languages, plus translation between supported

130

language pairs (per the upstream model card). Takes a 16 kHz mono WAV and

131

produces a transcript. Not a streaming model; word and segment timestamps

132

from the upstream model are not exposed in the v1 port.

133

134

135

## Downloads

136

137

138

| --- | --- | ---: | ---: |

139

| F32 | [canary-1b-v2-F32.gguf](https://huggingface.co/handy-computer/canary-1b-v2-gguf/resolve/main/canary-1b-v2-F32.gguf) | 3.92 GB | 1.92% |

140

| F16 | [canary-1b-v2-F16.gguf](https://huggingface.co/handy-computer/canary-1b-v2-gguf/resolve/main/canary-1b-v2-F16.gguf) | 1.97 GB | 1.92% |

141

| Q8_0 | [canary-1b-v2-Q8_0.gguf](https://huggingface.co/handy-computer/canary-1b-v2-gguf/resolve/main/canary-1b-v2-Q8_0.gguf) | 1.14 GB | 1.91% |

142

| Q6_K | [canary-1b-v2-Q6_K.gguf](https://huggingface.co/handy-computer/canary-1b-v2-gguf/resolve/main/canary-1b-v2-Q6_K.gguf) | 932 MB | 1.94% |

143

| Q5_K_M | [canary-1b-v2-Q5_K_M.gguf](https://huggingface.co/handy-computer/canary-1b-v2-gguf/resolve/main/canary-1b-v2-Q5_K_M.gguf) | 837 MB | 1.93% |

144

| Q4_K_M | [canary-1b-v2-Q4_K_M.gguf](https://huggingface.co/handy-computer/canary-1b-v2-gguf/resolve/main/canary-1b-v2-Q4_K_M.gguf) | 735 MB | 1.91% |

145

146

WER on the full LibriSpeech test-clean split (2,620 utterances), batch size 1, timestamps none. Figures without a commit were published before provenance was recorded.

147

148

Greedy decoding, no external LM. F32 reference baseline: 1.92%. NVIDIA's

149

self-reported number on the upstream model card is 2.18%; our F32 port comes in

150

slightly under the upstream-reported number (Δ −0.26pp) and is likely down to

151

scoring differences.

152

153

154

## Usage

155

156

Build transcribe.cpp from source:

157

158

```bash

159

git clone git@github.com:handy-computer/transcribe.cpp.git

160

cd transcribe.cpp

161

cmake -B build && cmake --build build

162

```

163

164

Run on a 16 kHz mono WAV:

165

166

```bash

167

build/bin/transcribe-cli \

168

-m canary-1b-v2-Q8_0.gguf \

169

input.wav

170

```

171

172

If your audio isn't already 16 kHz mono WAV, convert it first:

173

174

```bash

175

ffmpeg -i input.mp3 -ar 16000 -ac 1 output.wav

176

```

177

178

See the [transcribe.cpp model page](https://github.com/handy-computer/transcribe.cpp/blob/main/docs/models/canary-1b-v2.md) for performance

179

numbers, numerical validation, and reproduction steps.

180

181

## License

182

183

Inherited from the base model: **CC-BY-4.0**. See the

184

[upstream model card](https://huggingface.co/nvidia/canary-1b-v2) for full terms.

185

This repository includes the license text

186

([`LICENSE`](https://huggingface.co/handy-computer/canary-1b-v2-gguf/blob/main/LICENSE)) and a

187

[`NOTICE`](https://huggingface.co/handy-computer/canary-1b-v2-gguf/blob/main/NOTICE) recording

188

how these files were derived from the original.

189

190

---

191

192

## Original Model Card

193

194

> The section below is reproduced from

195

> [nvidia/canary-1b-v2](https://huggingface.co/nvidia/canary-1b-v2) at commit

196

> `87bc526` for offline reference. The upstream card is the

197

> authoritative source.

198

199

## 🐤 Canary 1B v2: Multitask Speech Transcription and Translation Model 

200

201

202

**``Canary-1b-v2``** is a powerful 1-billion parameter model built for high-quality speech transcription and translation across 25 European languages.

203

204

It excels at both automatic speech recognition (ASR) and speech translation (AST), supporting:

205

206

* **Speech Transcription (ASR) for 25 languages**

207

* **Speech Translation (AST) from English → 24 languages**

208

* **Speech Translation (AST) from 24 languages → English**

209

210

211

**Supported Languages:**

212

Bulgarian (**bg**), Croatian (**hr**), Czech (**cs**), Danish (**da**), Dutch (**nl**), English (**en**), Estonian (**et**), Finnish (**fi**), French (**fr**), German (**de**), Greek (**el**), Hungarian (**hu**), Italian (**it**), Latvian (**lv**), Lithuanian (**lt**), Maltese (**mt**), Polish (**pl**), Portuguese (**pt**), Romanian (**ro**), Slovak (**sk**), Slovenian (**sl**), Spanish (**es**), Swedish (**sv**), Russian (**ru**), Ukrainian (**uk**)

213

214

🗣️ **Experience `Canary-1b-v2` in action** at [Hugging Face Demo](https://huggingface.co/spaces/nvidia/canary-1b-v2)

215

216

`Canary-1b-v2` model is ready for commercial/non-commercial use.

217

218

219

## License/Terms of Use:

220

221

GOVERNING TERMS: Use of this model is governed by the [CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/legalcode.en) license.

222

223

## Discover more from NVIDIA:

224

For documentation, deployment guides, enterprise-ready APIs, and the latest open models—including Nemotron and other cutting-edge speech, translation, and generative AI—visit the NVIDIA Developer Portal at [developer.nvidia.com](https://developer.nvidia.com).

225

Join the community to access tools, support, and resources to accelerate your development with NVIDIA’s NeMo, Riva, NIM, and foundation models.<br>

226

227

### Explore more from NVIDIA: 

228

What is [Nemotron](https://www.nvidia.com/en-us/ai-data-science/foundation-models/nemotron/)? 

229

NVIDIA Developer [Nemotron](https://developer.nvidia.com/nemotron) 

230

[NVIDIA Riva Speech](https://developer.nvidia.com/riva?sortBy=developer_learning_library%2Fsort%2Ffeatured_in.riva%3Adesc%2Ctitle%3Aasc#demos)<br>

231

[NeMo Documentation](https://docs.nvidia.com/nemo-framework/user-guide/latest/nemotoolkit/asr/models.html) 

232

233

234

## Key Features

235

236

**`Canary-1b-v2`** is a scaled and enhanced version of the Canary model family, offering:

237

238

* Support for **25 European languages**, expanding from the **4 languages** in [canary-1b](https://huggingface.co/nvidia/canary-1b)/[canary-1b-flash](nvidia/canary-1b-flash) to **21 additional languages**

239

* **State-of-the-art performance** among models of similar size

240

* **Comparable quality to models 3× larger**, while being up to **10× faster**

241

* Automatic **punctuation** and **capitalization**

242

* Accurate **word-level** and **segment-level** timestamps

243

* Segment-level timestamps also available for **translated outputs**

244

* Released under a **permissive CC BY 4.0 license**

245

246

`Canary-1b-v2` model is the first model from NeMo team that leveraged full Nvidia's Granary dataset \[1] \[2], showcasing its multitask and multilingual capabilities.

247

248

For full details on the model architecture, training methodology, datasets, and evaluation results, check out the **[Canary-1b-v2 Technical Report](https://arxiv.org/abs/2509.14128)**.

249

250

For a deeper glimpse into the Canary family of models, explore this comprehensive [NeMo tutorial on multitask speech models](https://github.com/NVIDIA/NeMo/blob/main/tutorials/asr/Canary_Multitask_Speech_Model.ipynb).

251

252

253

254

255

### Automatic Speech Recognition (ASR)

256

257

![ASR WER Comparison](plots/asr.png)

258

259

*Figure 1: ASR WER comparison across different models. This does not include Punctuation and Capitalisation errors.*

260

261

---

262

263

### Speech Translation (AST)

264

265

#### X → English

266

267

![AST X-En Comparison](plots/x_en.png)

268

269

*Figure 2: AST X → En COMET scores comparison across different models*

270

271

#### English → X

272

273

274

![AST En-X Comparison](plots/en_x.png)

275

276

*Figure 3: AST En → X COMET scores comparison across different models*

277

278

---

279

280

### Evaluation Notes

281

282

**Note 1:** The above evaluations are conducted in two settings: (1) **All supported languages** (24 languages, excluding Latvian since `seamless-m4t-v2-large` and `seamless-m4t-medium` do not support it), and (2) **Common languages** (6 languages supported by all compared models: en, fr, de, it, pt, es).

283

284

**Note 2:** Performance differences may be partly attributed to Portuguese variant differences - our training data uses European Portuguese while most benchmarks use Brazilian Portuguese.

285

286

---

287

288

289

## Deployment Geography

290

291

Global

292

293

## Use case

294

295

This model serves developers, researchers, academics, and industries building applications that require speech-to-text capabilities, including but not limited to: conversational AI, voice assistants, transcription services, subtitle generation, and voice analytics platforms.

296

297

## Release Date

298

299

Huggingface [08/14/2025](https://huggingface.co/nvidia/canary-1b-v2)

300

301

## Model Architecture

302

303

`Canary-1b-v2` is an encoder-decoder architecture featuring a FastConformer Encoder \[3] and a Transformer Decoder \[4]. The model extracts audio features through the encoder and uses task-specific tokens—such as `<source language>` and `<target language>`—to guide the Transformer Decoder in generating text output.

304

305

It uses a unified SentencePiece Tokenizer \[5] with a vocabulary of **16,384 tokens**, optimized across all 25 supported languages. The architecture includes **32 encoder layers** and **8 decoder layers**, totaling **978 million parameters**.

306

307

For implementation details, see the [NeMo repository](https://github.com/NVIDIA/NeMo).

308

309

310

## Input

311

- **Input Type(s):** 16kHz Audio

312

- **Input Format(s):** `.wav` and `.flac` audio formats

313

- **Input Parameters:** 1D (audio signal)

314

- **Other Properties Related to Input:** Monochannel audio

315

316

## Output

317

- **Output Type(s):** Text

318

- **Output Format:** String

319

- **Output Parameters:** 1D (text)

320

- **Other Properties Related to Output:** Punctuation and Capitalization included.

321

322

Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA's hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions.

323

324

## How to Use This Model

325

326

327

To train, fine-tune or play with the model you will need to install [NVIDIA NeMo](https://github.com/NVIDIA/NeMo) \[6]. We recommend you install it after you've installed latest PyTorch version.

328

```bash

329

pip install -U nemo_toolkit['asr']

330

```

331

The model is available for use in the NeMo toolkit [6], and can be used as a pre-trained checkpoint for inference or for fine-tuning on another dataset.

332

333

#### Automatically instantiate the model

334

335

```python

336

from nemo.collections.asr.models import ASRModel

337

asr_ast_model = ASRModel.from_pretrained(model_name="nvidia/canary-1b-v2")

338

```

339

340

#### Transcribing using Python

341

First, let's get a sample:

342

```bash

343

wget https://dldata-public.s3.us-east-2.amazonaws.com/2086-149220-0033.wav

344

```

345

Then simply do:

346

```python

347

output = asr_ast_model.transcribe(['2086-149220-0033.wav'], source_lang='en', target_lang='en')

348

print(output[0].text)

349

```

350

351

#### Translating using Python

352

353

Be sure to specify necessary `target_lang` for proper translation:

354

355

```python

356

output = asr_ast_model.transcribe(['2086-149220-0033.wav'], source_lang='en', target_lang='fr')

357

print(output[0].text)

358

```

359

360

#### Transcribing with timestamps

361

362

> **Note:** Use [main branch of NeMo](https://github.com/NVIDIA/NeMo/) to get timestamps until it is released in NeMo 2.5.

363

364

To transcribe with timestamps:

365

```python

366

output = asr_model.transcribe(['2086-149220-0033.wav'], source_lang='en', target_lang='en', timestamps=True)

367

# by default, timestamps are enabled for word and segment level

368

word_timestamps = output[0].timestamp['word'] # word level timestamps for first sample

369

segment_timestamps = output[0].timestamp['segment'] # segment level timestamps

370

371

for stamp in segment_timestamps:

372

print(f"{stamp['start']}s - {stamp['end']}s : {stamp['segment']}")

373

```

374

375

#### Translating with timestamps

376

377

To translate with timestamps:

378

```python

379

output = asr_model.transcribe(['2086-149220-0033.wav'], source_lang='en', target_lang='fr', timestamps=True)

380

381

segment_timestamps = output[0].timestamp['segment'] # only supports segment level timestamps for translation

382

383

for stamp in segment_timestamps:

384

print(f"{stamp['start']}s - {stamp['end']}s : {stamp['segment']}")

385

```

386

387

For translation task, please, refer to segment-level timestamps for getting intuitive and accurate alignment.

388

389

> **Note:** If timestamps are not required for your work, you can reduce memory usage by restoring only the `.nemo` file without the auxiliary CTC model. To do this, extract the `.nemo` file, remove any *timestamps_asr_model* files, then repackage it into a new `.nemo` file.

390

391

392

## Software Integration

393

394

**Runtime Engine(s):**

395

396

* NeMo main branch (until it is released in NeMo 2.5)

397

398

**Supported Hardware Microarchitecture Compatibility:**

399

400

* NVIDIA Ampere

401

* NVIDIA Blackwell

402

* NVIDIA Hopper

403

404

**\[Preferred/Supported] Operating System(s):**

405

406

* Linux

407

408

**Hardware Specific Requirements:**

409

At least 6GB RAM for model to load.

410

411

#### Model Version

412

413

Current version: `Canary-1b-v2`. Previous versions can be [accessed](https://huggingface.co/collections/nvidia/canary-65c3b83ff19b126a3ca62926) here.

414

415

416

417

## Training and Evaluation Datasets

418

419

### Training

420

421

The model was trained using the NeMo toolkit \[4], following a 3-stage training procedure:

422

423

* Initialized from a 4-language ASR model

424

* Stage 1: Trained for 150,000 steps on X→En and English ASR tasks using 64 A100 GPUs

425

* Stage 2: Trained for 115,000 additional steps on the full dataset (ASR, X→En, En→X)

426

* Stage 3: Fine-tuned for 10,000 steps on a language-balanced high-quality subset of Granary and NeMo ASR Set 3.0

427

428

For all the stages of training, both languages and corpora are weighted using temperature sampling (τ = 0.5).

429

430

Training script: [speech\_to\_text\_aed.py](https://github.com/NVIDIA/NeMo/blob/main/examples/asr/speech_multitask/speech_to_text_aed.py)

431

432

Tokenizer script: [process\_asr\_text\_tokenizer.py](https://github.com/NVIDIA/NeMo/blob/main/scripts/tokenizers/process_asr_text_tokenizer.py)

433

434

---

435

436

### Training Dataset

437

438

`Canary-1b-v2` was trained on a massive multilingual speech recognition and translation dataset combining Nvidia's newly published [Granary](https://huggingface.co/datasets/nvidia/Granary) and in-house dataset NeMo ASR Set 3.0.

439

440

**Granary Dataset \[5] \[6] with improved pseudo-labels and efficiently filtered versions of the following corpora:**

441

442

* [YTC](https://huggingface.co/datasets/FBK-MT/mosel) \[7]

443

* [MOSEL](https://huggingface.co/datasets/FBK-MT/mosel) \[8]

444

* [YODAS](https://huggingface.co/datasets/espnet/yodas-granary) \[9]

445

446

Granary is now available on [Hugging Face](https://huggingface.co/datasets/nvidia/Granary).

447

448

To read more about the pseudo-labeling technique and [pipeline](https://github.com/NVIDIA/NeMo-speech-data-processor/tree/main/dataset_configs/multilingual/granary), please refer to the [Granary Paper](https://arxiv.org/abs/2505.13404).

449

450

**NeMo ASR Set 3.0 including human-labeled transcriptions from the following corpora:**

451

452

* Multilingual LibriSpeech (MLS)

453

* Mozilla Common Voice (v7.0)

454

* AMI (70 hrs)

455

* Fleurs

456

* LibriSpeech (960 hours)

457

* Fisher Corpus

458

* National Speech Corpus Part 1

459

* VCTK

460

* Europarl-ASR

461

462

463

464

**Total training hours:** 1.7M

465

466

* ASR: 660,000 hrs

467

* X→En: 360,000 hrs

468

* En→X: 690,000 hrs

469

* Non-speech: 36,000 hrs

470

471

All transcripts include punctuation and capitalization.

472

473

474

**Data Collection Method by dataset**

475

476

* Hybrid: Automated, Human

477

478

**Labeling Method by dataset**

479

480

* Hybrid: Synthetic, Human

481

482

483

---

484

485

### Evaluation Dataset

486

487

* Fleurs \[10], MLS \[11], CoVoST \[12]

488

* Hugging Face Open ASR Leaderboard \[13]

489

* Earnings-22 \[14], This American Life \[15] (long-form)

490

* MUSAN \[16]

491

492

**Data Collection Method by dataset**

493

494

* Human

495

496

**Labeling Method by dataset**

497

498

* Human

499

500

## Benchmark Results

501

502

This section reports the evaluation results of the ``Canary-1b-v2`` model across multiple tasks, including Automatic Speech Recognition (ASR), Speech Translation (AST), robustness to noise, and long-form transcription.

503

504

---

505

506

### Automatic Speech Recognition (ASR)

507

508

| **WER ↓** | Fleurs-25 Langs | CoVoST-13 Langs | MLS - 6 Langs |

509

| --------------- | -------------------- | -------------------- | ------------------ |

510

| **`Canary-1b-v2`** | 8.40% | 8.85% | 7.27% |

511

512

513

**Note:** Presented WERs do not include Punctuation and Capitalization errors.

514

515

---

516

517

#### Hugging Face Open ASR Leaderboard

518

519

|  **WER ↓**   | **RTFx**   | **Mean**   | **AMI**   | **GigaSpeech**   | **LS Clean**   | **LS Other**   | **Earnings22**   | **SPGISpech**   | **Tedlium**   | **Voxpopuli**   |

520

|:-----------:|:------:|:------:|:------:|:------:|:------:|:------:|:------:|:------:|:------:|:------:|

521

| `Canary-1b-v2` | 749 | 7.15 | 16.01 | 10.82 | 2.18 | 3.56 | 11.79 | 2.28 | 4.29 | 6.25 |

522

523

More details on evaluation can be found at [HuggingFace ASR Leaderboard](https://huggingface.co/spaces/hf-audio/open_asr_leaderboard)

524

525

---

526

527

### Speech Translation (AST)

528

529

#### X → English

530

531

| | **COMET ↑** | | **BLEU ↑** | |

532

| --------------- | --------------- | --------------- | --------------- | -------------- |

533

534

| **`Canary-1b-v2`** | 79.30 | 77.48 | 29.08 | 40.48 |

535

536

537

538

#### English → X

539

540

| | **COMET ↑** | | **BLEU ↑** | |

541

| --------------- | ------------- | --------------- | --------------- | -------------- |

542

543

| **`Canary-1b-v2`** | 84.56 | 80.29 | 29.4 | 32.33 |

544

545

---

546

547

548

### Noise Robustness

549

550

Performance across different Signal-to-Noise Ratios (SNR) using MUSAN music and noise samples \[16] on the [LibriSpeech Clean test set](https://www.openslr.org/12).

551

**Metric**: Word Error Rate (**WER**)

552

553

| **SNR (dB)** | 100 | 10 | 5 | 0 | -5 |

554

| --------------- | ----- | ----- | ----- | ----- | ----- |

555

| **`Canary-1b-v2`** | 2.18% | 2.29% | 2.80% | 5.08% | 19.38% |

556

557

558

### Hallucination Robustness

559

Number of characters per minute on [MUSAN](https://www.openslr.org/17) \[16] 48 hrs eval set:

560

| | **# of character per minute ↓** |

561

|:---------:|:----------:|

562

| **`Canary-1b-v2`** | 134.7 |

563

564

565

---

566

567

### Long-form Inference

568

569

`Canary-1b-v2` achieves strong performance on long-form transcription by using dynamic chunking with 1-second overlap between chunks, allowing for efficient parallel processing. This dynamic chunking feature is automatically enabled when calling `.transcribe()` on a single audio file, or when using `batch_size=1` with multiple audio files that are longer than 40 seconds.

570

571

| **Dataset** | **WER ↓** |

572

| ----------------------- | --------- |

573

| Earnings-22 | 13.78% |

574

| This American Life | 9.87% |

575

576

**Note:** Presented WERs do not include Punctuation and Capitalization errors.

577

578

579

---

580

581

## Inference

582

583

**Engine**:

584

585

* NVIDIA NeMo

586

587

**Test Hardware**:

588

589

* NVIDIA A10

590

* NVIDIA A100

591

* NVIDIA A30

592

* NVIDIA A5000

593

* NVIDIA H100

594

* NVIDIA L4

595

* NVIDIA L40

596

597

---

598

599

## Ethical Considerations

600

601

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their supporting model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.

602

603

For more detailed information on ethical considerations for this model, please see the Model Card++ Explainability, Bias, Safety & Security, and Privacy Subcards [here](https://developer.nvidia.com/blog/enhancing-ai-transparency-and-ethical-considerations-with-model-card/).

604

605

Please report security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).

606

607

## Bias:

608

609

Field | Response

610

:---------------------------------------------------------------------------------------------------|:---------------:

611

Participation considerations from adversely impacted groups [protected classes](https://www.senate.ca.gov/content/protected-classes) in model design and testing  |  None

612

Measures taken to mitigate against unwanted bias | None

613

614

## Explainability:

615

616

Field | Response

617

:------------------------------------------------------------------------------------------------------:|:---------------------------------------------------------------------------------:

618

Intended Domain                                                                   |  Speech to Text Transcription and Translation

619

Model Type                                                                                            |  Attention Encoder-Decoder

620

Intended Users                                                                                        |  This model is intended for developers, researchers, academics, and industries building conversational based applications.

621

Output | Text

622

Describe how the model works                                                                          |  Speech input is encoded into embeddings and passed into conformer-based model and output a text response.

623

Name the adversely impacted groups this has been tested to deliver comparable outcomes regardless of | Not Applicable

624

Technical Limitations & Mitigation                                                                    |  Transcripts and translations may be not 100% accurate. Accuracy varies based on source and target language and characteristics of input audio (Domain, Use Case, Accent, Noise, Speech Type, Context of speech, etc.)

625

Verified to have met prescribed NVIDIA quality standards | Yes

626

Performance Metrics                                                                                   | Word Error Rate (Speech Transcription) / BLEU score (Speech Translation) / COMET score (Speech Translation)

627

Potential Known Risks                                                                                 |  If a word is not trained in the language model and not presented in vocabulary, the word is not likely to be recognized. Not recommended for word-for-word/incomplete sentences as accuracy varies based on the context of input text

628

Licensing                                                                                             |  GOVERNING TERMS: Use of this model is governed by the [CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/legalcode.en) license.

629

630

## Privacy:

631

632

Field                                                                                                                              |  Response

633

:----------------------------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------:

634

Generatable or reverse engineerable personal data? | None

635

Personal data used to create this model?                                                                                       |  None

636

Is there provenance for all datasets used in training?                                                                                |  Yes

637

Does data labeling (annotation, metadata) comply with privacy laws?                                                                |  Yes

638

Is data compliant with data subject requests for data correction or removal, if such a request was made?                           |  No, not possible with externally-sourced data.

639

Applicable Privacy Policy | https://www.nvidia.com/en-us/about-nvidia/privacy-policy/

640

641

## Safety:

642

643

Field | Response

644

:---------------------------------------------------:|:----------------------------------

645

Model Application(s) | Speech to Text Transcription

646

Describe the life critical impact | None

647

Use Case Restrictions                              | Abide by [CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/legalcode.en) License

648

Model and dataset restrictions            |  The Principle of least privilege (PoLP) is applied limiting access for dataset generation and model development. Restrictions enforce dataset access during training, and dataset license constraints adhered to.

649

650

651

## References

652

653

\[1] [Granary: Speech Recognition and Translation Dataset in 25 European Languages](https://arxiv.org/abs/2505.13404)

654

655

\[2] [NVIDIA Granary Dataset Card](https://huggingface.co/datasets/nvidia/Granary)

656

657

\[3] [Fast Conformer with Linearly Scalable Attention for Efficient Speech Recognition](https://arxiv.org/abs/2305.05084)

658

659

\[4] [Attention is All You Need](https://arxiv.org/abs/1706.03762)

660

661

\[5] [Google Sentencepiece Tokenizer](https://github.com/google/sentencepiece)

662

663

\[6] [NVIDIA NeMo Toolkit](https://github.com/NVIDIA/NeMo)

664

665

\[7] [Youtube-Commons](https://huggingface.co/datasets/PleIAs/YouTube-Commons)

666

667

\[8] [MOSEL: 950,000 Hours of Speech Data for Open-Source Speech Foundation Model Training on EU Languages](https://arxiv.org/abs/2410.01036)

668

669

\[9] [YODAS: Youtube-Oriented Dataset for Audio and Speech](https://arxiv.org/pdf/2406.00899)

670

671

\[10] [FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech](https://arxiv.org/abs/2205.12446)

672

673

\[11] [MLS: A Large-Scale Multilingual Dataset for Speech Research](https://arxiv.org/abs/2012.03411)

674

675

\[12] [CoVoST 2 and Massively Multilingual Speech-to-Text Translation](https://arxiv.org/abs/2007.10310)

676

677

\[13] [HuggingFace Open ASR Leaderboard](https://huggingface.co/spaces/hf-audio/open_asr_leaderboard)

678

679

\[14] [Earnings-22 Benchmark](https://github.com/revdotcom/speech-datasets/tree/main/earnings22)

680

681

\[15] [Speech Recognition and Multi-Speaker Diarization of Long Conversations](https://arxiv.org/abs/2005.08072)

682

683

\[16] [MUSAN: A Music, Speech, and Noise Corpus](https://arxiv.org/abs/1510.08484)

684