README.md
4.3 KB · 121 lines · markdown Raw
1 ---
2 license: apache-2.0
3 library_name: gguf
4 pipeline_tag: text-to-speech
5 tags:
6 - tts
7 - text-to-speech
8 - voice-cloning
9 - voice-design
10 - mandarin-dialects
11 - ggml
12 - gguf
13 - qwen
14 - qwen3
15 - qwen3-tts
16 - cpp
17 language:
18 - zh
19 - en
20 - fr
21 - de
22 - es
23 - it
24 - pt
25 - ja
26 - ko
27 - ru
28 - ar
29 base_model:
30 - Qwen/Qwen3-TTS-Tokenizer-12Hz
31 - Qwen/Qwen3-TTS-12Hz-0.6B-Base
32 - Qwen/Qwen3-TTS-12Hz-0.6B-CustomVoice
33 - Qwen/Qwen3-TTS-12Hz-1.7B-Base
34 - Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice
35 - Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign
36 ---
37
38 # Qwen3-TTS GGUF
39
40 GGUF weights for [qwentts.cpp](https://github.com/ServeurpersoCom/qwentts.cpp),
41 a C++17/GGML port of Qwen3-TTS 12 Hz (Qwen team, Alibaba). Multilingual
42 zero shot TTS with named speakers and Mandarin dialects, 24 kHz mono.
43 Runs on CPU, CUDA, Metal, Vulkan.
44
45 ## Files
46
47 Two GGUFs load together :
48
49 qwen-talker-{size}-{mode}-{variant}.gguf Qwen3 LM + code predictor MTP head + optional speaker encoder, text -> 12 Hz codes
50 qwen-tokenizer-12hz-{variant}.gguf SEANet + ConvNeXt + DAC v2 + RVQ, 12 Hz codes <-> 24 kHz audio
51
52 Three modes are available across two talker sizes :
53
54 | mode | 0.6B | 1.7B | use case |
55 |-------------|------|------|---------------------------------------------------|
56 | base | yes | yes | zero shot TTS with named speakers and dialects |
57 | customvoice | yes | yes | zero shot voice cloning from a reference clip |
58 | voicedesign | no | yes | voice synthesis from attribute description |
59
60 The tokenizer is shared across every talker.
61
62 | variant | talker 0.6B | talker 1.7B | tokenizer | use case |
63 |---------|-------------|-------------|-----------|--------------------------------|
64 | F32 | 3.7 GB | 7.7 GB | 647 MB | reference, debug, conversion |
65 | BF16 | 1.8 GB | 3.9 GB | 359 MB | source faithful, max precision |
66 | Q8_0 | 993 MB | 2.1 GB | 291 MB | recommended default |
67 | Q4_K_M | 629 MB | 1.2 GB | 255 MB | lowest VRAM |
68
69 ## Quick start
70
71 ```
72 git clone --recurse-submodules https://github.com/ServeurpersoCom/qwentts.cpp.git
73 cd qwentts.cpp && ./buildcuda.sh
74 mkdir -p models
75 huggingface-cli download Serveurperso/Qwen3-TTS-GGUF \
76 qwen-talker-1.7b-base-Q8_0.gguf qwen-tokenizer-12hz-Q8_0.gguf \
77 --local-dir models
78 cd examples
79 ./base.sh # named speaker -> base.wav
80 ./clone.sh # voice cloning -> clone.wav
81 ./customvoice.sh # custom voice mode -> customvoice.wav
82 ./tts.sh # voice design -> tts.wav
83 ```
84
85 ## Backends
86
87 Set `GGML_BACKEND` to force a device, otherwise the runtime picks the
88 best one available.
89
90 | value | target |
91 |-----------|----------------------------------------------|
92 | `CUDA0` | NVIDIA GPU, fastest path on Ada / Blackwell |
93 | `Vulkan0` | Cross vendor GPU (AMD / Intel / NVIDIA) |
94 | `Metal` | Apple Silicon GPU |
95 | `CPU` | CPU fallback, x86 variant auto selected |
96
97 ## Quantization policy
98
99 Tokenizer GGUFs are not uniform quants. Three categories get a
100 dedicated treatment :
101
102 | tensor | dtype across all variants |
103 |--------------------------------------------------------------|---------------------------|
104 | RVQ codebooks, input_proj / output_proj, speaker encoder fc | F32 |
105 | 1D tensors (gamma, biases, norms, snake alpha and beta) | F32 |
106 | Conv kernels with non alignable rows (K=7,3,1) | F16 in Q* variants |
107
108 Conv kernel rows (K=7,3,1) never divide a K-quant block size, so the
109 quantizer skips the Q* intermediates and lands on F16 directly. This
110 is the last resort branch of llama.cpp's `tensor_type_fallback`
111 applied unconditionally for these kernels. F16 has no block size and
112 matches the runtime target dtype on every backend. The talker LM
113 (Qwen3 backbone, hidden divisible by 256) follows standard llama.cpp
114 K-quant across variants. The code predictor MTP head and the speaker
115 encoder live in the talker GGUF and share its quantization.
116
117 ## License
118
119 Upstream model : Qwen3-TTS by Alibaba / Qwen team, Apache 2.0
120 Audio codec : Qwen3-TTS-Tokenizer-12Hz (Qwen team), Apache 2.0
121 GGUF tooling : qwentts.cpp, MIT