README.md
8.2 KB · 207 lines · markdown Raw
1 ---
2 tags:
3 - unsloth
4 - qwen3
5 - qwen
6 base_model:
7 - Qwen/Qwen3-Coder-30B-A3B-Instruct
8 library_name: transformers
9 license: apache-2.0
10 license_link: https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct/blob/main/LICENSE
11 pipeline_tag: text-generation
12 ---
13 <div>
14 <p style="margin-bottom: 0; margin-top: 0;">
15 <strong>See <a href="https://huggingface.co/collections/unsloth/qwen3-680edabfb790c8c34a242f95">our collection</a> for all versions of Qwen3 including GGUF, 4-bit & 16-bit formats.</strong>
16 </p>
17 <p style="margin-bottom: 0;">
18 <em>Learn to run Qwen3-Coder correctly - <a href="https://docs.unsloth.ai/basics/qwen3-coder">Read our Guide</a>.</em>
19 </p>
20 <p style="margin-top: 0;margin-bottom: 0;">
21 <em>See <a href="https://docs.unsloth.ai/basics/unsloth-dynamic-v2.0-gguf">Unsloth Dynamic 2.0 GGUFs</a> for our quantization benchmarks.</em>
22 </p>
23 <div style="display: flex; gap: 5px; align-items: center; ">
24 <a href="https://github.com/unslothai/unsloth/">
25 <img src="https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png" width="133">
26 </a>
27 <a href="https://discord.gg/unsloth">
28 <img src="https://github.com/unslothai/unsloth/raw/main/images/Discord%20button.png" width="173">
29 </a>
30 <a href="https://docs.unsloth.ai/basics/qwen3-coder">
31 <img src="https://raw.githubusercontent.com/unslothai/unsloth/refs/heads/main/images/documentation%20green%20button.png" width="143">
32 </a>
33 </div>
34 <h1 style="margin-top: 0rem;">✨ Read our Qwen3-Coder Guide <a href="https://docs.unsloth.ai/basics/qwen3-coder">here</a>!</h1>
35 </div>
36
37 - Fine-tune Qwen3 (14B) for free using our Google [Colab notebook](https://docs.unsloth.ai/get-started/unsloth-notebooks)!
38 - Read our Blog about Qwen3 support: [unsloth.ai/blog/qwen3](https://unsloth.ai/blog/qwen3)
39 - View the rest of our notebooks in our [docs here](https://docs.unsloth.ai/get-started/unsloth-notebooks).
40 | Unsloth supports | Free Notebooks | Performance | Memory use |
41 |-----------------|--------------------------------------------------------------------------------------------------------------------------|-------------|----------|
42 | **Qwen3 (14B)** | [▶️ Start on Colab](https://docs.unsloth.ai/get-started/unsloth-notebooks) | 3x faster | 70% less |
43 | **GRPO with Qwen3 (8B)** | [▶️ Start on Colab](https://docs.unsloth.ai/get-started/unsloth-notebooks) | 3x faster | 80% less |
44 | **Llama-3.2 (3B)** | [▶️ Start on Colab](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Llama3.2_(1B_and_3B)-Conversational.ipynb) | 2.4x faster | 58% less |
45 | **Llama-3.2 (11B vision)** | [▶️ Start on Colab](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Llama3.2_(11B)-Vision.ipynb) | 2x faster | 60% less |
46 | **Qwen2.5 (7B)** | [▶️ Start on Colab](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Qwen2.5_(7B)-Alpaca.ipynb) | 2x faster | 60% less |
47
48 # Qwen3-Coder-30B-A3B-Instruct
49 <a href="https://chat.qwen.ai/" target="_blank" style="margin: 2px;">
50 <img alt="Chat" src="https://img.shields.io/badge/%F0%9F%92%9C%EF%B8%8F%20Qwen%20Chat%20-536af5" style="display: inline-block; vertical-align: middle;"/>
51 </a>
52
53 ## Highlights
54
55 **Qwen3-Coder** is available in multiple sizes. Today, we're excited to introduce **Qwen3-Coder-30B-A3B-Instruct**. This streamlined model maintains impressive performance and efficiency, featuring the following key enhancements:
56
57 - **Significant Performance** among open models on **Agentic Coding**, **Agentic Browser-Use**, and other foundational coding tasks.
58 - **Long-context Capabilities** with native support for **256K** tokens, extendable up to **1M** tokens using Yarn, optimized for repository-scale understanding.
59 - **Agentic Coding** supporting for most platform such as **Qwen Code**, **CLINE**, featuring a specially designed function call format.
60
61 ![image/jpeg](https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-Coder/qwen3-coder-30a3-main.jpg)
62
63 ## Model Overview
64
65 **Qwen3-Coder-30B-A3B-Instruct** has the following features:
66 - Type: Causal Language Models
67 - Training Stage: Pretraining & Post-training
68 - Number of Parameters: 30.5B in total and 3.3B activated
69 - Number of Layers: 48
70 - Number of Attention Heads (GQA): 32 for Q and 4 for KV
71 - Number of Experts: 128
72 - Number of Activated Experts: 8
73 - Context Length: **262,144 natively**.
74
75 **NOTE: This model supports only non-thinking mode and does not generate ``<think></think>`` blocks in its output. Meanwhile, specifying `enable_thinking=False` is no longer required.**
76
77 For more details, including benchmark evaluation, hardware requirements, and inference performance, please refer to our [blog](https://qwenlm.github.io/blog/qwen3-coder/), [GitHub](https://github.com/QwenLM/Qwen3-Coder), and [Documentation](https://qwen.readthedocs.io/en/latest/).
78
79
80 ## Quickstart
81
82 We advise you to use the latest version of `transformers`.
83
84 With `transformers<4.51.0`, you will encounter the following error:
85 ```
86 KeyError: 'qwen3_moe'
87 ```
88
89 The following contains a code snippet illustrating how to use the model generate content based on given inputs.
90 ```python
91 from transformers import AutoModelForCausalLM, AutoTokenizer
92
93 model_name = "Qwen/Qwen3-Coder-30B-A3B-Instruct"
94
95 # load the tokenizer and the model
96 tokenizer = AutoTokenizer.from_pretrained(model_name)
97 model = AutoModelForCausalLM.from_pretrained(
98 model_name,
99 torch_dtype="auto",
100 device_map="auto"
101 )
102
103 # prepare the model input
104 prompt = "Write a quick sort algorithm."
105 messages = [
106 {"role": "user", "content": prompt}
107 ]
108 text = tokenizer.apply_chat_template(
109 messages,
110 tokenize=False,
111 add_generation_prompt=True,
112 )
113 model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
114
115 # conduct text completion
116 generated_ids = model.generate(
117 **model_inputs,
118 max_new_tokens=65536
119 )
120 output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist()
121
122 content = tokenizer.decode(output_ids, skip_special_tokens=True)
123
124 print("content:", content)
125 ```
126
127 **Note: If you encounter out-of-memory (OOM) issues, consider reducing the context length to a shorter value, such as `32,768`.**
128
129 For local use, applications such as Ollama, LMStudio, MLX-LM, llama.cpp, and KTransformers have also supported Qwen3.
130
131 ## Agentic Coding
132
133 Qwen3-Coder excels in tool calling capabilities.
134
135 You can simply define or use any tools as following example.
136 ```python
137 # Your tool implementation
138 def square_the_number(num: float) -> dict:
139 return num ** 2
140
141 # Define Tools
142 tools=[
143 {
144 "type":"function",
145 "function":{
146 "name": "square_the_number",
147 "description": "output the square of the number.",
148 "parameters": {
149 "type": "object",
150 "required": ["input_num"],
151 "properties": {
152 'input_num': {
153 'type': 'number',
154 'description': 'input_num is a number that will be squared'
155 }
156 },
157 }
158 }
159 }
160 ]
161
162 import OpenAI
163 # Define LLM
164 client = OpenAI(
165 # Use a custom endpoint compatible with OpenAI API
166 base_url='http://localhost:8000/v1', # api_base
167 api_key="EMPTY"
168 )
169
170 messages = [{'role': 'user', 'content': 'square the number 1024'}]
171
172 completion = client.chat.completions.create(
173 messages=messages,
174 model="Qwen3-Coder-30B-A3B-Instruct",
175 max_tokens=65536,
176 tools=tools,
177 )
178
179 print(completion.choice[0])
180 ```
181
182 ## Best Practices
183
184 To achieve optimal performance, we recommend the following settings:
185
186 1. **Sampling Parameters**:
187 - We suggest using `temperature=0.7`, `top_p=0.8`, `top_k=20`, `repetition_penalty=1.05`.
188
189 2. **Adequate Output Length**: We recommend using an output length of 65,536 tokens for most queries, which is adequate for instruct models.
190
191
192 ### Citation
193
194 If you find our work helpful, feel free to give us a cite.
195
196 ```
197 @misc{qwen3technicalreport,
198 title={Qwen3 Technical Report},
199 author={Qwen Team},
200 year={2025},
201 eprint={2505.09388},
202 archivePrefix={arXiv},
203 primaryClass={cs.CL},
204 url={https://arxiv.org/abs/2505.09388},
205 }
206 ```
207