README.md
22.5 KB · 368 lines · markdown Raw
1 ---
2 language:
3 - en
4 library_name: transformers
5 pipeline_tag: text-generation
6 base_model: HuggingFaceTB/SmolLM3-3B
7 license: other
8 license_name: webai-non-commercial-license-ver.-1.0
9 license_link: https://huggingface.co/webAI-Official/webAI-ColVec1-4b/blob/main/LICENSE.md
10 tags:
11 - formal-logic
12 - reasoning
13 - lora
14 - model-merging
15 - wise-ft
16 - reinforcement-learning
17 - grpo
18 - smollm3
19 - twil-lm
20 ---
21
22 # TwIL-LM3
23
24 A 3B reasoning model for **formal logic** tasks, built from
25 [`HuggingFaceTB/SmolLM3-3B`](https://huggingface.co/HuggingFaceTB/SmolLM3-3B) through LoRA
26 supervised fine-tuning, checkpoint fusion, WiSE-FT weight interpolation, and entropy-weighted
27 GRPO reinforcement learning.
28
29 It improves in-domain formal-logic performance by **+26% relative** over its base model
30 (macro gate 0.336 → 0.422) **and improves held-out benchmark performance at the same time**
31 (+0.022 core average). It is the only arm in this project that gains on both tracks, which is
32 why it is the recommended release of the pair.
33
34 ![TwIL-LM3 formal and general reasoning benchmarks against gpt-oss-120b, Qwen3-8B, LFM2-2.6B and Llama-3.2-3B](benchmarks.jpg)
35
36 ## Highlights
37
38 * **Gains on both tracks at once** — the only arm in this project that does. In-domain macro gate
39 0.336 → 0.422, and the held-out 10-dataset macro 0.7193 → 0.7339 rather than the usual collapse
40 that follows task-specific fine-tuning.
41 * **Beats every arm up to and including LFM2.5-8B-A1B** — roughly three times its parameter count
42 — on all six Track A objective lanes and all four summary rows, not on average alone.
43 * **Competitive with 8B on strict scoring.** On strict-7, which gives no loose-match credit
44 anywhere, it sits 0.012 behind Qwen3-8B (0.1971 against 0.2093) at 2.6x fewer parameters, and
45 ahead of it on Lean formalisation (token-F1 0.5869 against 0.4022) and semantic parsing (0.4416
46 against 0.4257).
47 * **Structured formal output.** Tuned for the objects rather than the prose: FOL translation,
48 entailment labels, semantic parses, Lean statements and Lean proof critique.
49 * **The most efficient arm measured, at any scale.** 482-token Track B generations and 32.9
50 completed answers per second — about eight times gpt-oss-120b's rate — because it answers
51 short rather than because it decodes unusually fast.
52 * **Lowest maths-corpus perplexity of any released arm in the table** (3.8229), including
53 Qwen3-8B at 4.0083.
54 * **Runs anywhere.** 3.08B parameters in bf16, with Q4\_K\_M GGUF at 1.78 GiB for CPU or 4 GB of
55 VRAM.
56
57 It is not a general assistant: there is no safety or preference tuning here beyond what SmolLM3
58 carries, and instruction following regressed slightly. See
59 [Limitations](#limitations-and-caveats).
60
61 ## Model Details
62
63 | Property | Value |
64 | ------------------------- | ------------------------------------------------------------------------------------------- |
65 | Model ID | `webAI-Official/TwIL-LM3` |
66 | Base model | [`HuggingFaceTB/SmolLM3-3B`](https://huggingface.co/HuggingFaceTB/SmolLM3-3B) |
67 | Total parameters | 3.08B |
68 | Architecture | SmolLM3 decoder-only transformer; 36 layers, hidden size 2048 |
69 | Input / output | Text / text |
70 | Language | English |
71 | Tokenizer vocabulary size | 128,256 |
72 | Context window | 65,536 tokens |
73 | Checkpoint precision | bfloat16 (5.73 GiB), plus Q4\_K\_M / Q5\_K\_M / Q6\_K / Q8\_0 / F16 GGUF builds |
74 | Post-training | LoRA SFT → checkpoint fusion → WiSE-FT (λ = 0.25) → MGPO reinforcement learning (step 2071) |
75 | Reasoning format | Emits a `<think>…</think>` block before the answer |
76 | Evaluated decoding | Greedy, 2048 new tokens, `max_seq_len` 8192 |
77 | Specialisation | Formal logic: FOL translation, entailment, semantic parsing, Lean formalisation and critique |
78 | License | webAI Non-Commercial License ver. 1.0 |
79
80 The base model's 65,536-token context is carried through unchanged, but every score on this card
81 was measured inside an 8,192-token window; longer contexts are inherited rather than validated
82 here.
83
84 ## Results
85
86 ### Track A — in-domain formal logic
87
88 All arms below were run through the same harness, prompts and decoding settings described under
89 [Evaluation protocol](#evaluation-protocol). Throughput rows are reported because in-domain score
90 alone is misleading for a 3B model: `ans/s` is defined throughout as `tok/s ÷ mean generation
91 length`, so it measures completed answers rather than raw decode rate.
92
93 | lane / metric | TwIL-LM3 | TwIL-LM3* | SmolLM3-3B base | Llama-3.2-3B | LFM2-2.6B | LFM2.5-8B-A1B | Qwen3-8B | gpt-oss-120b ‡ |
94 |---|---:|---:|---:|---:|---:|---:|---:|---:|
95 | lean_formalize token_f1 | 0.5869 | **0.6456** | 0.4347 | 0.3690 | 0.1321 | 0.4655 | 0.4022 | 0.6306 |
96 | rule_induction derivation | 0.3192 | **0.9644** | 0.1029 | 0.0825 | 0.0615 | 0.1936 | 0.3680 | 0.6518 |
97 | entailment_label accuracy | 0.5750 | 0.6867 | 0.3750 | 0.3300 | 0.4700 | 0.5400 | 0.5800 | **0.7750** |
98 | mcq_answer accuracy | 0.1100 | **0.5200** | 0.0000 | 0.0000 | 0.0150 | 0.0750 | 0.0000 | 0.0700 |
99 | semantic_parse token_f1 | 0.4416 | **0.8762** | 0.4149 | 0.3102 | 0.3665 | 0.3778 | 0.4257 | 0.4331 |
100 | lean_critic accuracy | 0.6600 | 0.5200 | 0.6500 | 0.5300 | 0.5900 | 0.5500 | **0.7950** | 0.5550 |
101 | lm_corpus perplexity ↓ | 2.8972 | 3.1284 | 3.1818 | 2.8478 | 4.3815 | 4.9472 | **2.5440** | 912.23 § |
102 | math_corpus perplexity ↓ | 3.8229 | **3.5245** | 4.0685 | 4.7531 | 6.7472 | 8.3323 | 4.0083 | 1045.63 § |
103 | average, 6 lanes | 0.4488 | **0.7021** | 0.3296 | 0.2703 | 0.2725 | 0.3670 | 0.4285 | 0.5192 |
104 | **macro gate** | 0.4218 | **0.5896** | 0.3466 † | 0.2925 | 0.3473 | 0.3757 | 0.5336 | — |
105 | **strict-7** | 0.1971 | **0.3290** | 0.1493 | 0.1229 | 0.1579 | 0.1714 | 0.2093 | — |
106 | macro_primary | 0.4475 | 0.4958 | 0.4075 | 0.3450 | 0.4188 | 0.4213 | **0.5750** | — |
107 | tok/s | 15880 | 15840 | 15564 | 16160 | **25230** | 22480 | 9420 | 3374 |
108 | mean gen length | **564** | 572 | 999 | 696 | 2296 | 1830 | 2094 | 1005 |
109 | **ans/s** | **28.1** | 27.7 | 15.6 | 23.2 | 10.9 | 12.0 | 4.5 | 3.4 |
110
111 \* **TwIL-LM3\*** is our latest version of TwIL-LM3. **The weights will be released soon** — the
112 files in this repository are the current TwIL-LM3 release, not this one. Lanes marked — are not
113 yet reported for it.
114
115 ‡ **gpt-oss-120b** runs MXFP4 weights at tensor-parallel 2 — quantized and multi-GPU, so its
116 throughput rows are not directly comparable to the single-GPU BF16 arms. Its `procedural` lane
117 and the loose-match scorings were not collected, so the three summary rows below the six-lane
118 average cannot be computed for it; that is what the — cells mean, not a zero.
119
120 § The 120B's perplexities are three orders of magnitude off every other arm because its harmony
121 response format and tokenizer make the corpus lanes score a different quantity. The number is
122 reported for completeness but is not a comparable measurement.
123
124 † The base column here comes from the external-comparison run rather than the paired base-vs-TwIL
125 run, hence 0.3466 against the 0.3356 quoted in the summary at the top of this card — run-to-run
126 variation of the same checkpoint. The paired run is the correct basis for the improvement claim.
127
128 **`average, 6 lanes`** is the plain mean of the six objective rows above it, each at whatever
129 scoring that row reports. It is a coarser summary than the three that follow — it mixes token-F1
130 with accuracy — but it is the only summary row every arm here can be compared on, including the
131 120B.
132
133 The next three rows aggregate more carefully. None of them include the perplexity lanes or the
134 token-F1 scorings, which are not on a common 0–1 accuracy scale.
135
136 **`macro gate`** is the headline metric and the one the training pipeline gates on. It is the
137 equal-weight mean of five objectives: the four bounded classification lanes (`entailment_label`,
138 `mcq_answer`, `procedural`, `lean_critic`) plus `rule_induction`, scored by its continuous
139 derivation score. Rule induction is included specifically so a fine-tune cannot pass the gate
140 while quietly regressing inductive reasoning. In the gate, `mcq_answer` and `procedural` are
141 credited as `max(exact_match, loose_match)`: for free-text answer lanes, a response that is
142 correct but differently formatted is a formatting artefact rather than a reasoning failure. This
143 affects the aggregate only — the per-lane rows above stay strict.
144
145 **`macro_primary`** is the same mean over the four classification lanes alone, without
146 `rule_induction`. It is the narrower "bounded classification" view, kept for comparability with
147 earlier reports; the gate is the metric to read for overall in-domain capability.
148
149 **`strict-7`** is the mean of seven lanes scored under strict metrics only (`fol_translation`,
150 `entailment_label`, `mcq_answer`, `semantic_parse` and `lean_formalize` exact match,
151 `lean_critic` and `procedural` accuracy), with no loose-match credit anywhere. It is deliberately
152 harsh — exact match on generative lanes is near zero for every arm — so it is useful for ranking
153 models against each other but not as an absolute capability measure.
154
155 TwIL-LM3 beats every arm up to and including LFM2.5-8B-A1B, and does so on all six objective
156 lanes and all four summary rows, not on average alone. Against the strongest of them it is
157 0.4218 to 0.3757 on the gate at roughly a third of the total parameters, with the margin coming
158 from the lanes the pipeline targets directly: `lean_formalize` token-F1 0.5869 against 0.4655,
159 `rule_induction` 0.3192 against 0.1936, `semantic_parse` 0.4416 against 0.3778.
160
161 It does not beat the two largest arms. Qwen3-8B leads it on the gate 0.5336 to 0.4218 and
162 gpt-oss-120b leads the six-lane average 0.5192 to 0.4488. That gap is worth reading carefully in
163 Qwen's case: almost all of it is loose-match credit. Qwen answers MCQ correctly but never in the
164 requested format — strict accuracy 0.0000 against TwIL-LM3's 0.1100, while its loose match is
165 0.745 — and the macro rows credit `max(exact_match, loose_match)`. On `strict-7`, which gives no
166 loose-match credit anywhere, the two are 0.2093 to 0.1971, a gap of 0.012 rather than 0.11. Qwen
167 also wins `lean_critic` outright at 0.7950 and has the lowest `lm_corpus` perplexity at 2.5440.
168 The 120B leads three lanes outright and is genuinely stronger at entailment (0.7750) and rule
169 induction (0.6518).
170
171 The size and speed context matters for both. Qwen3-8B is 2.6x the parameters and produces 4.5
172 answers/sec against TwIL-LM3's 28.1; the 120B is 40x the parameters and produces 3.4. TwIL-LM3
173 is the strongest arm here at its own scale and the most efficient arm at any scale.
174
175 The unreleased TwIL-LM3\* moves the gate to 0.5896 and strict-7 to 0.3290, roughly +0.17 and
176 +0.13 over the current release. The gains are concentrated in the two lanes where TwIL-LM3 is
177 weakest in absolute terms rather than relative ones — `rule_induction` 0.3192 → 0.9644 and
178 `semantic_parse` token-F1 0.4416 → 0.8762 — plus strict MCQ accuracy 0.1100 → 0.5200. It gives
179 back `lean_critic` (0.6600 → 0.5200) and a little `lm_corpus` perplexity, so it is not uniformly
180 better.
181
182 It is also the most efficient arm in the table by a wide margin — 28.1 answers/sec, from
183 generations averaging 564 tokens where every other arm except Llama runs past 690. The Liquid
184 models decode faster in raw tokens per second, 25230 and 22480 against 15880, but their length
185 more than cancels it.
186
187 ### Track B — held-out benchmarks
188
189 | dataset | TwIL-LM3 | SmolLM3-3B base | Llama-3.2-3B | LFM2-2.6B | LFM2.5-8B-A1B | Qwen3-8B | gpt-oss-120b ‡ |
190 |---|---:|---:|---:|---:|---:|---:|---:|
191 | gsm8k | 0.8733 | 0.8833 | 0.8300 | 0.8767 | 0.9133 | 0.9567 | **0.9767** |
192 | svamp | 0.8500 | 0.8567 | 0.8200 | 0.9000 | 0.9133 | **0.9400** | **0.9400** |
193 | gsm_symbolic | 0.7567 | 0.7633 | 0.8067 | **0.9767** | 0.9267 | 0.8133 | 0.8467 |
194 | arc_cot | 0.8467 | 0.8400 | 0.7967 | 0.8667 | 0.9033 | 0.9633 | **0.9667** |
195 | logicbench | 0.7167 | 0.6467 | 0.5733 | 0.6267 | 0.7200 | **0.8567** | 0.8533 |
196 | strategyqa | 0.6500 | 0.6333 | 0.6533 | 0.6433 | 0.6667 | 0.7400 | **0.7867** |
197 | drop | 0.7467 | 0.7000 | 0.6733 | 0.6900 | 0.6633 | **0.8833** | 0.8500 |
198 | csqa | 0.7367 | 0.7067 | 0.7500 | 0.7433 | 0.7700 | **0.8633** | 0.8367 |
199 | musr | 0.4957 | 0.4997 | 0.4932 | 0.4867 | 0.5703 | 0.6301 | **0.6852** |
200 | mmlu_redux | 0.6667 | 0.6633 | 0.6000 | 0.7133 | 0.8367 | 0.8500 | **0.9467** |
201 | ifeval | 0.6433 | 0.6767 | 0.7167 | 0.7300 | **0.8900** | 0.8400 | 0.7900 |
202 | rudas_ood | 0.0365 | 0.0209 | **0.0733** | 0.0017 | 0.0061 | 0.0468 | 0.0000 ¶ |
203 | bbh_logic | 0.6633 | 0.6667 | 0.5333 | 0.5713 | 0.7700 | 0.6367 | **0.9980** |
204 | math500 | 0.6900 | 0.7000 | 0.4233 | 0.7133 | 0.7800 | 0.6100 | **0.8433** |
205 | **macro (10 CoT datasets)** | 0.7339 | 0.7193 | 0.6997 | 0.7523 | 0.7884 | 0.8493 | **0.8689** |
206 | **macro (all 14)** | 0.6694 | 0.6612 | 0.6245 | 0.6814 | 0.7378 | 0.7591 | **0.8086** |
207 | tok/s | 15880 | 15564 | 16160 | 25230 | 22480 | 9420 | 3374 |
208 | mean gen length | **482** | 626 | 510 | ≈796 | ≈1327 | ≈1931 | 801 |
209 | **ans/s** | **32.9** | 24.9 | 31.7 | ≈31.7 | ≈16.9 | 4.9 | 4.2 |
210
211 ‡ MXFP4 weights, tensor-parallel 2 — quantized and multi-GPU, so not directly comparable to the
212 single-GPU BF16 rows. ¶ 74% of its `rudas_ood` generations hit the length cap, so that cell is a
213 truncation artefact rather than a measured score; excluding the row, its 13-dataset macro is
214 0.8708.
215
216 Lengths marked ≈ are derived from stored generations using each model's characters-per-token
217 ratio rather than re-tokenized directly; the method reproduces the three directly measured
218 lengths to within 3.5%.
219
220 The honest summary of this table is that TwIL-LM3 does not lead it. Larger models score higher,
221 in order of size, and the 120B leads nine of fourteen rows. Two things are worth extracting
222 anyway. First, TwIL-LM3 improves on its own base while sitting mid-table (0.7339 against 0.7193
223 on the 10-dataset macro), which is the point of the WiSE-FT stage — in-domain gains without
224 transfer collapse. Second, it produces the shortest generations of any arm here at 482 tokens
225 and consequently the most answers per second at 32.9, roughly eight times the 120B's rate.
226
227 ## Usage
228
229 ```python
230 import torch
231 from transformers import AutoModelForCausalLM, AutoTokenizer
232
233 model_id = "webAI-Official/TwIL-LM3"
234 tok = AutoTokenizer.from_pretrained(model_id)
235 model = AutoModelForCausalLM.from_pretrained(
236 model_id, torch_dtype=torch.bfloat16, device_map="auto"
237 )
238
239 messages = [{"role": "user", "content":
240 "Does 'All dogs are mammals. Rex is a dog.' entail 'Rex is a mammal'? "
241 "Answer entailment, contradiction, or neutral."}]
242 inputs = tok.apply_chat_template(
243 messages, add_generation_prompt=True,
244 return_tensors="pt", return_dict=True,
245 ).to(model.device)
246
247 out = model.generate(**inputs, max_new_tokens=2048, do_sample=False)
248 print(tok.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
249 ```
250
251 `return_dict=True` matters on transformers 5.x, where `apply_chat_template` returns a
252 `BatchEncoding` rather than a bare tensor; the above works on both 4.x and 5.x.
253
254 The reported numbers use **greedy decoding** (`do_sample=False`) and a **2048-token** generation
255 budget. Note that the shipped `generation_config.json` inherits SmolLM3's sampling defaults
256 (`do_sample=true`, `temperature=0.6`, `top_p=0.95`), so `do_sample=False` must be passed
257 explicitly to reproduce the evaluation. The model opens a `<think>...</think>` reasoning block
258 before answering, so a short generation budget truncates reasoning and scores far worse.
259
260 ### GGUF / llama.cpp
261
262 Quantized GGUF builds ship in this repository alongside the safetensors weights. The `smollm3`
263 architecture is supported by llama.cpp, and the chat template, `<|im_end|>` EOS and BOS are
264 carried into the GGUF metadata, so chat mode works without extra flags.
265
266 | file | quant | size | bits/weight | notes |
267 |---|---|---:|---:|---|
268 | `TwIL-LM3-Q4_K_M.gguf` | Q4_K_M | 1.78 GiB | 4.96 | recommended default; runs on CPU or 4 GB of VRAM |
269 | `TwIL-LM3-Q5_K_M.gguf` | Q5_K_M | 2.06 GiB | 5.74 | a little more headroom than Q4_K_M |
270 | `TwIL-LM3-Q6_K.gguf` | Q6_K | 2.35 GiB | 6.56 | close to Q8_0 quality at two-thirds the size |
271 | `TwIL-LM3-Q8_0.gguf` | Q8_0 | 3.05 GiB | 8.50 | near-lossless, for quality-sensitive use |
272 | `TwIL-LM3-F16.gguf` | F16 | 5.73 GiB | 16.00 | unquantized, for requantization or reference runs |
273
274 ```bash
275 llama-cli -m TwIL-LM3-Q4_K_M.gguf -cnv --temp 0 -n 2048
276 ```
277
278 Two things matter for reproducing the scores above under llama.cpp. Pass `--temp 0`, because the
279 evaluation is greedy while the packaged sampling defaults are not. And leave the generation
280 budget large — 2048 tokens or more — since the model emits a `<think>` block before answering
281 and a short budget truncates it, which costs far more accuracy than the quantization does.
282
283 F16 and Q8_0 were produced directly by `convert_hf_to_gguf.py` from the released bf16 weights; the
284 K-quants (Q4_K_M, Q5_K_M, Q6_K) were quantized from the F16 build with `llama-quantize`, without
285 an importance matrix. All five were smoke-tested for load and generation on CPU. Note that F16 is
286 not bit-identical to the released weights: bf16 and f16 carry the same 16 bits but trade exponent
287 range against mantissa precision, so the conversion is a narrowing one, in practice negligible
288 for inference.
289
290 The published Track A and Track B numbers were measured on the **bf16** weights through vLLM, not
291 on any of these GGUF builds, so expect small deviations — most likely at Q4_K_M — that have not
292 been quantified here.
293
294 ## How it was built
295
296 Four stages on top of the base model:
297
298 1. **LoRA supervised fine-tuning** on a synthetic formal-logic corpus covering the Track A
299 objectives (first-order-logic translation, entailment labelling, semantic parsing, Lean
300 formalisation and critique, procedural reasoning, rule induction).
301 2. **Checkpoint fusion** — parameter-space averaging of intermediate SFT checkpoints selected
302 by a diversity probe, rather than taking the final checkpoint.
303 3. **WiSE-FT interpolation** toward the pretrained base, `W = (1 − λ)·W_base + λ·W_finetuned`
304 with **λ = 0.25** — i.e. only a quarter of the fine-tuned delta is retained. λ was chosen by
305 constrained optimisation: maximise in-domain score subject to minimal degradation on held-out
306 benchmarks. This conservative λ is the direct reason held-out capability survives.
307 4. **MGPO** — entropy-weighted GRPO reinforcement learning against a programmatic verifier, with
308 partial credit for loose matches and token-F1 so that all-fail prompt groups still produce
309 gradient. Published checkpoint is **step 2071**.
310
311 A sibling arm that skipped stage 3's conservative interpolation scores considerably higher
312 in-domain (macro gate 0.515) but gives back roughly twelve points of held-out capability. This
313 release is the balanced point of that trade; the other was not published.
314
315 ## Limitations and caveats
316
317 **Truncation.** At a 2048-token budget, 4.4% of Track A generations hit the cap — better than
318 the base's 17.4%, but still above the 2% threshold our protocol requires to mark a comparison
319 `rankable`. The Track A macro gate should therefore be read as indicative rather than exact.
320 Because a truncated response scores zero regardless of reasoning quality, both numbers are
321 pessimistic, and the base substantially more so — meaning the true Track A gap is probably
322 narrower than +0.086.
323
324 **Scope.** Tuned for formal logic. The Track B suite does not cover code generation or tool use
325 (HumanEval, LiveCodeBench and BFCL were not run for this model or its base), so this release
326 makes no claim about those.
327
328 **Not a chat model.** It was optimised against automatic verifiers on logic tasks. It has had no
329 safety tuning beyond whatever the base model carries, and no instruction-following alignment
330 work — IFEval regressed slightly.
331
332 **Failed consolidation stage.** A post-RL self-distillation round (SDFT) was attempted and made
333 both tracks worse at every budget tried (−18% Track A at one epoch on this family). It is not
334 part of this model. See the accompanying `SDFT_RESULT.md` in the project repository.
335
336 ## Evaluation protocol
337
338 - Track A: `n = 200` per objective, greedy (`temperature = 0`), `max_new_tokens = 2048`, one
339 retry at 4096 for truncated rows, `max_seq_len = 8192`, seed 42.
340 - Track B: 300 examples per task, greedy, `max_gen_toks = 4096`, `max_model_len = 8192`,
341 `repetition_penalty = 1.0`, chat template applied, vLLM backend.
342 - Both tracks use the same protocol for the model and its base, in a paired run over identical
343 sampled rows.
344
345 `repetition_penalty = 1.0` is load-bearing. A 1.1 penalty produced apparent 20-point swings on
346 Track B that were pure decoding artefact; the decoding kwargs are hashed into the protocol
347 identity so a mismatched runner fails loudly instead of quietly producing a different number.
348
349 ## Relationship to TwIL-LM
350
351 [**TwIL-LM2**](https://huggingface.co/webAI-Official/TwIL-LM) is the 1.7B member of this family, built
352 from SmolLM2 by the same pipeline. It reaches a higher in-domain score relative to its own base —
353 and leads every arm we have measured on Track A strict-7, at any size — but it gives back
354 held-out capability; this model is the one that improves both. Both repositories now ship full
355 merged models on `main`, loaded directly with `AutoModelForCausalLM`; the original LoRA-adapter
356 release is archived on that repository's `TwIL-LM1` branch.
357
358 ## License and attribution
359
360 Released under the **webAI Non-Commercial License ver. 1.0** — see `LICENSE.md` in this
361 repository.
362
363 The base model, [`HuggingFaceTB/SmolLM3-3B`](https://huggingface.co/HuggingFaceTB/SmolLM3-3B),
364 is Apache 2.0; its licence text is retained as `apache-2.0-LICENSE.txt` and all credit for the
365 base model goes to the HuggingFaceTB team. Apache 2.0 permits distributing derivative works
366 under different terms provided attribution is preserved, which is what the pair of licence files
367 in this repository does.
368