README.md · TwIL-8B

1

---

2

language:

3

- en

4

library_name: transformers

5

pipeline_tag: text-generation

6

base_model: ibm-granite/granite-4.2-8b

7

license: other

8

license_name: webai-non-commercial-license-ver.-1.0

9

license_link: https://huggingface.co/webAI-Official/TwIL-8B/blob/main/LICENSE.md

10

tags:

11

- granite

12

- granite-4.2

13

- twil

14

- reasoning

15

- math

16

- code

17

- instruction-following

18

- reinforcement-learning

19

- grpo

20

- gguf

21

---

22

23

# TwIL-8B

24

25

An 8.8B reasoning model for **math, code and instruction following**, post-trained from

26

[`ibm-granite/granite-4.2-8b`](https://huggingface.co/ibm-granite/granite-4.2-8b) with

27

full-parameter supervised fine-tuning and GRPO/DAPO reinforcement learning. It ships in two

28

variants that share one architecture and tokenizer: **balanced** (the default) and **verbose**.

29

See [Variants](#variants).

30

31

Against open models of similar size and up to 3.5x larger, TwIL-8B posts the highest MATH-500

32

(98.2) and LiveCodeBench (81.6 †) scores in the comparison and leads Qwen3.5-9B on IFBench

33

(+9.9 †). It is level with Qwen3.5-9B on contest math while using fewer tokens per problem. It

34

trails on knowledge-heavy benchmarks: GPQA Diamond, MMLU-Pro † and HLE †.

35

36

![TwIL-8B math, knowledge, instruction-following and code benchmarks against Qwen3.5-9B, Gemma 4 12B and Gemma 4 31B](benchmarks.png)

37

38

*TwIL-8B scores are for the balanced variant. † marks benchmarks measured on the verbose variant,

39

which have not yet been run on balanced. Competitor scores are published figures and were not

40

re-run; see [Evaluation protocol](#evaluation-protocol).*

41

42

## Highlights

43

44

* **Highest MATH-500 and AIME 2025 in the comparison.** MATH-500 98.2 against 96.6 for

45

Qwen3.5-9B and 87.9 for Gemma 4 31B; AIME 2025 88.3 against 87.1 for Gemma 4 31B.

46

* **Best LiveCodeBench in the group.** 81.6 † against 80.0 for Gemma 4 31B, 72.0 for Gemma 4 12B

47

and 65.6 for Qwen3.5-9B (+16.0). It is within 2.9–6.2 points of GPT-5.5, Claude Opus 4.8 and

48

Grok 4.3.

49

* **Strong instruction following.** IFBench 74.4 † is 9.9 points above Qwen3.5-9B, level with

50

Gemma 4 12B (74.0) and 1.6 behind Gemma 4 31B. It is also 12.2 points above Claude Opus 4.8

51

and 1.5 behind GPT-5.5. IFEval 91.8 is level with Qwen3.5-9B (91.5).

52

* **Level with Qwen3.5-9B on contest math, ahead of Gemma 4 12B.** Across the four contests both

53

publish, the averages are within 0.4 points. TwIL-8B leads on HMMT Nov 2025 (+3.8) and HMMT

54

Feb 2025 (+1.8) and trails on AIME 2026 (−4.2) and HMMT Feb 2026 (−3.0). On AIME 2026 it is

55

10.8 points above Gemma 4 12B and within 1 point of Gemma 4 31B.

56

* **Fewer tokens per problem.** About 19.5k output tokens per AIME 2026 problem against 31.2k

57

for Qwen3.5-9B (38% fewer), and 32.9k against 40.6k on HMMT Feb 2026 (19% fewer). It also

58

uses fewer tokens than Claude Opus 4.8 on AIME 2026 (21.0k), though at lower accuracy.

59

* **88–93% of the frontier on reasoning.** It reaches 93% of the best frontier score on

60

LiveCodeBench †, 92% on IFBench †, 90% on HMMT Nov 2025 and 88% on AIME 2026 and HMMT Feb 2025.

61

HMMT Feb 2026 is the exception, at 69%.

62

* **Runs locally.** 8.79B parameters in bf16 (16.38 GiB), with a 4.98 GiB Q4_K_M GGUF for CPU or

63

modest-VRAM GPUs.

64

65

It is **not** a knowledge model. GPQA Diamond (70.2) trails Qwen3.5-9B by 11.5 and Gemma 4 31B

66

by 14.1, and MMLU-Pro † and HLE † follow the same pattern. It also remains verbose next to the

67

frontier: GPT-5.5 solves AIME 2026 with about 5k tokens per problem. There is no safety or

68

preference tuning beyond what Granite 4.2 carries. See [Limitations](#limitations-and-caveats).

69

70

## Model Details

71

72

| Property | Value |

73

| --- | --- |

74

| Model ID | `webAI-Official/TwIL-8B` |

75

| Base model | [`ibm-granite/granite-4.2-8b`](https://huggingface.co/ibm-granite/granite-4.2-8b) |

76

| Total parameters | 8.79B (8,791,592,960) |

77

| Architecture | Granite decoder-only dense transformer (`GraniteForCausalLM`); 40 layers, hidden size 4096, 32 attention heads / 8 KV heads |

78

| Input / output | Text / text |

79

| Language | English |

80

| Tokenizer vocabulary size | 100,352 |

81

| Context window | 131,072 tokens (inherited from the base model) |

82

| Variants | `balanced` (default, repository root) and `verbose` (`verbose/` folder) |

83

| Checkpoint precision | bfloat16 (16.38 GiB per variant), plus GGUF builds |

84

| Post-training | Full-parameter SFT → GRPO/DAPO RL (verbose) → length-aware RL (balanced) |

85

| Reasoning format | Emits a `<think> … </think>` block before the answer (default chat template) |

86

| Evaluated decoding | High-effort pass@1, thinking enabled |

87

| Specialisation | Contest math, code and instruction following |

88

| License | webAI Non-Commercial License ver. 1.0 (base model: Apache License 2.0, IBM) |

89

90

### Variants

91

92

Both variants use the same architecture, tokenizer, chat template and generation config. They

93

differ only in how much they reason before answering.

94

95

96

| --- | --- | --- | --- |

97

| **balanced** (default) | `model-balanced.safetensors` (repository root) | `TwIL-8B-Q4_K_M-balanced.gguf` | General use. Shorter reasoning traces and a lower cost per answer. |

98

| **verbose** | `verbose/model-verbose.safetensors` | `TwIL-8B-F16-verbose.gguf`, `TwIL-8B-Q8_0-verbose.gguf`, `TwIL-8B-Q4_K_M-verbose.gguf` | Hard contest-style problems where accuracy matters more than token cost. Longer reasoning traces. |

99

100

`from_pretrained("webAI-Official/TwIL-8B")` loads **balanced**. Pass `subfolder="verbose"` to

101

load **verbose** (see [Usage](#usage)).

102

103

## Benchmarks

104

105

### Against open peers

106

107

108

| --- | ---: | ---: | ---: | ---: |

109

| ***Math*** | | | | |

110

| MATH-500 | **98.2** | 96.6 | — | 87.9 |

111

| AIME 2025 | **88.3** | — | — | 87.1 |

112

| AIME 2026 | 88.3 | **92.5** | 77.5 | 89.2 |

113

| HMMT Feb 2025 | 85.0 | 83.2 | — | **88.7** |

114

| HMMT Feb 2026 | 68.2 | 71.2 | — | **77.2** |

115

| HMMT Nov 2025 | 86.7 | 82.9 | — | **87.5** |

116

| ***Science & knowledge*** | | | | |

117

| GPQA Diamond | 70.2 | 81.7 | 78.8 | **84.3** |

118

| MMLU-Pro † | 76.5 | 82.5 | 77.2 | **85.2** |

119

| HLE † | 12.6 | 14.9 | 5.2 | **19.5** |

120

| ***Instruction following*** | | | | |

121

| IFEval | 91.8 | 91.5 | 97.2 | **98.9** |

122

| IFBench † | 74.4 | 64.5 | 74.0 | **76.0** |

123

| ***Code*** | | | | |

124

| LiveCodeBench † | **81.6** | 65.6 | 72.0 | 80.0 |

125

| HumanEval † | 93.2 | 93.9 | — | **94.5** |

126

| HumanEval+ † | 88.3 | 87.2 | — | **91.5** |

127

| MBPP † | 93.7 | — | — | — |

128

| MBPP+ † | 79.1 | — | — | — |

129

| CRUXEval † | 97.1 | — | — | — |

130

131

† Measured on the verbose variant; not yet run on balanced. — means no published result could be

132

found. Bold marks the best score in each row where at least two models report one. All scores are

133

percentages.

134

135

TwIL-8B leads three rows outright (MATH-500, AIME 2025 and LiveCodeBench †). It is the strongest

136

model here at its own size on code and instruction following: against Qwen3.5-9B it is ahead on

137

LiveCodeBench † by 16.0 and IFBench † by 9.9, and the two are level on IFEval, HumanEval † and

138

HumanEval+ †. Contest math is close to even with Qwen3.5-9B (two contests each way), and ahead of

139

Gemma 4 12B by 10.8 on the one contest it publishes.

140

141

Gemma 4 31B, at about 3.5x the parameters, leads most remaining rows. TwIL-8B is ahead of it on

142

MATH-500, AIME 2025 and LiveCodeBench †, within one point on AIME 2026 and HMMT Nov 2025, and

143

behind on knowledge, instruction following and the hardest contest set (HMMT Feb 2026, −9.0).

144

MBPP †, MBPP+ † and CRUXEval † have no comparable published competitor numbers.

145

146

Notes on the competitor figures:

147

148

* Gemma 4 31B's HMMT scores were measured by the Qwen team (Qwen3.6-27B model card), not by Google.

149

Its MATH-500 (87.9) and AIME 2025 come from Red Hat's lighteval runs of the unquantized model;

150

the MATH-500 figure is unusually low and may reflect answer extraction.

151

* Google's IFEval figures for Gemma 4 (97.2 and 98.9) do not state the metric variant. Red Hat

152

measured 94.6 prompt-level strict for Gemma 4 31B.

153

* HumanEval and HumanEval+ comparators are community runs. The Gemma 4 31B run used a 512-token

154

cap, which leaves no room for thinking. The Qwen3.5-9B run had thinking enabled.

155

* Competitor LiveCodeBench scores are v6. Qwen3.5-9B's AIME 2026 and HMMT Feb 2026 scores are from

156

MathArena; its only public AIME 2025 results are non-thinking runs and are excluded.

157

158

### Against frontier models

159

160

GPT-5.5, Claude Opus 4.8 and Grok 4.3 at their highest published reasoning effort. The last column

161

is TwIL-8B's score as a share of the best frontier score on that row.

162

163

164

| --- | ---: | ---: | ---: | ---: | ---: |

165

| AIME 2026 | 88.3 | 100.0 | 100.0 | — | 88% |

166

| HMMT Feb 2025 | 85.0 | 96.5 | 96.5 | — | 88% |

167

| HMMT Feb 2026 | 68.2 | 98.5 | 95.5 | — | 69% |

168

| HMMT Nov 2025 | 86.7 | 96.5 | 96.5 | — | 90% |

169

| GPQA Diamond | 70.2 | 93.6 | 93.6 | 90.1 | 75% |

170

| MMLU-Pro † | 76.5 | 88.1 | 89.6 | 85.8 | 85% |

171

| HLE † | 12.6 | 45.8 | 48.7 | 37.2 | 26% |

172

| IFBench † | 74.4 | 75.9 | 62.2 | 81.3 | 92% |

173

| LiveCodeBench † | 81.6 | 85.3 | 87.8 | 84.5 | 93% |

174

175

TwIL-8B is closest to the frontier on **instruction following and code**. It beats Claude Opus 4.8

176

on IFBench † by 12.2 points and is within 2.9–6.2 points of all three models on LiveCodeBench †.

177

Contest math lands at 88–90% of the frontier, with HMMT Feb 2026, the hardest set, as the outlier

178

at 69%. **Knowledge is the largest gap**: GPQA Diamond trails by about 20–23 points and HLE † by 25–36.

179

These benchmarks reward stored knowledge, which tends to scale with parameter count more than with

180

reasoning-focused training.

181

182

These three are the most recent frontier releases with measured results on this benchmark set.

183

Newer releases (GPT-6.1 Sol, Claude Opus 5.5 and Grok 4.7) no longer report AIME, HMMT or GPQA.

184

Their only measured overlap is Artificial Analysis HLE: 52.9, 61.4 and 43.1 respectively.

185

186

### Head-to-head

187

188

Averages are taken only over benchmarks where both models have a score, so the benchmark count

189

differs by row and the averages are not comparable across rows.

190

191

| Opponent | Shared benchmarks | Wins–losses | Shared average (TwIL-8B vs opponent) | TwIL-8B leads on | TwIL-8B trails on |

192

| --- | ---: | :---: | :---: | --- | --- |

193

| Qwen3.5-9B | 13 | 7–6 | 78.1 vs 77.6 | LiveCodeBench † +16.0, IFBench † +9.9, HMMT Nov 2025 +3.8, HMMT Feb 2025 +1.8, MATH-500 +1.6 | GPQA Diamond −11.5, MMLU-Pro † −6.0, AIME 2026 −4.2, HMMT Feb 2026 −3.0 |

194

| Gemma 4 12B | 7 | 4–3 | 70.8 vs 68.8 | AIME 2026 +10.8, LiveCodeBench † +9.6, HLE † +7.4 | GPQA Diamond −8.6, IFEval −5.4 |

195

| Gemma 4 31B | 14 | 3–11 | 78.8 vs 82.0 | MATH-500 +10.3, LiveCodeBench † +1.6, AIME 2025 +1.2 | GPQA Diamond −14.1, HMMT Feb 2026 −9.0, MMLU-Pro † −8.7, IFEval −7.1 |

196

| GPT-5.5 | 9 | 0–9 | 71.5 vs 86.7 | — | HLE † −33.2, HMMT Feb 2026 −30.3, GPQA Diamond −23.4 |

197

| Claude Opus 4.8 | 9 | 1–8 | 71.5 vs 85.6 | IFBench † +12.2 | HLE † −36.1, HMMT Feb 2026 −27.3, GPQA Diamond −23.4 |

198

| Grok 4.3 | 5 | 0–5 | 63.1 vs 75.8 | — | HLE † −24.6, GPQA Diamond −19.9, MMLU-Pro † −9.3 |

199

200

### Tokens per problem

201

202

Mean output tokens per problem, with accuracy. Competitor figures are from MathArena; frontier

203

counts are rounded to the nearest 1k.

204

205

| Contest | **TwIL-8B** | Qwen3.5-9B | Claude Opus 4.8 | GPT-5.5 |

206

| --- | --- | --- | --- | --- |

207

| AIME 2026 | **19.5k** · 88.3% | 31.2k · 92.5% | 21.0k · 100.0% | 5.0k · 100.0% |

208

| HMMT Feb 2026 | **32.9k** · 68.2% | 40.6k · 71.2% | 30.0k · 95.5% | 8.0k · 98.5% |

209

210

Against Qwen3.5-9B, the closest open peer by size, TwIL-8B uses 38% fewer tokens on AIME 2026

211

and 19% fewer on HMMT Feb 2026, for 4.2 and 3.0 fewer points respectively. GPT-5.5 remains far

212

more token-efficient. Harnesses and tokenizers differ, so treat this comparison as directional.

213

214

## Usage

215

216

```python

217

import torch

218

from transformers import AutoModelForCausalLM, AutoTokenizer

219

220

model_id = "webAI-Official/TwIL-8B"

221

variant = None # balanced (default); use "verbose" for the verbose variant

222

223

tok = AutoTokenizer.from_pretrained(model_id, subfolder=variant or "")

224

model = AutoModelForCausalLM.from_pretrained(

225

model_id, subfolder=variant or "", dtype=torch.bfloat16, device_map="auto"

226

)

227

228

messages = [{"role": "user", "content":

229

"How many positive integers n < 1000 are divisible by 3 but not by 5? "

230

"Reason step by step, then give the final integer."}]

231

inputs = tok.apply_chat_template(

232

messages, add_generation_prompt=True,

233

return_tensors="pt", return_dict=True,

234

).to(model.device)

235

236

out = model.generate(**inputs, max_new_tokens=32768)

237

print(tok.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

238

```

239

240

Use a Transformers release with Granite 4.2 support. `return_dict=True` matters on Transformers

241

5.x, where `apply_chat_template` returns a `BatchEncoding` rather than a bare tensor.

242

243

The chat template opens a `<think>` block by default, so the model reasons before it answers.

244

Give it a large generation budget: on contest math, balanced averages about 19.5k tokens per AIME

245

2026 problem and 33k per HMMT Feb 2026 problem, and verbose writes more. A short budget truncates

246

the reasoning and scores far worse. The shipped `generation_config.json` samples with

247

`temperature=1.0` and `top_p=0.95`. The template also accepts `enable_thinking=False` and

248

`reasoning_effort="low"` through `chat_template_kwargs`. Both lower latency, but low-effort runs

249

score substantially lower and none of the numbers on this card use them.

250

251

To download a single variant:

252

253

```bash

254

# balanced

255

hf download webAI-Official/TwIL-8B --exclude "verbose/*" --exclude "*.gguf" --local-dir TwIL-8B-balanced

256

# verbose

257

hf download webAI-Official/TwIL-8B --include "verbose/*" --local-dir TwIL-8B

258

```

259

260

### GGUF / llama.cpp

261

262

263

| --- | --- | --- | --- |

264

265

| `TwIL-8B-Q4_K_M-verbose.gguf` | verbose | Q4_K_M | 4.98 GiB |

266

| `TwIL-8B-Q8_0-verbose.gguf` | verbose | Q8_0 | 8.70 GiB |

267

| `TwIL-8B-F16-verbose.gguf` | verbose | F16 | 16.38 GiB |

268

269

```bash

270

llama-cli -hf webAI-Official/TwIL-8B --hf-file TwIL-8B-Q4_K_M-balanced.gguf -cnv -c 40960 -n 32768

271

```

272

273

Name the file with `--hf-file`, because each quant exists for more than one variant. Keep the

274

context and generation budget large for the same reason as above. The model uses a ChatML-style

275

template with `<|im_end|>` as EOS. All scores on this card were measured on the bf16 weights,

276

not on the GGUF builds, so expect small deviations, most likely at Q4_K_M.

277

278

## How it was built

279

280

1. **Supervised fine-tuning.** Full-parameter SFT of `ibm-granite/granite-4.2-8b` on reasoning

281

traces covering math, STEM and instruction following.

282

2. **Reinforcement learning.** GRPO/DAPO RL on math, code, STEM and instruction-following

283

prompts. This produces the **verbose** variant.

284

3. **Length-aware RL.** A further RL stage with a length-aware reward that favours shorter

285

reasoning traces, measured against per-domain length references for math, code, STEM and

286

instruction following. This produces the **balanced** variant.

287

288

The math, STEM and instruction-following training pools were decontaminated with 13-gram matching

289

against AIME (2025, 2026 and earlier years), HMMT Feb 2025 and Feb 2026, MATH-500, GPQA Diamond,

290

MMLU-Pro, HLE, IFEval, IFBench and LiveCodeBench. Both variants ship as full merged models, not

291

LoRA adapters.

292

293

## Limitations and caveats

294

295

* **Knowledge.** GPQA Diamond, MMLU-Pro † and HLE † trail larger open models and the frontier by

296

wide margins.

297

* **Coverage.** Rows marked † were measured on the verbose variant and have not yet been run on

298

balanced. SWE-bench, SuperGPQA, tool use and agentic tasks were not evaluated.

299

* **Cross-harness comparisons.** Competitor numbers come from each vendor's model card or from

300

third-party evaluations, with their own prompts, sampling and token budgets. The AIME and HMMT

301

sets have 30–33 problems, so one problem is worth about 3 points and gaps of that size are

302

within noise.

303

* **HLE scoring.** TwIL-8B's HLE uses a multiple-choice scoring rule. Competitor HLE scores are

304

text-only and judge-graded, so that comparison is approximate.

305

* **LiveCodeBench version.** Competitor scores are v6. The version and date window of TwIL-8B's

306

run are not recorded, so confirm them before quoting that gap.

307

* **Contamination.** HMMT Nov 2025 and the HumanEval, MBPP and CRUXEval suites were not in the

308

decontamination set, and decontamination of the code training data is not recorded.

309

* **Token cost.** Even balanced spends about 19.5k tokens per AIME 2026 problem, roughly 4x

310

GPT-5.5.

311

* **Not a general assistant.** There is no additional safety, preference or tool-use training

312

beyond what Granite 4.2 carries.

313

314

## Evaluation protocol

315

316

* **TwIL-8B:** internal high-effort pass@1 with thinking enabled, on the same harness for every

317

benchmark and both variants. Scores are shown as percentages (0.982 → 98.2).

318

* **Competitors:** reproduced from the public sources below, accessed 3 November 2026. No

319

competitor benchmark was re-run. Frontier models are at their highest published reasoning

320

effort. Model-predicted estimates on aggregator sites are excluded.

321

* **Naming:** "GPQA" means GPQA Diamond. AIME 2025 and AIME 2026 are the single-year sets.

322

323

## Relationship to TwIL-LM

324

325

TwIL-8B belongs to the same webAI TwIL family as

326

[TwIL-LM3](https://huggingface.co/webAI-Official/TwIL-LM3) and

327

[TwIL-LM3-Pro](https://huggingface.co/webAI-Official/TwIL-LM3-Pro). Those are 3B formal-logic

328

models; TwIL-8B targets contest math, code and instruction following at 8B, on IBM's Granite 4.2

329

8B base. Like them, it ships as a full model on `main`, loaded directly with

330

`AutoModelForCausalLM`.

331

332

## License and attribution

333

334

Released under the **webAI Non-Commercial License ver. 1.0**; see `LICENSE.md` in this repository.

335

336

The base model, [`ibm-granite/granite-4.2-8b`](https://huggingface.co/ibm-granite/granite-4.2-8b),

337

is Copyright IBM Corporation and is distributed under the Apache License 2.0; its license text is

338

retained as `apache-2.0-LICENSE.txt` and all credit for the base model goes to IBM. Apache 2.0

339

permits distributing derivative works under different terms provided attribution is preserved,

340

which is what the pair of license files in this repository does.

341

342

## Sources

343

344

Competitor figures on this card come from:

345

346

* [Gemma 4 model card](https://ai.google.dev/gemma/docs/core/model_card_4) (Google, thinking mode):

347

Gemma 4 12B and 31B AIME 2026, GPQA Diamond, MMLU-Pro, HLE and LiveCodeBench v6.

348

* [Gemma 4 Technical Report, Table 5](https://arxiv.org/abs/2607.02770): Gemma 4 IFEval and IFBench.

349

* [Qwen3.6-27B model card](https://huggingface.co/Qwen/Qwen3.6-27B), Gemma4-31B baseline column:

350

Gemma 4 31B HMMT Feb 2025, Nov 2025 and Feb 2026.

351

* [RedHatAI/gemma-4-31B-it-FP8-dynamic](https://huggingface.co/RedHatAI/gemma-4-31B-it-FP8-dynamic),

352

BF16 baseline: Gemma 4 31B MATH-500 and AIME 2025.

353

* [EvalPlus submission #300](https://github.com/evalplus/evalplus/issues/300): Gemma 4 31B HumanEval

354

and HumanEval+.

355

* [Qwen3.5-9B model card](https://huggingface.co/Qwen/Qwen3.5-9B) (thinking mode): GPQA Diamond,

356

MMLU-Pro, IFEval, IFBench, HMMT Feb 2025, HMMT Nov 2025 and LiveCodeBench v6.

357

* [MathArena, Qwen3.5-9B](https://matharena.ai/models/qwen_qwen3_5_9b): AIME 2026 and HMMT Feb 2026

358

accuracy and tokens per problem.

359

* [SOTA2 MATH-500 leaderboard](https://www.sota2.com/research/sota/math-on-math-500-score):

360

Qwen3.5-9B MATH-500.

361

* [Artificial Analysis via OpenRouter, Qwen3.5 9B](https://openrouter.ai/qwen/qwen3.5-9b:batch):

362

Qwen3.5-9B HLE.

363

* [nivvis/Qwen3.5-9B-EQ-v5.1](https://huggingface.co/nivvis/Qwen3.5-9B-EQ-v5.1-GGUF), vanilla

364

baseline row: Qwen3.5-9B HumanEval and HumanEval+.

365

* [Introducing GPT-5.5](https://openai.com/index/introducing-gpt-5-5/) and the

366

[Claude Opus 4.8 System Card](https://www.anthropic.com/claude-opus-4-8-system-card): GPQA Diamond.

367

* MathArena via [BenchLeader AIME 2026](https://www.benchleader.com/benchmarks/matharena_aime_2026),

368

[BenchLeader HMMT Feb 2026](https://www.benchleader.com/benchmarks/matharena_hmmt_feb_2026) and

369

[The Aggregate HMMT Feb 2025](https://theaggregate.ai/benchmark?slug=matharena-hmmt-feb-2025):

370

GPT-5.5 and Claude Opus 4.8 contest accuracy and tokens per problem.

371

* [GLM-5.2 model card](https://huggingface.co/zai-org/GLM-5.2): GPT-5.5 and Claude Opus 4.8 HMMT

372

Nov 2025.

373

* benchgap.net for [GPT-5.5](https://benchgap.net/model/gpt-5-5),

374

[Claude Opus 4.8](https://benchgap.net/model/claude-opus-4-8) and

375

[Grok 4.3](https://benchgap.net/model/grok-4-3) (measured rows only): Artificial Analysis HLE and

376

IFBench, and Vals AI MMLU-Pro and LiveCodeBench.

377

* [Artificial Analysis via OpenRouter, Grok 4.3](https://openrouter.ai/x-ai/grok-4.3): Grok 4.3

378

GPQA Diamond, HLE and IFBench.

379

* benchgap.net for the newest frontier releases, [GPT-6.1 Sol](https://benchgap.net/model/gpt-6-1-sol),

380

[Claude Opus 5.5](https://benchgap.net/model/claude-opus-5-5) and

381

[Grok 4.7](https://benchgap.net/model/grok-4-7): Artificial Analysis HLE.

382