README.md · JiRackDeltaNet_27b

1

---

2

language:

3

- en

4

- zh

5

- ja

6

- ko

7

- fr

8

- es

9

- pt

10

- de

11

- it

12

- ru

13

- ar

14

- vi

15

- th

16

tags:

17

- text-generation

18

- deltanet

19

- qwen3.8

20

- gguf

21

- cpu

22

- efficient

23

- low-memory

24

- ternary

25

- bitnet

26

- jirack

27

- web-ui

28

- routing

29

- tool-call

30

- robotics

31

- ollama

32

- ternary-weights

33

- ternary-transformer

34

- quantization

35

license: mit

36

---

37

38

# Qwen 3.8 27b migrated to Ternary Architedure

39

- Benefits high quality CPU inference TQ_2 on Llama.cpp and Ollama via QAT

40

- Robotcs, Routing, Coding, Multimedia, Advanced tool calling via JiRackDeltaNetTokenizer

41

- JiRack DeltaNet understand video and images that best for Robotics also

42

43

## PARTNERSHIP

44

45

- NVIDIA

46

- FISERV

47

48

# JiRack DeltaNet 27B (CPU)

49

A fast and efficient 27B model optimized for CPU inference. Built on a Qwen3.8-style DeltaNet architecture (hybrid attention + SSM), with an updated tokenizer that includes **Routing**, **Media**, **Vision**, **Sound**, **Tool call**, and **Robotics** tags. Ready-to-run GGUF quantizations, and native Ollama support with reasoning disabled by default for fast, direct responses.

50

- JiRack is a cloud-ready model that helps save money on cloud infrastructure. It can be used as an expert model in RAG deployments, with the ONNX JiRack Java server as an alternative.

51

52

# JiRack service options

53

- Current quantizations were done from the FP16 model.

54

- If you need custom compression or fine-tuning, please write to me and I'll perform QAT from your dataset, tailored specifically to your task.

55

- Plus double QAT via ONNX QAT.

56

- Adapt train process to avoid catastrophic forgetting with NDA

57

- Adapt train process to avoid fast plateau in training with NDA

58

- Adapts to agentic or instruct models for tool calling, using the JiRack tokenizer to enable high-quality tool calling on small models — built as a domain-specific tool expert.

59

- Deployment and scale

60

61

# JiRack Codding Agent IDE

62

- It is Agent Coding IDE for JiRack Models to run via Ollama on home PC

63

- It good choose for Agent Coding IDE such as Cursor , Windsurf IDE or Devin IDE etc but more safe that ask you to apply changes and review.

64

- Web site https://www.jirack.com

65

- Final release version https://huggingface.co/CMSManhattan/JiRackDeltaNet_27b/resolve/main/jirack_ide_final.zip

66

67

68

# Ollama production support

69

- JiRack DeltaNet 27B runs natively on Ollama with reasoning disabled by default (no forced `<think>` blocks).

70

- Available now: **`cmsmanhattan/JiRackDeltaNet_27b-q4-reasoning`** — https://ollama.com/cmsmanhattan

71

- Runtime override also supported: `ollama run cmsmanhattan/JiRackDeltaNet_27b-q4-reasoning --think=false`

72

- Follow fresh Ollama platform updates

73

74

75

# Spring Boot AI tool calls examples for JiRack DeltaNet series

76

- Tool call library on java for Enterprise https://github.com/alibaba/spring-ai-alibaba

77

78

# GoEx AI tool calls examples for JiRack DeltaNet series

79

- Tool call library on python https://github.com/ShishirPatil/gorilla

80

81

# JiRack DeltaNet tool calls to boost tool call quality

82

- Use JiRack Precision tokenizer tags for tool calls with ToolBench https://github.com/OpenBMB/ToolBench

83

- https://huggingface.co/xalss/Qwen2-7B-Instruct-glaive-function-calling

84

- https://huggingface.co/datasets/NousResearch/hermes-function-calling-v1

85

- Add JiRack tool call tags in the dataset and modify tool call processor if needed

86

87

88

# JiRack RoboTech

89

- Advanced Tokenizer with Robotics & Routing & Tool calls Tokenizer and other

90

- [CMSManhattan/JiRackDeltaNetTokenizer](https://huggingface.co/CMSManhattan/JiRackDeltaNetTokenizer)

91

92

93

## Available Variants on Docker with UI

94

| Tag | Quant | Size | Approx. RAM | Description |

95

|-----|-------|------|-------------|-------------|

96

97

| `cmsmanhattan/jirack_deltanet_27b-cpu-q4:latest` | Q4_K_M | ~16.8 GB | ~18–24 GB | Recommended balance |

98

| `cmsmanhattan/jirack-deltanet-27b-cpu-q3:latest` | Q3_K_M | ~13.9 GB | ~15–20 GB | Good quality / size trade-off |

99

| `cmsmanhattan/jirack-deltanet-27b-cpu-q2:latest` | Q2_K | ~11.2 GB | ~12–17 GB | Maximum compression |

100

101

## Quick Start

102

103

### Run with Ollama (recommended — reasoning off by default)

104

- https://ollama.com/cmsmanhattan

105

```bash

106

ollama pull cmsmanhattan/JiRackDeltaNet_27b-q4-reasoning

107

ollama run cmsmanhattan/JiRackDeltaNet_27b-q4-reasoning "What is the capital of France?"

108

```

109

110

### Run with Docker

111

- 27B docker images can be provided by request.

112

- Build docker on local from source or request from me.

113

- Docker images use hidden reasoning logic to make chat clear

114

115

116

**Q8/int8**

117

```bash

118

docker run -d \

119

--name jirack_deltanet_27b \

120

-p 7869:7869 \

121

--cpus=16 \

122

-e THREADS=16 \

123

-e THREADS_BATCH=16 \

124

--restart unless-stopped \

125

cmsmanhattan/jirack_deltanet_27b-cpu-q8:latest

126

```

127

128

**Q6**

129

```bash

130

docker run -d \

131

--name jirack_deltanet_27b \

132

-p 7869:7869 \

133

--cpus=16 \

134

-e THREADS=16 \

135

-e THREADS_BATCH=16 \

136

--restart unless-stopped \

137

cmsmanhattan/jirack_deltanet_27b-cpu-q6:latest

138

```

139

140

**Default CPU (Q4/int4 recommended)**

141

```bash

142

docker run -d \

143

--name jirack_deltanet_27b \

144

-p 7869:7869 \

145

--restart unless-stopped \

146

cmsmanhattan/jirack_deltanet_27b-cpu-q4:latest

147

```

148

**Q3**

149

```bash

150

docker run -d \

151

--name jirack_deltanet_27b \

152

-p 7869:7869 \

153

--cpus=16 \

154

-e THREADS=16 \

155

-e THREADS_BATCH=16 \

156

--restart unless-stopped \

157

cmsmanhattan/jirack_deltanet_27b-cpu-q3:latest

158

```

159

**Q2 (lowest memory)**

160

```bash

161

docker run -d \

162

--name jirack_deltanet_27b \

163

-p 7869:7869 \

164

--cpus=16 \

165

-e THREADS=16 \

166

-e THREADS_BATCH=16 \

167

--restart unless-stopped \

168

cmsmanhattan/jirack_deltanet_27b-cpu-q2:latest

169

```

170

**Full precision**

171

```bash

172

docker run -d \

173

--name jirack_deltanet_27b \

174

-p 7869:7869 \

175

--cpus=16 \

176

-e THREADS=16 \

177

-e THREADS_BATCH=16 \

178

--restart unless-stopped \

179

cmsmanhattan/jirack-deltanet-27b-cpu:latest

180

```

181

**Multi CPU**

182

```bash

183

docker run -d \

184

--name jirack_deltanet_27b \

185

-p 7869:7869 \

186

--cpus=16 \

187

-e THREADS=16 \

188

-e THREADS_BATCH=16 \

189

--restart unless-stopped \

190

--memory=32g \

191

--cpus=8 \

192

cmsmanhattan/jirack-deltanet-27b-q4:latest

193

```

194

195

### Docker Compose Example

196

```yaml

197

services:

198

jirack:

199

image: cmsmanhattan/jirack_deltanet_27b-cpu-q4:latest

200

container_name: jirack_deltanet_27b

201

ports:

202

- "7869:7869"

203

volumes:

204

- .:/app

205

- ./web:/app/web

206

environment:

207

- MAX_TOKENS=2048

208

- TEMPERATURE=0.7

209

- TOP_P=0.8

210

- TOP_K=20

211

- DEFAULT_STREAM=False

212

- INTRA_THREADS=4

213

- USE_ENV_ALLOCATOR=1

214

- THREADS=16

215

- THREADS_BATCH=16

216

deploy:

217

resources:

218

limits:

219

memory: 32g

220

```

221

222

## Ollama platform

223

- CPU without JiRack UI

224

- GPU without JiRack UI

225

- Ollama images displays reasoning logic by default but it can be off .

226

- Use JiRack UI from https://www.jirack.com

227

- cmsmanhattan/JiRackDeltaNet_27b-reasoning:latest 54 GB

228

- cmsmanhattan/JiRackDeltaNet_27b-q2-reasoning:latest 10 GB

229

- cmsmanhattan/JiRackDeltaNet_27b-q3-reasoning:latest 13 GB

230

- cmsmanhattan/JiRackDeltaNet_27b-q4-reasoning:latest 16 GB

231

- cmsmanhattan/JiRackDeltaNet_27b-q6-reasoning:latest 22 GB

232

- cmsmanhattan/JiRackDeltaNet_27b-q8-reasoning:latest 29 GB

233

234

235

## Access the UI

236

Once the container is running, open your browser and navigate to:

237

`http://localhost:7869`

238

This opens the JiRack UI — a clean web interface.

239

240

## Changing the Port

241

The listening port can be easily modified directly from the **Settings** panel within the JiRack UI.

242

243

## Licensing

244

245

- Model weights are released under the MIT License — free to use, modify, and distribute for any purpose, including commercial. No royalties, no per-user fees, no subscription.

246

- The Docker image with UI and the pre-built Ollama quantizations are separate paid products. If you prefer to build your own secure deployment — take the weights, assemble your own stack, and you're done.

247

248

- The JiRack DeltaNet 27B model for Docker and Ollama is provided under a commercial license ($12 per user per year).

249

- All JiRack UI clients are provided under a commercial license.

250

- However, the UI clients can be used for free when running together with the official JiRack Docker containers, as long as they are not redistributed separately.

251

252

For commercial licensing, cluster deployment, or enterprise use of JiRack models, please contact us.

253

- **JiRack MS Windows 11 Desktop Client (with Ollama API):**

254

https://huggingface.co/kgrabko/JiRackTernary_1b/resolve/main/jirack-chat.zip

255

- **Live email chat with the model:** support@cmsmanhattan.com

256

257

## Hardware Recommendations

258

259

### Recommended Hardware for JiRack DeltaNet 27B (single Docker container)

260

| Use Case          | CPU                        | RAM       | Recommended Quant | Expected Speed      | Recommendation |

261

|-------------------|----------------------------|-----------|--------------------|---------------------|----------------|

262

| Recommended       | Ryzen 9 / Intel i9 / Xeon  | 24–32 GB  | Q4_K_M             | Good interactive    | Best choice    |

263

| High Performance  | High-core server CPU       | 48 GB+    | Full / Q4          | Excellent           | Excellent      |

264

| Low Memory        | Modern 12+ core CPU        | 16–24 GB  | Q3_K_M or Q2_K     | Usable              | Acceptable     |

265

| Edge / Minimal    | Strong workstation CPU     | 16 GB     | Q2_K               | Acceptable          | Budget option  |

266

267

## Important Memory Notes

268

Even though the quantized 27B models are relatively compact for their size, we recommend the following for best experience:

269

- Q4_K_M: 18–24 GB system RAM minimum

270

- Q3_K_M / Q2_K: 15–20 GB system RAM

271

- Full precision: 48 GB+ system RAM recommended

272

273

Reasons for extra headroom:

274

- KV-cache consumption during generation

275

- Runtime overhead and temporary buffers

276

- System stability and avoiding out-of-memory errors

277

- Room for larger context windows

278

279

**Minimum recommended (Q4):** 18 GB system RAM

280

**Ideal:** 24–32 GB system RAM

281

282

I added the default model in full precision. This serves as the base for quantization, allowing us to find the optimal balance between model size and performance.

283

284

## Architecture Notes

285

- **Qwen3.8-style DeltaNet architecture**: hybrid attention + SSM design (`qwen35` in GGUF metadata)

286

- **Updated tokenizer**: Extended with new special tags for **Routing**, **Tool call**, and **Robotics**

287

- **No-forced-reasoning support**: chat template patched so `<think>` blocks default to closed; native support on Ollama

288

- Hidden 5120, 65 layers, attention heads 24 / KV heads 4, feed-forward 17408

289

- SSM: conv kernel 4, state size 128, group count 16, time-step rank 48, inner size 6144

290

- RoPE θ = 10,000,000, RMSNorm ε = 1e-6

291

- Context length: up to 262,144 tokens

292

- Ready-to-run GGUF quantizations (Q2_K, Q3_K_M, Q4_K_M)

293

294

## Benchmarks

295

JiRack DeltaNet 27B is built on the Qwen3.5/Qwen3.8-style DeltaNet architecture. The table below reproduces the published base-model benchmark results from [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) for reference — these reflect the upstream base model's capabilities, not JiRack-specific fine-tuning or quantization results.

296

297

### Text Performance

298

299

|                                                         | Qwen3.8-27B | Qwen3.6-27B | Qwen3.7-Plus | Muse Glimmer-30B | Opus4.6 Max |

300

|---------------------------------------------------------|-------------|-------------|--------------|-------------------|-------------|

301

| **Coding** | | | | | |

302

| Agentic terminal coding — Terminal Bench 2.1 (Terminus)  | 73.0        | 63.4        | 64.0         | 51.7              | **78.2**    |

303

| Agentic coding — SWE-bench Pro                           | **61.7**    | 53.5        | 57.6         | 51.2              | 53.4        |

304

| Repo-level code generation — NL2Repo-Bench               | 42.3        | 36.2        | 41.1         | --                | **47.6**    |

305

| Agentic coding — DeepSWE 1.1                             | **42.2**    | 13.3        | 14.2         | --                | --          |

306

| Software engineering — QwenSWEBench                      | **79.0**    | 49.3        | 59.2         | --                | 63.8        |

307

| **Agent** | | | | | |

308

| Long-horizon office work — CoWorkBench                   | **70.7**    | 61.0        | 65.1         | --                | 68.2        |

309

| Professional job tasks — JobBench                        | **33.4**    | 21.8        | 27.6         | --                | --          |

310

| Frontier agentic tasks — Agents' Last Exam (Pass@1/Score)| **20.4 / 42.9** | 10.6 / 27.3 | 13.2 / 33.6 | --          | --          |

311

| **General** | | | | | |

312

| Instruction following — IFBench                          | **79.5**    | 69.1        | 79.1         | 77.0              | 62.5        |

313

| Scientific reasoning — GPQA Diamond                       | 89.2        | 87.8        | 90.3         | 83.5              | **91.3**    |

314

| Multidisciplinary reasoning — HLE                         | 30.8        | 24.0        | 34.7         | 22.0              | **40.0**    |

315

| Competitive coding — LiveCodeBench v6                     | **90.3**    | 83.9        | 89.6         | --                | 88.8        |

316

317

### VL Performance

318

319

320

|------------------------------------------|-------------|-------------|--------------|-------------------|-------------|

321

| **Agentic Multimodal Intelligence** | | | | | |

322

| Computer use — OSWorld-Verified          | **84.3**    | 63.9        | 73.3         | 65.9              | 72.7        |

323

| Browser use — WebArena-Verified          | **64.8**    | 48.8        | 55.3         | --                | --          |

324

| Mobile use — AndroidWorld                | **81.9**    | 70.3        | 81.0         | --                | 62.0        |

325

| Application recreation — RecreationBench | **47.1**    | 29.8        | 30.2         | --                | --          |

326

| Multimodal tool use — ClawEval-MM (Pass@3/Avg) | 57.4 / 56.9 | 42.6 / 50.4 | **57.4 / 60.1** | --          | 52.5 / 54.7 |

327

| Multimodal SWE — SWE-MM                  | **38.6**    | 25.7        | 30.0         | --                | 27.1        |

328

| Visual web development — Vision2Web      | **62.9**    | 45.0        | 42.1         | --                | --          |

329

| **General Multimodal Intelligence** | | | | | |

330

| Visual math — MathVision (w/o CI / w/ CI)| 90.0 / **94.6** | 85.1 / -- | **90.3** / -- | --            | 65.5 / --   |

331

| General visual reasoning — BabyVision (w/o CI / w/ CI) | **65.7 / 85.6** | 28.9 / -- | 64.7 / 70.4 | --      | 12.6 / --   |

332

| Scientific chart analysis — CharXiv (RQ) (w/o CI / w/ CI) | 83.7 / **90.2** | 78.4 / -- | **85.8** / 85.9 | 78.8      | 66.0 / --   |

333

| Document intelligence — OmniDocBench 1.5 | 91.1        | 89.4        | **91.4**     | 75.8              | 86.6        |

334

| Real-world perception — RealWorldQA      | 85.9        | 84.1        | **86.9**     | --                | 73.9        |

335

| Embodied intelligence — ERQA             | 65.5        | 62.5        | **69.8**     | --                | 40.8        |

336

337

*Source: [Qwen/Qwen3.8-27B model card](https://huggingface.co/Qwen/Qwen3.8-27B). Best result in each row is bolded. Empty cells (--) indicate results not yet available. See the source card for full evaluation methodology and footnotes.*

338

339

## 📧 Contact & Licensing

340

For joint venture opportunities, hardware integration, or licensing inquiries:

341

- **Email:** grabko@cmsmanhattan.com

342

- **Phone:** +1 (516) 777-0945

343

- **Location:** New York, USA

344

345

346

## License

347

MIT License