README.md · chandra-ocr-2

1

---

2

library_name: transformers

3

license: openrail

4

license_link: LICENSE

5

tags:

6

- ocr

7

- pdf

8

- markdown

9

- layout

10

---

11

12

<p align="center">

13

<img src="datalab-logo.png" alt="Datalab Logo" width="150"/>

14

</p>

15

16

# Chandra OCR 2

17

18

Chandra 2 is a state of the art OCR model from [Datalab](https://www.datalab.to) that outputs markdown, HTML, and JSON. It is highly accurate at extracting text from images and PDFs, while preserving layout information.

19

20

Try Chandra in the [free playground](https://www.datalab.to/playground), or use the [hosted API](https://www.datalab.to/) for higher accuracy and speed.

21

22

## What's New in Chandra 2

23

24

- 85.8% olmocr bench score (sota), 77.8% multilingual bench score (12% improvement over Chandra 1)

25

- Significant improvements to math, tables, complex layouts

26

- Improved layout, especially on wider documents

27

- Significantly better image captioning

28

- 90+ language support with major accuracy gains

29

30

## Features

31

32

- Convert documents to markdown, HTML, or JSON with detailed layout information

33

- Excellent handwriting support

34

- Reconstructs forms accurately, including checkboxes

35

- Strong performance with tables, math, and complex layouts

36

- Extracts images and diagrams, with captions and structured data

37

- Support for 90+ languages

38

39

<img src="handwritten_form.png" width="600px"/>

40

41

## Quickstart

42

43

```shell

44

pip install chandra-ocr

45

46

# With vLLM (recommended, easy install)

47

chandra_vllm

48

chandra input.pdf ./output

49

50

# With HuggingFace (requires torch)

51

pip install chandra-ocr[hf]

52

chandra input.pdf ./output --method hf

53

```

54

55

## Usage

56

57

### With vLLM (recommended)

58

59

```python

60

from chandra.model import InferenceManager

61

from chandra.model.schema import BatchInputItem

62

from PIL import Image

63

64

# Start vLLM server first with: chandra_vllm

65

manager = InferenceManager(method="vllm")

66

batch = [

67

BatchInputItem(

68

image=Image.open("document.png"),

69

prompt_type="ocr_layout"

70

)

71

]

72

result = manager.generate(batch)[0]

73

print(result.markdown)

74

```

75

76

### With HuggingFace Transformers

77

78

```python

79

from transformers import AutoModelForImageTextToText, AutoProcessor

80

from chandra.model.hf import generate_hf

81

from chandra.model.schema import BatchInputItem

82

from chandra.output import parse_markdown

83

from PIL import Image

84

import torch

85

86

model = AutoModelForImageTextToText.from_pretrained(

87

"datalab-to/chandra-ocr-2",

88

dtype=torch.bfloat16,

89

device_map="auto",

90

)

91

model.eval()

92

model.processor = AutoProcessor.from_pretrained("datalab-to/chandra-ocr-2")

93

model.processor.tokenizer.padding_side = "left"

94

95

batch = [

96

BatchInputItem(

97

image=Image.open("document.png"),

98

prompt_type="ocr_layout"

99

)

100

]

101

102

result = generate_hf(batch, model)[0]

103

markdown = parse_markdown(result.raw)

104

print(markdown)

105

```

106

107

## Benchmarks

108

109

### olmOCR Benchmark

110

111

<img src="bench.png" width="600px"/>

112

113

| **Model** |  ArXiv   | Old Scans Math |  Tables  | Old Scans | Headers and Footers | Multi column | Long tiny text | Base |    Overall     | Source |

114

|:----------|:--------:|:--------------:|:--------:|:---------:|:-------------------:|:------------:|:--------------:|:----:|:--------------:|:------:|

115

| Datalab API               | **90.4** | **90.2** | 90.7 | **54.6** |        91.6         |     83.7     |      92.3      | **99.9** | **86.7 ± 0.8** | Own benchmarks |

116

| Chandra 2                 |   86.9   |   89.1   | **92.1** |   51.1   |        91.4         |     82.1     |    **93.7**    | **99.9** |   85.8 ± 0.8   | Own benchmarks |

117

| dots.ocr 1.5              |   85.9   |   85.5   | 90.7 |   48.2   |        94.0         |   **85.3**   |      81.6      | 99.7 |   83.9         | dots.ocr repo |

118

| Chandra 1                 |   82.2   |   80.3   |   88.0   |   50.4   |        90.8         |     81.2     |      92.3      | **99.9** |   83.1 ± 0.9   | Own benchmarks |

119

| olmOCR 2                  |   83.0   |   82.3   |   84.9   |   47.7   |      **96.1**       |     83.7     |      81.9      | 99.6 |   82.4         | olmocr repo |

120

| dots.ocr                  |   82.1   |   64.2   |   88.3   |   40.9   |        94.1         |     82.4     |      81.2      | 99.5 |   79.1 ± 1.0   | dots.ocr repo |

121

| olmOCR v0.3.0             |   78.6   |   79.9   |   72.9   |   43.9   |        95.1         |     77.3     |      81.2      | 98.9 |   78.5 ± 1.1   | olmocr repo |

122

| Datalab Marker v1.10.0    |   83.8   |   69.7   |   74.8   |   32.3   |        86.6         |     79.4     |      85.7      | 99.6 |   76.5 ± 1.0   | Own benchmarks |

123

| Deepseek OCR              |   75.2   |   72.3   |   79.7   |   33.3   |      **96.1**       |     66.7     |      80.1      | 99.7 |   75.4 ± 1.0   | Own benchmarks |

124

| Mistral OCR API           |   77.2   |   67.5   |   60.6   |   29.3   |        93.6         |     71.3     |      77.1      | 99.4 |   72.0 ± 1.1   | olmocr repo |

125

| GPT-4o (Anchored)         |   53.5   |   74.5   |   70.0   |   40.7   |        93.8         |     69.3     |      60.6      | 96.8 |   69.9 ± 1.1   | olmocr repo |

126

| Qwen 3 VL 8B              |   70.2   |   75.1   |   45.6   |   37.5   |        89.1         |     62.1     |      43.0      | 94.3 |   64.6 ± 1.1   | Own benchmarks |

127

| Gemini Flash 2 (Anchored) |   54.5   |   56.1   |   72.1   |   34.2   |        64.7         |     61.5     |      71.5      | 95.6 |   63.8 ± 1.2   | olmocr repo |

128

129

## Examples

130

131

| Type | Name | Link |

132

|------|------|------|

133

| Tables | Statistical Distribution | [View](https://github.com/datalab-to/chandra/blob/master/assets/examples/tables/complex_tables.png) |

134

| Tables | Financial Table | [View](https://github.com/datalab-to/chandra/blob/master/assets/examples/tables/financial_table.png) |

135

| Forms | Registration Form | [View](https://github.com/datalab-to/chandra/blob/master/assets/examples/forms/handwritten_form.png) |

136

| Forms | Lease Form | [View](https://github.com/datalab-to/chandra/blob/master/assets/examples/forms/lease_filled.png) |

137

| Math | CS229 Textbook | [View](https://github.com/datalab-to/chandra/blob/master/assets/examples/math/cs229.png) |

138

| Math | Handwritten Math | [View](https://github.com/datalab-to/chandra/blob/master/assets/examples/math/handwritten_math.png) |

139

| Math | Chinese Math | [View](https://github.com/datalab-to/chandra/blob/master/assets/examples/math/chinese_math.png) |

140

| Handwriting | Cursive Writing | [View](https://github.com/datalab-to/chandra/blob/master/assets/examples/handwriting/cursive_writing.png) |

141

| Handwriting | Handwritten Notes | [View](https://github.com/datalab-to/chandra/blob/master/assets/examples/handwriting/handwritten_notes.png) |

142

| Languages | Arabic | [View](https://github.com/datalab-to/chandra/blob/master/assets/examples/languages/arabic.png) |

143

| Languages | Japanese | [View](https://github.com/datalab-to/chandra/blob/master/assets/examples/languages/japanese.png) |

144

| Languages | Hindi | [View](https://github.com/datalab-to/chandra/blob/master/assets/examples/languages/hindi.png) |

145

| Languages | Russian | [View](https://github.com/datalab-to/chandra/blob/master/assets/examples/languages/russian.png) |

146

| Other | Charts | [View](https://github.com/datalab-to/chandra/blob/master/assets/examples/other/charts.png) |

147

| Other | Chemistry | [View](https://github.com/datalab-to/chandra/blob/master/assets/examples/other/chemistry.png) |

148

149

150

### Multilingual Benchmark (43 Languages)

151

152

The table below covers the 43 most common languages, benchmarked across multiple models. For a comprehensive evaluation across 90 languages (Chandra 2 vs Gemini 2.5 Flash only), see the [full 90-language benchmark](#full-90-language-benchmark).

153

154

<img src="multilingual.png" width="600px"/>

155

156

157

|---|:---:|:---:|:---:|:---:|:---:|

158

| ar | 67.6% | 68.4% | 34.0% | 84.4% | 55.6% |

159

| bn | 85.1% | 72.8% | 45.6% | 55.3% | 23.3% |

160

| ca | 88.7% | 85.1% | 84.2% | 88.0% | 78.5% |

161

| cs | 88.2% | 85.3% | 84.7% | 79.1% | 78.8% |

162

| da | 90.1% | 91.1% | 88.4% | 86.0% | 87.7% |

163

| de | 93.8% | 94.8% | 83.0% | 88.3% | 93.8% |

164

| el | 89.9% | 85.6% | 85.5% | 83.5% | 82.4% |

165

| es | 91.8% | 89.3% | 88.7% | 86.8% | 97.1% |

166

| fa | 82.2% | 75.1% | 69.6% | 61.8% | 56.4% |

167

| fi | 85.7% | 83.4% | 78.4% | 86.0% | 84.7% |

168

| fr | 93.3% | 93.7% | 89.6% | 86.1% | 91.1% |

169

| gu | 73.8% | 70.8% | 44.6% | 47.6% | 11.5% |

170

| he | 76.4% | 70.4% | 38.9% | 50.9% | 22.3% |

171

| hi | 80.5% | 78.4% | 70.2% | 82.7% | 41.0% |

172

| hr | 93.4% | 90.1% | 85.9% | 88.2% | 81.3% |

173

| hu | 88.1% | 82.1% | 82.5% | 84.5% | 84.8% |

174

| id | 91.3% | 91.6% | 86.7% | 88.3% | 89.7% |

175

| it | 94.4% | 94.1% | 89.1% | 85.7% | 91.6% |

176

| ja | 87.3% | 86.9% | 85.4% | 80.0% | 76.1% |

177

| jv | 87.5% | 73.2% | 85.1% | 80.4% | 69.6% |

178

| kn | 70.0% | 63.2% | 20.6% | 24.5% | 10.1% |

179

| ko | 89.1% | 81.5% | 82.3% | 84.8% | 78.4% |

180

| la | 78.0% | 73.8% | 55.9% | 70.5% | 54.6% |

181

| ml | 72.4% | 64.3% | 18.1% | 23.8% | 11.9% |

182

| mr | 80.8% | 75.0% | 57.0% | 69.7% | 20.9% |

183

| nl | 90.0% | 88.6% | 85.3% | 87.5% | 83.8% |

184

| no | 89.2% | 90.3% | 85.5% | 87.8% | 87.4% |

185

| pl | 93.8% | 91.5% | 83.9% | 89.7% | 90.4% |

186

| pt | 97.0% | 95.2% | 84.3% | 89.4% | 90.8% |

187

| ro | 86.2% | 84.5% | 82.1% | 76.1% | 77.3% |

188

| ru | 88.8% | 85.5% | 88.7% | 82.8% | 72.2% |

189

| sa | 57.5% | 51.1% | 33.6% | 44.6% | 12.5% |

190

| sr | 95.3% | 90.3% | 82.3% | 89.7% | 83.0% |

191

| sv | 91.9% | 92.8% | 82.1% | 91.1% | 92.1% |

192

| ta | 82.9% | 77.7% | 50.8% | 53.9% | 8.1% |

193

| te | 69.4% | 58.6% | 19.5% | 33.3% | 9.9% |

194

| th | 71.6% | 62.6% | 47.0% | 66.7% | 53.8% |

195

| tr | 88.9% | 84.1% | 68.1% | 84.1% | 78.2% |

196

| uk | 93.1% | 91.0% | 88.5% | 87.9% | 81.9% |

197

| ur | 54.1% | 43.2% | 28.1% | 57.6% | 16.9% |

198

| vi | 85.0% | 80.4% | 81.6% | 89.5% | 83.6% |

199

| zh | 87.8% | 88.7% | 88.3% | 70.0% | 70.4% |

200

| **Average** | **80.4%** | **77.8%** | **69.4%** | **67.6%** | **60.5%** |

201

202

### Full 90-Language Benchmark

203

204

We also have a more comprehensive evaluation covering 90 languages, comparing Chandra 2 against Gemini 2.5 Flash. The average scores are lower than the 43-language table above because this includes many lower-resource languages. Chandra 2 averages **72.7%** vs Gemini 2.5 Flash at **60.8%**.

205

206

See the [full 90-language results](https://github.com/datalab-to/chandra/blob/master/FULL_BENCHMARKS.md).

207

208

## Throughput

209

210

Benchmarked with vLLM on a single NVIDIA H100 80GB GPU using a diverse mix of documents (math, tables, scans, multi-column layouts) from the olmOCR benchmark set.  This set is significantly slower than real-world usage - we estimate 2 pages/s in real-world usage.

211

212

213

|---|:---:|:---:|:---:|:---:|

214

| vLLM, 96 concurrent sequences | 1.44 | 60s | 156s | 0% |

215

216

## Commercial Usage

217

218

Code is Apache 2.0. Model weights use a modified OpenRAIL-M license: free for research, personal use, and startups under $2M funding/revenue. Cannot be used competitively with our API. For broader commercial licensing, see [pricing](https://www.datalab.to/pricing?utm_source=gh-chandra).

219

220

## Credits

221

222

- [Huggingface Transformers](https://github.com/huggingface/transformers)

223

- [vLLM](https://github.com/vllm-project/vllm)

224

- [olmocr](https://github.com/allenai/olmocr)

225

- [Qwen 3.5](https://github.com/QwenLM/Qwen3)