README.md
4.7 KB · 102 lines · markdown Raw
1 ---
2 license: apache-2.0
3 pipeline_tag: tabular-classification
4 tags:
5 - arxiv:2609.04540
6 ---
7
8 # Mitra-v2 Classifier
9
10 Mitra-v2 classifier is a tabular foundation model that is pre-trained on purely synthetic datasets sampled from a mix of random classifiers, including the new Hybrid SCM prior. It is the second generation of the Mitra classifier ([autogluon/mitra-classifier](https://huggingface.co/autogluon/mitra-classifier)), pre-trained with a 10x longer context, three times as many features, and an improved optimizer. On the TabArena and TALENT benchmarks it delivers state-of-the-art accuracy at the level of TabFM and EXAONE Tabular, while surpassing TabPFN-3 by a wide margin. The regression model is at [autogluon/mitra-regressor-2](https://huggingface.co/autogluon/mitra-regressor-2), and the inference and fine-tuning code with our evaluation results is at [autogluon/mitra-finetune](https://huggingface.co/autogluon/mitra-finetune).
11
12 ## Architecture
13
14 Mitra-v2 is based on a 12-layer 2D Transformer of 75.7 M parameters (attention across rows and across columns), pre-trained by incorporating an in-context learning paradigm. The architecture is unchanged from Mitra-v1; the gains come from the scaled-up synthetic pre-training distribution and the optimizer.
15
16 ## Usage
17
18 To use Mitra-v2 classifier, install AutoGluon and the `mitra-finetune` package by running:
19
20 ```sh
21 pip install uv
22 uv pip install "autogluon.tabular[mitra]>=1.6" "tabarena>=0.1.0"
23 uv pip install git+https://huggingface.co/autogluon/mitra-finetune
24 ```
25
26 A minimal example showing how to fine-tune and predict with the Mitra-v2 classifier using the same recipe as our reported results (50-step fine-tuning with 8-fold bagging). The recipe fine-tunes and bags eight copies of the model and requires a CUDA GPU; each `predict_proba` or `predict` call runs one bagged fine-tune:
27
28 ```python
29 import pandas as pd
30 from sklearn.model_selection import train_test_split
31 from sklearn.datasets import load_wine
32 from huggingface_hub import snapshot_download
33 from mitra_finetune import MitraFinetune
34
35 # Load dataset
36 wine_data = load_wine()
37 X = pd.DataFrame(wine_data.data, columns=wine_data.feature_names)
38 y = pd.Series(wine_data.target, name="target")
39 X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42, stratify=y)
40
41 # Download the Mitra-v2 classifier weights
42 ckpt_dir = snapshot_download("autogluon/mitra-classifier-2")
43
44 # Fine-tune and predict
45 model = MitraFinetune(checkpoint_dir=ckpt_dir, problem_type="classification")
46 model.fit(X_train, y_train)
47 proba = model.predict_proba(X_test)
48 pred = proba.argmax(axis=1)
49 print("Accuracy:", (pred == y_test.values).mean())
50 ```
51
52 A minimal example showing how to perform inference with the Mitra-v2 classifier directly in AutoGluon (the weights are a drop-in replacement for the Mitra-v1 classifier):
53
54 ```python
55 from autogluon.tabular import TabularDataset, TabularPredictor
56
57 train_data = TabularDataset(pd.concat([X_train, y_train], axis=1))
58 test_data = TabularDataset(pd.concat([X_test, y_test], axis=1))
59
60 mitra_predictor = TabularPredictor(label="target")
61 mitra_predictor.fit(
62 train_data,
63 hyperparameters={
64 "MITRA": {"hf_model": "autogluon/mitra-classifier-2", "fine_tune": False}
65 },
66 )
67
68 mitra_predictor.leaderboard(test_data)
69 ```
70
71 Set `"fine_tune": True` to fine-tune inside AutoGluon. Note that AutoGluon's stock defaults differ from the `mitra-finetune` recipe used for the reported benchmark numbers.
72
73 ## License
74
75 This project is licensed under the Apache-2.0 License.
76
77 ## Reference
78
79 [Mitra-v2 Technical Report](https://arxiv.org/abs/2609.04540) (Amazon, 2026), also available on the [Hub](https://huggingface.co/autogluon/mitra-finetune/blob/main/Mitra_v2_Technical_Report.pdf).
80
81 ```
82 @article{mitrav2_2026,
83 title={{Mitra-v2} Technical Report},
84 author={Tao, Yefan and Zhang, Xiyuan and Liu, Xinyi and Han, Boran and Maddix, Danielle and Fang, Haoyang and Han, Zhen and Gai, Jiading and Liu, Xuanqing and Bohlke-Schneider, Michael and Wang, Yuyang (Bernie) and Friedland, Gerald and Mah, Kevan and Lee, Chris and Kong, Chris},
85 journal={arXiv preprint arXiv:2609.04540},
86 year={2026}
87 }
88 ```
89
90 The original Mitra:
91
92 ```
93 @article{zhang2025mitra,
94 title={Mitra: Mixed synthetic priors for enhancing tabular foundation models},
95 author={Zhang, Xiyuan and Maddix, Danielle C and Yin, Junming and Erickson, Nick and Ansari, Abdul Fatir and Han, Boran and Zhang, Shuai and Akoglu, Leman and Faloutsos, Christos and Mahoney, Michael W and others},
96 journal={arXiv preprint arXiv:2510.21204},
97 year={2025}
98 }
99 ```
100
101 Amazon Science blog: [Mitra: Mixed synthetic priors for enhancing tabular foundation models](https://www.amazon.science/blog/mitra-mixed-synthetic-priors-for-enhancing-tabular-foundation-models)
102