Amazon's Mitra Beats Gradient Boosted Trees on Small Tabular Data
AutoGluon's Mitra brings foundation-model style in-context learning to tabular regression, pretrained entirely on synthetic data and shipping in AutoGluon 1.4.
Takeaways
- AutoGluon released Mitra Regressor, a 72M-parameter tabular foundation model with Apache-2.0 weights.
- Pretrained purely on synthetic data mixing structural causal models with tree-based priors.
- Uses in-context learning: training rows are fed as context, no per-task retraining required.
- State-of-the-art on TabRepo, TabZilla, AMLB and TabArena for small datasets under 5,000 rows.
- Ships inside AutoGluon 1.4 via
pip install autogluon.tabular[mitra], GPU or CPU. - Full details in the NeurIPS 2025 paper on mixed synthetic priors.
Tabular data has stubbornly resisted the foundation model wave that swept through text and vision. Gradient boosted trees still win most Kaggle contests, and every new dataset typically means training a fresh model from scratch. Mitra Regressor, from the AutoGluon team at Amazon, is one of the more serious attempts to change that. Its weights are now openly available on Hugging Face alongside a classifier sibling.
Trained on no real spreadsheets
Mitra is a tabular foundation model pre-trained entirely on synthetic datasets sampled from a mixture of random regressors. The architecture is modest: a 12-layer, 72M-parameter Transformer trained with in-context learning. Rather than ingesting scraped CSVs, it learned exclusively from tasks generated on the fly from a curated blend of prior distributions. Real-world tabular data tends to be limited and heterogeneous, with varying feature types, dependencies, and noise levels, so simulating diverse synthetic datasets that span a wide range of data patterns is more practical than collecting them. If you have used TabPFN, the philosophy will feel familiar, but Mitra pushes harder on which priors go into the pretraining mixture.
The mixed-priors recipe
The core research contribution, described in the NeurIPS 2025 paper, is a principled method for selecting which synthetic data generators to combine. The authors find that effective priors yield good performance on real tasks, exhibit diversity that prevents overfitting, and contribute patterns not found in other priors. Applying those three criteria, the team settled on a specific blend: structural causal models, which combine graphs of causal dependencies between variables with probabilistic equations describing how each variable affects its dependents, plus tree-based methods including gradient boosting, random forests, and decision trees. At inference time, the pretrained model conditions on labeled training samples from the downstream task as in-context examples, giving it broad task coverage without requiring large amounts of real downstream data.