Skip to main content
Back

NVIDIA's Kumo Tabular Tops 8 Benchmarks on Spreadsheet Data Without Training

NVIDIA's technical report on Kumo Tabular explains how a 214M-parameter open model trained only on synthetic tables tops eight tabular leaderboards and still predicts quickly.

  • NVIDIA's Kumo Tabular report details open tabular foundation models at 27M, 61M and 214M parameters.
  • Kumo Tabular-L ranks first on eight benchmarks, including TabArena, BeyondArena, TALENT, ScoringBench and TabBench-Bio.
  • Kumo Tabular-L predicts 36x faster than LimiX-2 on TabArena and 1.7x faster than TabPFN-3.5 on BeyondArena.
  • Pretrained only on synthetic causal-graph tables with realistic noise and missing values. No real data is used.
  • It is weaker on grouped splits (fourth place) and tables above 100K training rows (ninth place).
  • Weights are on Hugging Face (OpenMDW-1.1). The code and prior are on GitHub (Apache 2.0).

NVIDIA has published a full technical report on Kumo Tabular, an open family of tabular foundation models whose largest checkpoint ranks first on eight benchmarks in the company’s evaluation. Kumo targets spreadsheet-style data such as loan applications, sensor logs, and patient records. It uses labeled rows as context to predict classification labels or regression values for unlabeled rows, without per-dataset training, hyperparameter search, or manual feature engineering.

The downloadable artifacts predate the report. At publication, Hugging Face showed more than 280,000 downloads over the preceding month, while the source repository contained the pretraining code, inference code, and synthetic data generator. The report now documents the architecture, training recipe, synthetic data, and ablations that isolate the contribution of individual changes.

Why tables need pretraining

Gradient-boosted decision trees (GBDTs) have long been the default for structured data. A conventional workflow fits each dataset separately: collect labels, engineer features, tune hyperparameters, and validate the resulting model. Knowledge learned from one table does not transfer to the next task.

Tabular foundation models (TFMs) are pretrained across millions of prediction tasks. At inference, a new table’s labeled rows become context for predicting its unlabeled rows, much as examples in a large language model prompt establish a task. TabPFN introduced this approach for small classification tables. Later systems added regression and support for larger datasets, while the open TabICLv2 model approached the leading closed systems. Kumo builds directly on TabICLv2 and competes primarily with LimiX-2 and TabPFN-3.5.

From cells to predictions

  1. Cell embedding. Each cell token combines a value with two neighboring cells in the same row. Values become sine and cosine features at learned frequencies. Missing values receive a learned embedding instead of imputed values, and training-row tokens include the target label.
  2. Table encoder. Column attention, which compares values within a column, alternates with row attention across features. Earlier encoders completed all column layers before starting row layers, preventing column attention from using cross-column information. Kumo passes column attention through learned inducing points that summarize each column, reducing computation to linear growth with row count. Four readout tokens then compress each row into a fixed-size embedding.
  3. In-context transformer. Query rows attend to labeled context rows, while context rows attend to one another. Their internal key and value representations can therefore be computed once and cached. The M and L models use two shared key-value heads for query rows through a grouped-query attention variant called Test-GQA, reducing cache size by 4 times for M and 8 times for L. The output head produces logits for up to 10 classes or 999 quantiles that approximate a regression distribution.

A length-aware attention temperature scales the attention logits as tables grow, preventing the softmax distribution from becoming too diffuse across tens of thousands of rows or many columns. The S model has 27 million parameters, M has 61 million, and L has 214 million. Between 83% and 95% of those parameters reside in the in-context transformer.