Skip to main content
Back

Meta's Contriever Beats BM25 With Zero Labeled Training Data

Meta AI Research showed contrastive pretraining alone can produce a dense retriever competitive with BM25, no labeled query-document pairs required.

  • Meta AI Research's Contriever trains a dense retriever with pure contrastive learning, no labeled query-document pairs.
  • Matches or beats BM25 on 11 of 15 BEIR datasets for Recall@100 with zero supervision.
  • Recipe: random cropping for positives, MoCo queue for large negative sets, mixed Wikipedia and CCNet corpus.
  • The MS MARCO fine-tuned variant plus cross-encoder re-ranking set state-of-the-art nDCG@10 on BEIR at release.
  • mContriever transfers cross-lingually to 25+ languages after English-only fine-tuning, unlike BM25.
  • Now a standard baseline retriever in RAG research, powering systems like ATLAS and frameworks like RAGLAB.

Dense retrievers built on transformers dominate leaderboards when you have millions of labeled query-document pairs. They famously fall apart when you don't. Contriever, from Meta AI Research, is the counterexample that quietly became a backbone for much of the retrieval-augmented generation work you see today. The recipe is simpler than most people assume, which explains why the model still shows up as a baseline in RAG pipelines and academic benchmarks.

The paper behind it, published in Transactions on Machine Learning Research, asks a blunt question: can you train a dense retriever with zero labeled data and still match BM25, the term-frequency method that has held its ground for two decades? The answer turns out to be yes, with a specific recipe worth breaking down.

Training a retriever without labels

Dense retrievers work by encoding a query and a document into vectors, then scoring them with a dot product. Training them traditionally requires pairs of queries and their relevant documents, which is expensive to annotate at scale and essentially unavailable for most languages. On the BEIR benchmark, prior unsupervised models were outperformed by BM25, few-shot transfer from MS MARCO underperformed, and the field lacked a strong unsupervised pretraining signal.

Contriever's answer is contrastive learning borrowed from computer vision. Every document is unique, so if you generate two views of the same document and teach the model to pull those views together while pushing other documents away, you get a retriever for free.

The recipe, piece by piece

The framework uses simple contrastive pretraining, and without any supervision the resulting model is competitive with BM25 for Recall@100 on BEIR. The concrete recipe has a few moving parts:

  • Independent cropping for positives. Instead of the older Inverse Cloze Task, they sample two random spans from the same document. Both views look like natural text, distributions match between queries and keys, and the overlap encourages some lexical matching behavior similar to BM25.