Skip to main content
Back

IBM's Granite Embedding Packs 8K Context Into a Tiny 47M Model

IBM's 47M parameter Granite embedding model brings 8K context, ModernBERT speed, and Apache 2.0 licensing to compact retrieval workloads.

  • IBM released granite-embedding-small-english-r2, a 47M parameter English embedding model under Apache 2.0.
  • Produces 384 dim vectors with 8192 token context, built on ModernBERT with Flash Attention 2.
  • Hits 55.6 average NDCG@10 across six MTEB retrieval benchmarks, beating sub-100M open competitors.
  • Encodes around 199 docs per second on an H100, roughly 44% faster than e5-small-v2 and bge-small-en-v1.5.
  • Trained without MS-MARCO, using only enterprise friendly data plus IBM internal and synthetic pairs.
  • See the Granite Embedding R2 paper and the GitHub repo for training details.

IBM shipped a new compact text embedding model in its Granite lineup, and it is punching well above its weight. Granite-embedding-small-english-r2 is a 47M parameter bi-encoder that produces 384 dimensional vectors, supports an 8192 token context window, and lands on Hugging Face under Apache 2.0 with several million downloads already logged.

The combination is what stands out. Most sub-100M open embedding models cap at 512 tokens, which forces aggressive chunking for anything longer than a short passage. Granite r2 gives you 16x that budget in a model small enough to run on a CPU.

Small model, ModernBERT bones

The r2 refresh swaps the old encoder backbone for ModernBERT, and that choice is doing most of the work. The architecture ships with alternating attention window lengths to speed up processing, rotary position embeddings for extended sequences, a freshly trained tokenizer optimized for code and text, Flash Attention 2 support, and no bias terms in the linear layers.

The specs are compact by design:

  • 47M parameters, 12 layers, 12 attention heads
  • 384 dimensional output embeddings
  • 1536 intermediate size with GeGLU activations
  • 50,368 token vocabulary
  • 8192 max sequence length

The benchmark story

On IBM's own aggregate of six MTEB retrieval benchmarks, the small model averages 55.6 NDCG@10. That trails its 149M sibling at 59.5, but sits comfortably ahead of the usual sub-100M references like