Skip to main content
Back

Google DeepMind's EmbeddingGemma Beats Models Twice Its Size on Your Device

Google's 308M-parameter open embedding model tops the MTEB leaderboard under 500M, runs in sub-200MB RAM, and works fully offline.

  • Google released EmbeddingGemma, a 308M-parameter open text embedding model based on Gemma 3.
  • State-of-the-art on MTEB for models under 500M parameters, matching models nearly twice its size.
  • Runs in under 200MB RAM with quantization, sub-15ms inference on EdgeTPU at 256 tokens.
  • Supports 100+ languages, 2K context, and Matryoshka output dimensions from 768 down to 128.
  • Works with sentence-transformers, llama.cpp, Ollama, MLX, LiteRT, LangChain, and LlamaIndex.
  • Trained on 320B tokens using encoder-decoder init and geometric distillation from larger teachers.

Google DeepMind has released EmbeddingGemma, a compact text embedding model that punches well above its weight class. The 308 million parameter design targets on-device AI, enabling techniques like Retrieval Augmented Generation (RAG) and semantic search that run directly on your hardware, even without an internet connection.

You no longer need to ship every query to a cloud embedding API to get frontier-quality vectors. On a laptop, phone, or edge device, EmbeddingGemma produces embeddings competitive with models roughly twice its size, at a fraction of the memory footprint.

What's under the hood

EmbeddingGemma is a 300M parameter open embedding model built from Gemma 3 (with T5Gemma initialization) and the same research stack behind Gemini. It's a 24-layer encoder-only model that generates 768-dimensional embeddings and supports a context window of 2,048 tokens.

The 308M parameter total breaks down into roughly 100M model parameters and 200M embedding parameters. It's available now on Hugging Face, Kaggle, and Vertex AI under the Gemma license, free to download and run locally.

Benchmarks that matter

On the Massive Text Embedding Benchmark, EmbeddingGemma achieves state-of-the-art results across multilingual, English, and code domains, outperforming prior top models (both proprietary and open) with fewer than 500M parameters, and matching models twice its size. That lead persists when quantizing weights or truncating embedding outputs.

That last point is the interesting one for anyone deploying at scale. Most embedding models degrade sharply when you compress their vectors or quantize their weights. EmbeddingGemma's ranking holds up under both, which means the on-device numbers aren't marketing artifacts from an unquantized reference implementation.