Skip to main content
Back

Alibaba's Qwen3-Embedding-4B Beats Gemini's Best at Half the Size

Qwen releases a 4B open-weight embedding model that hits 69.45 on MTEB Multilingual, closes in on the 8B leader, and supports 100+ languages with 32K context.

  • Qwen3-Embedding-4B released under Apache 2.0 as part of a 0.6B/4B/8B series on Hugging Face.
  • Scores 69.45 mean task on MTEB Multilingual, closing the gap with the 8B flagship at 70.58.
  • Supports 100+ languages, 32K context, and Matryoshka output dimensions from 32 to 2560.
  • Built on Qwen3-4B-Base via LoRA, with three-stage training and synthetic pairs from Qwen3 itself.
  • Instruction-aware queries deliver 1-5% retrieval boost, top score on instruction-retrieval subtask.
  • Works with sentence-transformers, vLLM, and TEI out of the box.

Alibaba's Qwen team has released a new open-weight text embedding model that punches well above its weight class. Qwen3-Embedding-4B sits in the middle of a three-model release (0.6B, 4B, and 8B) and posts benchmark numbers within striking distance of the 8B flagship, while remaining small enough to serve on a single mid-range GPU.

The whole series ships under Apache 2.0 on Hugging Face and ModelScope, with the technical report and training code on GitHub. That makes it one of the strongest fully open alternatives to closed embedding APIs like OpenAI's text-embedding-3-large and Google's Gemini embedding.

What is actually in the box

The Qwen3 Embedding series is built on the Qwen3 foundation model and targets text embedding, retrieval, and reranking. The 4B variant is fine-tuned from Qwen3-4B-Base and ships with these specs:

  • 4B parameters, 36 layers, BF16 weights
  • 32K token context window
  • Embedding dimension up to 2560, with user-defined output sizes from 32 to 2560 (Matryoshka Representation Learning)
  • Support for 100+ languages, including programming languages
  • Instruction-aware: each query can carry a task description that steers the embedding

Alongside the embedding models, Qwen shipped a matching set of rerankers (0.6B, 4B, 8B), so you can build a full retrieve-then-rerank pipeline from one model family.

Benchmark reality check

On the MTEB Multilingual leaderboard, the 8B model took the top spot as of June 5, 2025 with a score of 70.58. The 4B version lands at 69.45 mean task score, roughly one point behind its bigger sibling while using half the parameters. For comparison,