Skip to main content
Back

Microsoft's Harrier Tops Multilingual Benchmarks as a Free Open-Source Alternative

Microsoft's Bing team just dropped a three-model embedding family topping the multilingual MTEB v2 leaderboard with a 32k token context and MIT licensing.

Model, Full Access
  • Microsoft's Bing team released Harrier-OSS-v1, three multilingual embedding models topping the MTEB v2 leaderboard.
  • Family includes 270M, 0.6B, and 27B parameter variants, all MIT licensed on Hugging Face.
  • All three support a 32,768 token context, covering 100+ languages with decoder-only architectures.
  • Built on Gemma 3 and Qwen 3 backbones with last-token pooling instead of BERT-style mean pooling.
  • Trained on 2B+ contrastive examples plus synthetic GPT-5 data, with distillation for smaller variants.
  • The 27B flagship scores 74.3 on multilingual MTEB v2, beating proprietary models from OpenAI and Amazon.

Microsoft's Bing team has quietly shipped one of the more interesting open-weight releases in the retrieval space this year. Harrier-OSS-v1 is a family of three multilingual text embedding models, and according to internal numbers, all three land at the top of the multilingual MTEB v2 leaderboard at release time. The 27B flagship posts a score of 74.3, while the smaller 0.6B and 270M siblings hit 69.0 and 66.5 respectively.

The release matters because Harrier came out of Bing's production search stack rather than a research lab, and the entire family ships under an MIT license on Hugging Face. If you have been paying OpenAI or Cohere per million tokens for multilingual embeddings, there is now a self-hostable option that claims to beat them on the standard benchmark.

The lineup at a glance

Microsoft split the family into three sizes so you can pick a tradeoff between latency and quality. All three share the same 32,768 token context window, the headline capability change compared to older BERT-derived embedding models.

ModelParametersEmbedding dimMax tokensMTEB v2
harrier-oss-v1-270m270M64032,76866.5
harrier-oss-v1-0.6b0.6B1,02432,76869.0
harrier-oss-v1-27b27B5,37632,76874.3

Goodbye BERT, hello decoder-only

The architectural choice is the most interesting part of this release. The Harrier family drops the traditional bidirectional encoder architecture that has dominated the embedding landscape since Sentence-BERT and moves entirely to decoder-only backbones. The 270M and 27B variants are built on Gemma 3, while the 0.6B variant uses Qwen 3.

Rather than averaging token representations, Harrier uses causal models with last-token pooling and L2 normalization to generate semantic vectors. In practice the model runs a normal decoder forward pass and then takes the hidden state of the final non-padding token as the sentence embedding.