Skip to main content
Back

Cohere Labs' Aya Expanse Beats Llama and Gemma Across 23 Languages

Cohere Labs' Aya Expanse 8B brings serious multilingual muscle to the open-weights arena, beating Gemma 2 9B and Llama 3.1 8B across 23 languages with novel training tricks.

  • Aya Expanse 8B is an open-weight 8B multilingual model from Cohere Labs supporting 23 languages, available on Hugging Face.
  • It beats Gemma 2 9B, Llama 3.1 8B, and Ministral 8B in multilingual benchmarks with win-rates up to 76.6%.
  • Training combines three novel techniques: data arbitrage (multi-teacher synthetic data), multilingual preference training, and model merging by language family.
  • Supports Arabic, Chinese, Hindi, Japanese, Korean, Russian, Vietnamese and 16 more languages with an 8K context window.
  • Free for research under CC-BY-NC; runs on 16GB+ VRAM (8GB quantized); compatible with Transformers, vLLM, Ollama, and LM Studio.
  • A 32B version is also available with 128K context, outperforming Llama 3.1 70B on multilingual tasks.

Most open-weight LLMs are quietly English-first. They support other languages on paper, but their actual performance degrades fast once you leave the Latin-script, high-resource comfort zone. Aya Expanse 8B from Cohere Labs is a direct attack on that problem: an 8-billion-parameter model built from the ground up to be a genuinely strong multilingual model, not just an English model with translation bolted on.

The gap it's closing

The Aya Expanse family was built to address the critical challenge of developing highly performant multilingual models that match or surpass the capabilities of monolingual ones. That framing matters. The goal isn't just "supports 23 languages" -- it's parity with monolingual models in each of those languages. Cohere For AI, the research arm of Cohere, has been working on closing language divides in foundational AI datasets and models.

Evaluations on the Arena-Hard-Auto dataset, translated into 23 languages, show that Aya Expanse 8B outperforms leading open-weight models in its parameter class, including Gemma 2, Qwen 2.5, and Llama 3.1, achieving up to a 76.6% win-rate. Those are pairwise win-rates judged by GPT-4o, which is a conservative setup -- GPT-4o-mini scores showed even stronger results.

Three research bets that paid off

The performance gains didn't come from one trick. Aya Expanse leverages several years of research at Cohere For AI and Cohere, including advancements in data arbitrage, multilingual preference training, and model merging. Each of these is a distinct research contribution, and they stack.

Data Arbitrage -- Instead of relying on a single "teacher" model to generate synthetic training data (the standard approach), Cohere trains a pool of models, one per language group, then uses a reward model (called an Arbiter) to pick the best completion for each prompt across the whole pool. This sidesteps a core problem: no single model is the best teacher across all 23 languages. The SFT model trained with this technique alone already showed a 9.1% win-rate improvement over the previous Aya 23 model against Gemma 2 9B.