Cohere Labs' Aya Vision 8B Beats Models 10x Its Size Across 23 Languages
Cohere Labs' Aya Vision 8B brings genuinely multilingual image-text understanding to 23 languages, beating models 10x its size on key benchmarks
Takeaways
- Cohere Labs released Aya Vision 8B, an open-weight vision-language model supporting image+text understanding across 23 languages.
- The 8B model beats Qwen2.5-VL 7B, Pixtral 12B, Gemini Flash 1.5 8B, and Llama-3.2 11B Vision with up to 81% win rates on multilingual benchmarks.
- It even wins 63% of head-to-head comparisons against Llama-3.2 90B Vision, a model over 10x its size.
- Architecture combines Command R7B (text), SigLIP2 (vision encoder), and a multimodal adapter, trained in two stages with multilingual synthetic data and model merging.
- Available free for non-commercial use on Hugging Face under CC-BY-NC 4.0; also deployable via vLLM, SGLang, and Docker.
- Weak spots include fine-grained OCR tasks like reading serial numbers or precise receipt totals, where GPT-4o still leads.
Aya Vision 8B is Cohere Labs' open-weight vision-language model built from the ground up for multilingual use. Most multimodal models are English-first with multilingual support bolted on. Aya Vision flips that priority: it was designed to understand and reason about images across 23 languages simultaneously, and the benchmark numbers back that up.
The gap it's filling
The Aya Vision family addresses one of the biggest challenges in AI: bringing multilingual performance to multimodal models. The problem is real and persistent. Existing open-weight vision-language models (VLMs) are trained predominantly on English data, which means their image-understanding capabilities degrade sharply when users interact in Arabic, Hindi, Vietnamese, or Greek. While AI has made significant progress, a large gap remains in model performance across different languages, especially in multimodal tasks involving text and images.
What it is and how it's built
Aya Vision 8B is a vision-language model that uses a multilingual language model based on Command R7B and further post-trained with the Aya Expanse recipe, paired with SigLIP2-patch14-384 vision encoder through a multimodal adapter for vision-language understanding. Think of it as three components snapped together: a strong multilingual text brain, a vision encoder that reads images, and a connector layer that translates between the two.
Image handling is designed for real-world messiness. Aya Vision uses 169 visual tokens to encode an image tile at 364x364 pixels, supports up to 12 input tiles plus a thumbnail, totaling up to 2,197 image tokens, and handles a 16K context length. To keep inference fast despite that token count, the team uses a technique called Pixel Shuffle, which compresses image tokens by 4x before passing them to the language model decoder.