Mistral Large 2 Beats Llama 405B Using a Third of the Parameters
Mistral's 123B dense flagship packs a 128K context, 80+ programming languages, and benchmarks that trade blows with Llama 3.1 405B at a third the size.
Takeaways
- Mistral Large 2 is a 123B dense transformer with a 128K context window
- Scores 84% MMLU, 92% HumanEval, 93% GSM8K on standard benchmarks
- Beats Llama 3.1 405B on code and math with under a third the parameters
- Native function calling with parallel and sequential tool use, plus JSON output
- Supports 10+ natural languages and 80+ programming languages
- Weights on Hugging Face under research-only license, commercial via API
Mistral Large 2, officially tagged Mistral-Large-Instruct-2407, is a 123B parameter dense transformer with a 128K context window, native function calling, and benchmark numbers that put it alongside GPT-4 class systems. The announcement frames it as a direct challenge to models three times its size.
Weights are live on Hugging Face under the Mistral Research License, which permits research and non-commercial modification. Commercial use goes through Mistral's API on la Plateforme under the name mistral-large-2407, or through cloud partners.
Why 123B?
Mistral sized this model to run at high throughput on a single node, which keeps it out of the awkward gap between 70B models and the multi-hundred-billion parameter tier. Serving it on one 8-GPU box avoids the multi-node inference complexity and latency penalties that come with larger architectures. The catch: you need more than 300GB of cumulative VRAM, which means 8x A100 80GB or 8x H100 in practice.
Benchmarks vs. Llama 3.1 405B
Mistral's headline claim is that Large 2 outperforms Llama 3.1 405B on code generation and math using under a third of the parameters. The model card reports: