Skip to main content
Back

Sarvam AI Open-Sources Sarvam-30B, Beating Larger Models on Indian Languages

Sarvam open-sources a 30B Mixture-of-Experts reasoning model with 2.4B active parameters, tuned for 22 Indian languages and built entirely on Indian compute.

Model, Full Access
  • Sarvam AI open-sourced Sarvam-30B, a 30B MoE reasoning model with 2.4B active parameters, under Apache 2.0.
  • Trained from scratch on 16T tokens using IndiaAI Mission compute, entirely in India.
  • Wins 89% of pairwise comparisons on Indian language benchmarks across 22 scheduled languages and 12 scripts.
  • Delivers 3x to 6x higher throughput than Qwen3 on H100 with a rebuilt inference stack.
  • Ships with FP8, GGUF, and MXFP4 variants for server, edge, and Apple Silicon deployment.
  • Full technical details on the Sarvam blog, powering the Samvaad voice agent platform in production.

Sarvam AI has open-sourced Sarvam-30B, a Mixture-of-Experts reasoning model that punches well above its active-parameter weight and is engineered from the ground up for Indian languages. It ships under Apache 2.0 alongside a larger 105B sibling, with weights available on Hugging Face and India's AI Kosh registry.

The entire stack, from tokenizer to inference kernels, was rebuilt with Indic deployment in mind. Both models are reasoning-oriented and trained from scratch on datasets curated in-house across every stage: pre-training, supervised fine-tuning, and reinforcement learning. All compute ran inside India under the IndiaAI mission.

Inside the model

Sarvam-30B is a Mixture-of-Experts model with 2.4B non-embedding active parameters, designed for practical deployment. It combines reasoning, coding, and conversational quality across Indian languages. That active-parameter number is the headline: only a small slice of the network fires per token, keeping inference cheap.

The architecture makes several deliberate choices for throughput:

  • 19 layers, a dense FFN intermediate size of 8192, MoE intermediate size of 1024, top-6 routing, grouped KV heads (num_key_value_heads=4), and an extremely high rope_theta of 8e6 for long-context stability without RoPE scaling
  • 128 experts with a shared expert, a routed scaling factor of 2.5, and auxiliary-loss-free router balancing
  • A 32K context window for conversational workloads, per the accompanying architecture summary

The auxiliary-loss-free balancing is worth flagging. Traditional MoE training adds a load-balancing loss to prevent a few experts from hogging traffic, but this can pull against the language-modeling objective. Sarvam instead uses a bias term inside the router to nudge utilization uniform. Sigmoid-based routing scores, rather than traditional softmax gating, were used to reduce routing collapse during training.

How it was trained

Pre-training ran on 16T tokens for the 30B, spanning code, general web data, specialized knowledge corpora, mathematics, and multilingual content. A substantial portion of the multilingual budget went to the ten most-spoken Indian languages. Training was split into three phases covering long-horizon pre-training, mid-training, and long-context extension.