Cohere's North Mini Code Beats Models 40x Larger at Agentic Coding
Cohere open-sources North Mini Code, a 30B MoE coding model with 3B active params, built for agentic software engineering and on-prem deployment
Takeaways
- First open-source model from Cohere: North Mini Code is a 30B MoE model with only 3B active parameters, released under Apache 2.0.
- Strong benchmark performance: Scores 33.4 on the Artificial Analysis Coding Index, beating Qwen3.5 35B, Gemma 4 26B, and even models 4x its size like Devstral 2 (123B).
- 2.8x throughput advantage: Outperforms Devstral Small 2 in output throughput and inter-token latency under identical hardware conditions.
- Trained with multi-harness RLVR: Post-trained using verifiable rewards across 70k+ tasks from ~5k repositories, with explicit cross-harness generalization for real-world agent compatibility.
- Available now for free: Weights on Hugging Face, deployable via vLLM, SGLang, Ollama, or Cohere's managed API; minimum hardware is 1x H100 at FP8.
- Sovereign AI focus: Designed for on-prem and air-gapped deployment, with community feedback explicitly shaping future model releases.
Cohere just made its first move into open-source coding models, and it's a meaningful one. North Mini Code is a 30-billion-parameter Mixture-of-Experts (MoE) model -- a type of architecture where only a fraction of the network is active at any time -- that activates just 3 billion parameters per token. The result is a model that punches above its weight class in both performance and speed, available today under an Apache 2.0 license.
This is Cohere's first open-source model aimed squarely at developers, and it signals a strategic shift: the company known for enterprise-grade closed APIs is now betting on community-driven development to shape its next generation of models.
What it's built for
North Mini Code is optimized for code generation, agentic software engineering, and terminal tasks. The emphasis on "agentic" here is deliberate -- this isn't just a code-completion model. It is specifically designed and trained for agentic software engineering tasks , meaning it's meant to operate inside multi-step agent loops: understanding a codebase, writing and executing code, running terminal commands, and orchestrating sub-agents.
The practical use cases Cohere highlights include:
- Autonomous software engineering workflows (fixing GitHub issues, implementing features)
- Terminal-based agentic tasks (navigating filesystems, running shell commands)
- Complex code generation for scientific and algorithmic problems
- Code review and system architecture mapping
Benchmarks: competitive at its size class
On Artificial Analysis' Coding Index, North Mini Code achieves a score of 33.4, outperforming Qwen3.5 (35B-A3B), Gemma 4 (26B-A4B), Devstral Small 2 (24B Dense), and even substantially larger models such as Nemotron 3 Super (120B-A12B), Mistral Small 4 (119B-A6B), and Devstral 2 (123B). That last part is the headline: a 3B-active-parameter model beating models with 10-40x more active compute is a strong signal of training efficiency.

Speed is another strong suit. North Mini Code achieved up to 2.8x higher output throughput than Devstral Small 2 under identical concurrency levels and hardware configurations, and demonstrated a 30% advantage in inter-token latency. Time-to-first-token is one area where Devstral Small 2 maintains a slight edge, so for latency-sensitive single-shot completions, the gap narrows.
The architecture: sparse MoE with hybrid attention
North-Mini-Code-1.0 is a decoder-only Transformer-based sparse Mixture-of-Experts model. It uses an efficient attention implementation, interleaved between sliding-window attention with RoPE and global attention with no positional embeddings, in a 3:1 ratio. Sliding-window attention (which only attends to nearby tokens) is cheaper to compute, while global attention (which can attend to any token) handles long-range dependencies -- alternating between them at a 3:1 ratio keeps costs low without sacrificing context understanding.
The feed-forward block is an MoE block with 128 experts, of which 8 are activated per token. Each expert block is an FFN block with SwiGLU activation. The router applies a sigmoid activation function to the logits before the top-k selection. The model also supports a 256K token context window with up to 64K tokens of output -- unusually generous for a model this size.
How it was trained
The training pipeline is where things get technically interesting. The post-training uses a two-stage cascaded supervised fine-tuning (SFT) followed by reinforcement learning with verifiable rewards (RLVR), focusing on agentic coding. The first stage SFT data focuses on coding capabilities integrated within a wider mix: 70% code datasets, 43% agentic tool-use data, and 27% single-turn competitive or scientific programming data.
RLVR -- reinforcement learning where the reward signal comes from verifiable outcomes like passing unit tests rather than human preference scores -- is increasingly the dominant approach for coding models. In total, Cohere used over 70k verifiable tasks across ~5k unique repositories, deduplicating environments against SWE-Bench and SWE-Bench-Pro to avoid source leakage during evaluation.
One notable training decision: rather than optimizing for a single agent harness (like SWE-Agent), Cohere trained North Mini Code using multiple scaffolds rather than optimizing for a single one. This matters because real-world coding agents vary wildly in how they expose tools -- some use rich CLI interfaces, others just raw bash. Cross-harness transfer can be cheaply acquired without degrading benchmark performance.
The RL phase itself used an asynchronous training setup to handle the highly variable length of coding agent rollouts. A trainer runs alongside a vLLM sidecar that serves rollouts continuously, with policy weights exported into vLLM every few learner steps, so the sampler is at most slightly off-policy at any moment. The algorithm used is CISPO, a log-likelihood objective with token-level importance sampling correction that enhances RLOO with stronger regularization.
RLVR training improved the final model from the SFT initialization by 7.9% (absolute) pass@1 in Terminal-Bench v2 and 3.0% (absolute) in SWE-Bench. Human evaluators also preferred the RLVR model over the SFT-only checkpoint with a 66.1% aggregate win rate.
Getting your hands on it
The weights are freely available on Hugging Face under an Apache 2.0 license. You can deploy via Model Vault (Cohere's managed inference platform), try it in OpenCode, or use a Cohere API key. The minimum hardware requirement is a single H100 at FP8 precision. An FP8-quantized version is also available for those who want to reduce memory footprint further.
For local serving with vLLM, the setup looks like this:
pip install "git+https://github.com/vllm-project/vllm.git"
pip install cohere_melody>=0.9.0
vllm serve CohereLabs/North-Mini-Code-1.0 \
-tp 2 \
--max-model-len 320000 \
--tool-call-parser cohere_command4 \
--reasoning-parser cohere_command4 \
--enable-auto-tool-choiceThe model supports interleaved thinking -- it can emit chain-of-thought reasoning tokens before tool calls -- and Cohere strongly recommends keeping those reasoning traces in the context window across agent turns for best performance. It's also compatible with Transformers, SGLang, and quantized via llama.cpp, Ollama, and LM Studio.
The bigger picture
The open-source coding model space has become intensely competitive. Mistral's Devstral, Qwen's coding variants, and Google's Gemma all target the same efficient-but-capable niche. What makes North Mini Code's positioning distinct is the explicit focus on sovereign AI -- the ability to run entirely on your own infrastructure, free from vendor API dependencies.
North Mini Code advances Cohere's mission to make sovereign AI a practical reality, giving developers direct access to agentic coding capabilities. For teams in regulated industries, air-gapped environments, or those simply wary of sending proprietary code to third-party APIs, a permissively licensed, on-prem-deployable coding agent is a genuinely useful addition to the toolkit.
Cohere is also framing this as a community-driven project, explicitly asking developers to share builds and feedback to shape future iterations. Whether that translates into a sustained open-source commitment or a one-time release remains to be seen -- but the model weights are live on Hugging Face right now, and the official documentation includes deployment guides and cookbooks to get started.