NVIDIA's Nemotron 3.5 Lightning Runs 4x Faster to Handle AI Agents' Grunt Work
NVIDIA's new 30B mixture-of-experts model runs 3B active parameters, hits 1M context, and targets the boring execution layer of always-on agents.
Takeaways
- NVIDIA released Nemotron 3.5 Lightning, a 30B MoE with 3B active parameters for agent execution.
- Hybrid Mamba-2 + MoE + Attention architecture with a validated 1M-token context window.
- Claims up to 4x output speed versus similar-sized models via three speculative decoding strategies.
- Scores 86% on PinchBench and 52.8 on SWE-bench Verified in NVFP4 format.
- Ships alongside NeMo Switchyard, an open source library for routing agent steps between models.
- Released under OpenMDW-1.1 with full weights, training data, and recipes; runs on DGX Spark, RTX 5090, H100, or A100.
NVIDIA has released a new open model aimed at a slice of the agent stack that rarely gets the spotlight: the grunt work. Nemotron 3.5 Lightning is a 30B parameter, open Mixture-of-Experts model with 3B active parameters, optimized for high-volume, low-latency execution in always-on agents and agentic workflows. The release ships with weights, training data, recipes, and a companion routing library called NeMo Switchyard.
Long-running agents spend most of their time on high-volume execution. Tool calls, result validation, and subagent delegation dominate the token budget, and routing every one of those steps to a frontier reasoning model adds cost and latency. Lightning is designed to be the workhorse that handles those repetitive steps, leaving the frontier reasoning to bigger models when planning actually matters.
Small active footprint, million-token window
Architecturally, Lightning is a hybrid MoE that interleaves Mamba-2 blocks, MoE layers, and select Attention layers. Mamba-2 is a state-space model that scales linearly with sequence length, which is how NVIDIA squeezes a validated 1M-token context window out of a model this size. Only 3B of the 30B parameters activate per token, keeping inference cheap while giving the model enough total capacity to specialize across domains.
The default checkpoint ships in NVFP4, a 4-bit floating point format tuned for Blackwell tensor cores. That footprint is small enough to run locally. NVIDIA lists Jetson, the GeForce RTX 5090, DGX Spark, and DGX Station as supported local systems, scaling up through RTX PRO workstations to data centres and cloud. On Hopper and Ampere, the same weights run through W4A16 kernels, so an A100 or H100 can serve it without needing FP4 hardware.