NVIDIA Repackages Kimi-K3.1-Pro for Blackwell GPUs With Near-Zero Accuracy Loss
NVIDIA ships a Blackwell-ready NVFP4 build of Moonshot's 2.8T parameter multimodal MoE, with accuracy effectively matching the full-precision reference.
Takeaways
- NVIDIA released an NVFP4 quantized Kimi-K3.1-Pro, a 2.8T param multimodal MoE with 104B active.
- Mixed precision: routed experts NVFP4, shared experts and KDA attention MXFP8, MLA in BF16.
- Near-lossless accuracy versus the MXFP4 baseline across GPQA Diamond, MMMU Pro, SciCode, Terminal-Bench.
- Kimi Vendor Verifier scores match Moonshot's own serving on ToolCall, DeepSWE, MMMU-Pro.
- Ready to serve via vLLM and SGLang on Blackwell GB300 with 8-way tensor parallelism.
- Full 1,048,576-token context preserved; quantized with NVIDIA Model Optimizer v0.48.0.
NVIDIA has published a pre-quantized NVFP4 checkpoint of Moonshot AI’s Kimi-K3.1-Pro on Hugging Face. The checkpoint repackages the large multimodal Mixture-of-Experts model for vLLM and SGLang inference on NVIDIA Blackwell GPUs.
Blackwell operators can load the prepared weights without running their own large-scale conversion job. NVIDIA also reports scores close to Moonshot’s original MXFP4 checkpoint, providing a reference for how hardware-specific quantization affects reasoning, coding, tool use, vision, and long-context performance.
One checkpoint, five precision choices
Kimi-K3.1-Pro was post-trained from Kimi K3 and contains 2.8 trillion total parameters, with 104 billion activated for each token. Its Mixture-of-Experts architecture routes each token through a subset of experts, reducing computation while retaining a much larger expert pool in memory.
| Component | Details |
|---|---|
| Architecture | Multimodal Mixture of Experts |
| Parameters | 2.8T total, 104B active per token |
| Context window | 1,048,576 tokens |
| Inputs | Text and RGB images through MoonViT-V2 |
| Output | Text |
| Target hardware | NVIDIA Blackwell, tested on GB300 |
| Serving engines | vLLM and SGLang |
The artifact uses several precision formats, assigning more bits to components that are sensitive to quantization:
- Routed MoE expert weights use NVFP4, converted from MXFP4 with
input_scalesfixed at1.0. - Shared experts and Kimi Delta Attention blocks use MXFP8, with activation scales computed at runtime.
- Multi-head Latent Attention layers remain in BF16 to preserve accuracy.
- The KV indexer cache uses FP8.
- The state-space model state remains in FP32.
Why NVFP4 fits Blackwell
NVFP4 represents each value with a four-bit E2M1 floating-point format. Groups of 16 values share an E4M3 FP8 scale, with an optional tensor-level FP32 scale. MXFP4 uses groups of 32 values and an E8M0 scale. NVFP4’s smaller groups provide finer local scaling at the cost of additional scale metadata.