Skip to main content
Back

NVIDIA Repackages Kimi-K3.1-Pro for Blackwell GPUs With Near-Zero Accuracy Loss

NVIDIA ships a Blackwell-ready NVFP4 build of Moonshot's 2.8T parameter multimodal MoE, with accuracy effectively matching the full-precision reference.

  • NVIDIA released an NVFP4 quantized Kimi-K3.1-Pro, a 2.8T param multimodal MoE with 104B active.
  • Mixed precision: routed experts NVFP4, shared experts and KDA attention MXFP8, MLA in BF16.
  • Near-lossless accuracy versus the MXFP4 baseline across GPQA Diamond, MMMU Pro, SciCode, Terminal-Bench.
  • Kimi Vendor Verifier scores match Moonshot's own serving on ToolCall, DeepSWE, MMMU-Pro.
  • Ready to serve via vLLM and SGLang on Blackwell GB300 with 8-way tensor parallelism.
  • Full 1,048,576-token context preserved; quantized with NVIDIA Model Optimizer v0.48.0.

NVIDIA has published a pre-quantized NVFP4 checkpoint of Moonshot AI’s Kimi-K3.1-Pro on Hugging Face. The checkpoint repackages the large multimodal Mixture-of-Experts model for vLLM and SGLang inference on NVIDIA Blackwell GPUs.

Blackwell operators can load the prepared weights without running their own large-scale conversion job. NVIDIA also reports scores close to Moonshot’s original MXFP4 checkpoint, providing a reference for how hardware-specific quantization affects reasoning, coding, tool use, vision, and long-context performance.

One checkpoint, five precision choices

Kimi-K3.1-Pro was post-trained from Kimi K3 and contains 2.8 trillion total parameters, with 104 billion activated for each token. Its Mixture-of-Experts architecture routes each token through a subset of experts, reducing computation while retaining a much larger expert pool in memory.

Model and deployment profile
Component Details
Architecture Multimodal Mixture of Experts
Parameters 2.8T total, 104B active per token
Context window 1,048,576 tokens
Inputs Text and RGB images through MoonViT-V2
Output Text
Target hardware NVIDIA Blackwell, tested on GB300
Serving engines vLLM and SGLang

The artifact uses several precision formats, assigning more bits to components that are sensitive to quantization:

  • Routed MoE expert weights use NVFP4, converted from MXFP4 with input_scales fixed at 1.0.
  • Shared experts and Kimi Delta Attention blocks use MXFP8, with activation scales computed at runtime.
  • Multi-head Latent Attention layers remain in BF16 to preserve accuracy.
  • The KV indexer cache uses FP8.
  • The state-space model state remains in FP32.

Why NVFP4 fits Blackwell

NVFP4 represents each value with a four-bit E2M1 floating-point format. Groups of 16 values share an E4M3 FP8 scale, with an optional tensor-level FP32 scale. MXFP4 uses groups of 32 values and an E8M0 scale. NVFP4’s smaller groups provide finer local scaling at the cost of additional scale metadata.