Skip to main content
Back

DeepSeek's V3.2-Exp Slashes Long-Context Costs 50% With Sparse Attention

DeepSeek's experimental V3.2 model swaps in a two-stage sparse attention mechanism, delivers V3.1-parity benchmarks, and cuts API prices by 50%+.

Model, Full Access
  • DeepSeek released V3.2-Exp, a 685B model built on V3.1-Terminus with new sparse attention.
  • DeepSeek Sparse Attention uses a lightning indexer plus fine-grained token selector to skip irrelevant context.
  • Benchmarks show near-parity with V3.1-Terminus; some agentic tool-use scores actually improved.
  • DeepSeek API prices cut by 50%+, effective at launch, reflecting the efficiency gains.
  • Day-0 support in vLLM and SGLang across H100/H200 and Blackwell B200/GB200 hardware.
  • RoPE bug in reference indexer code was fixed; older checkpoints of the demo may underperform.

DeepSeek has quietly shipped one of the more interesting architectural experiments in open-weight LLMs this year. DeepSeek-V3.2-Exp is a 685B parameter model that swaps standard attention for a new mechanism called DeepSeek Sparse Attention (DSA), aimed at making long-context inference cheaper without sacrificing quality. It ships under MIT license, in FP8, with day-0 support in vLLM and SGLang, and it comes bundled with a 50%+ price cut on the DeepSeek API.

An intermediate step, not a full generation

The Exp suffix is deliberate. V3.2-Exp builds on V3.1-Terminus by introducing DeepSeek Sparse Attention, a mechanism designed to explore and validate optimizations for training and inference efficiency in long-context scenarios. Think of it as a controlled A/B test the DeepSeek team ran in public: keep everything else about V3.1-Terminus fixed, swap the attention mechanism, and see what happens.

To evaluate the impact rigorously, the team aligned the training configurations of V3.2-Exp with V3.1-Terminus. That makes the benchmark comparison unusually clean, and the results are essentially a wash: MMLU-Pro tied at 85.0, AIME 2025 up slightly (89.3 vs 88.4), Codeforces up (2121 vs 2046), GPQA-Diamond down a bit (79.9 vs 80.7), HMMT 2025 down (83.6 vs 86.1). Agentic tool-use scores actually improved on BrowseComp, BrowseComp-zh, and Terminal-bench.

Cost per million tokens comparison between V3.1-Terminus and V3.2-Exp across prefill and decode

Inside DeepSeek Sparse Attention

Traditional attention is O(n²): every token attends to every other token, so doubling the context quadruples compute. DSA breaks that pattern with a two-stage pipeline. A module called a "lightning indexer" prioritizes specific excerpts from the context window. A separate "fine-grained token selection system" then picks specific tokens from within those excerpts to load into the model's limited attention window. Together they let the model operate over long portions of context with comparatively small server loads.