Skip to main content
Back

DeepSeek's V4-Flash Runs 1M Tokens at 54x Cheaper Than Claude

DeepSeek releases V4-Flash-Base, a 284B open-weight MoE model with 1M-token context and a new hybrid attention architecture that slashes inference costs

Model, Full Access
  • DeepSeek-V4-Flash-Base released: 284B open-weight MoE model, 13B active params, MIT license, available on Hugging Face.
  • 1M-token context by default: New hybrid attention (CSA + HCA) cuts KV cache to 7% of V3.2's footprint at 1M tokens, making long context actually affordable.
  • Pricing undercuts all frontier models: V4-Flash costs $0.14/M input and $0.28/M output — 18x cheaper than GPT-5.4, with a 98% discount on cached tokens.
  • Agent-first design: V4 preserves reasoning traces across multi-turn tool calls, fixes JSON tool-call parsing failures with a new XML schema, and was trained with RL in a purpose-built sandbox.
  • Self-hosting is non-trivial: Flash requires 96GB+ VRAM minimum; no stable llama.cpp/Ollama support yet — API is the practical path for most teams.
  • Migration deadline: Legacy deepseek-chat and deepseek-reasoner endpoints retire July 24, 2026 — update to deepseek-v4-flash now.

DeepSeek-V4-Flash-Base is the raw pretrained checkpoint behind DeepSeek's new V4-Flash model , a 284-billion parameter Mixture-of-Experts (MoE) model that activates only 13 billion parameters per token. It ships alongside DeepSeek-V4-Pro-Base as part of the broader V4 family, all released under the MIT license with open weights on Hugging Face. The headline number is a native 1 million token context window , and unlike most models that technically "support" long contexts, V4 was architecturally redesigned from the ground up to make that context window actually affordable to use.

A new architecture, not just more parameters

V4 is not V3.2 with more parameters. It's a new architecture with a fundamentally different approach to long context. The core innovation is a hybrid attention stack that replaces standard transformer attention , which grows quadratically with sequence length , with two complementary mechanisms that work together across layers.

  • Compressed Sparse Attention (CSA): CSA compresses KV entries by 4x along the sequence dimension using softmax-gated pooling with a learned positional bias. A lightning indexer (FP4, ReLU-scored multi-head dot product) picks the top-k compressed blocks per query. Think of it as the model's sharp, selective memory , it finds the most relevant chunks of the context without reading everything.
  • Heavily Compressed Attention (HCA): HCA compresses KV entries by 128x and drops the sparse selection. Every query attends densely to every compressed block. The compressed sequence is short enough that dense attention is cheap. This gives the model a cheap, global view of the full context.
  • Sliding window: Both branches keep a 128-token sliding window of recent, uncompressed tokens so the model never loses fidelity on the most recent text.

The result of interleaving these two mechanisms across layers is dramatic. At 1M tokens, V4-Flash drops to 10% of the per-token FLOPs and 7% of the KV cache of V3.2. That's not a marginal improvement , it's the difference between a 1M context window being a datacenter-only feature and something you can actually run.