DeepSeek Opens DSpark, Boosting AI Inference Speed up to 400%
DeepSeek open-sources DeepSpec, a full-stack toolkit for training and evaluating speculative decoding draft models, alongside the DSpark algorithm paper
Takeaways
- DeepSeek open-sourced DeepSpec, a full-stack MIT-licensed codebase for training and evaluating speculative decoding draft models.
- Three algorithms included out of the box: DSpark (DeepSeek's own), DFlash, and Eagle3, with support for Qwen and Gemma target models.
- DSpark is already live in production on DeepSeek-V4 Flash and Pro, delivering 51%--400% throughput gains and 16.3%--30.9% better token acceptance over prior methods.
- DSpark's key innovation is a semi-autoregressive draft architecture paired with confidence-scheduled verification that skips low-probability tokens under high load.
- The DeepSeek-V4-Pro-DSpark checkpoint is on Hugging Face -- the same base model with the DSpark module attached, no architecture changes.
- Storage caveat: the target cache for training can reach ~38 TB for a Qwen3-4B setup, so plan accordingly before running the pipeline.
DeepSeek just open-sourced DeepSpec, a full-stack Python codebase for training and evaluating speculative decoding draft models. The release comes bundled with a technical paper for DSpark, a novel speculative decoding algorithm already running in production on DeepSeek-V4 Flash and Pro. Together, they represent DeepSeek's most complete public contribution to the inference efficiency space yet.
What is speculative decoding, and why does it matter?
Speculative decoding is a technique that accelerates LLM inference without altering the model's output distribution. The core idea is to introduce a lightweight "draft model" that pre-generates a set of candidate tokens, which are then batch-validated and accepted or rejected by the target model. This transforms serial, token-by-token generation into parallel batch verification, substantially reducing end-to-end latency.
While foundation models such as DeepSeek, Qwen, and GLM have rapidly advanced to state-of-the-art performance, high-quality draft models remain scarce and underdeveloped, leaving substantial headroom for improving the effectiveness of speculative decoding. DeepSpec is a direct answer to that gap.
What DeepSpec actually is
DeepSpec is a full-stack codebase for training and evaluating draft models for speculative decoding. It contains data preparation utilities, draft model implementations, training code, and evaluation scripts. The workflow is a clean three-stage pipeline:
- Data Preparation -- download prompts, regenerate target answers, and build a target cache.
- Training -- train a draft model against the cached target outputs.
- Evaluation -- measure speculative-decoding acceptance rates on benchmark tasks.