Skip to main content
Back

Z Lab's DFlash Hits 4.3x Faster LLM Inference With Parallel Diffusion Drafting

DFlash's block-diffusion draft model hits 4.3x throughput on Qwen 3.5 397B, now the default speculative decoding engine in SGLang

  • 4.3x throughput on Qwen 3.5 397B-A17B: DFlash + SGLang's new Spec V2 engine beats baseline and native MTP speculation on every benchmark tested.
  • Block diffusion drafting: DFlash proposes an entire block of tokens in a single forward pass, unlike autoregressive drafters that generate tokens one at a time.
  • KV injection: Target model hidden states are injected into every draft layer's KV cache, keeping acceptance rates high even with deep draft models.
  • Now SGLang's default: Spec V2 with overlap scheduling (33%+ speedup alone) is now the default speculative decoding engine in SGLang.
  • Production-proven: The same DFlash recipe powers Xiaomi MiMo at over 1,000 tokens/second in production.
  • Free and open: Weights are on Hugging Face under Apache 2.0; training code is in the DFlash GitHub repo.

Speculative decoding has long been the go-to trick for squeezing more tokens per second out of large language models without touching model quality. The idea is simple: a small, fast "draft" model proposes several tokens at once, and the big target model verifies them all in a single parallel pass. If the drafts are good, you get a free speedup. If not, you fall back gracefully. The catch has always been that most draft models are themselves autoregressive -- they still generate tokens one at a time, which caps how much you can actually gain.

DFlash, from Z Lab, breaks that ceiling. DFlash is a speculative decoding framework that employs a lightweight block diffusion model for parallel drafting, generating draft tokens in a single forward pass. Now, in a joint release between Z Lab, Modal, and the SGLang team, DFlash and Spec V2 hit more than 4.3x baseline throughput for LLM inference and are now the default speculative decoding engine in SGLang. The team has also released a new DFlash draft model specifically trained for Qwen 3.5 397B-A17B, available now on Hugging Face under the Apache 2.0 license.

The bottleneck that everyone ignored

Autoregressive LLMs require inherently sequential decoding, leading to high inference latency and poor GPU utilization. Speculative decoding mitigates this bottleneck, but existing methods still rely on autoregressive drafting, which remains sequential and limits practical speedups. Methods like EAGLE and the native multi-token prediction (MTP) modules built into models like DeepSeek-V4 still generate draft tokens one by one inside the draft model -- a poor fit for the massively parallel hardware we run inference on.

Diffusion LLMs offer a promising alternative by enabling parallel generation, but current diffusion models typically underperform compared with autoregressive models. DFlash threads this needle by using diffusion only where it helps -- in the draft stage -- while letting the verified autoregressive target model guarantee output quality.