Skip to main content
Back

FlashMap Drafts 16 Dependent Tokens at Once, Hitting 6.1x Faster Decoding

A flow-map drafter conditions on hidden states to generate entire token blocks in one shot, hitting 6.1x lossless speedups on Qwen3-8B.

  • FlashMap is a new speculative decoding drafter based on continuous flow maps.
  • Generates an entire token block in one pass while modeling inter-token dependencies.
  • Delivers up to 6.1x lossless speedup on Qwen3-8B, beating DFlash on every benchmark.
  • Conditions the flow map on target model hidden states for better acceptance rates.
  • Open-source code with SGLang and vLLM integration available.
  • Pretrained drafters for Qwen3-8B and Qwen3-4B released on Hugging Face.

The FlashMap paper introduces a speculative-decoding drafter that generates an entire token block with a continuous flow map. The method retains parallel block generation while modeling dependencies among tokens within the block.

On Qwen3-8B, the authors report up to 6.1x lossless acceleration over ordinary target-model decoding and a 7% higher average speedup than a matched DFlash baseline.

Why drafting still costs time

Speculative decoding pairs an accurate target model with a smaller drafter to reduce the number of sequential target-model calls. Each decoding cycle follows four steps:

  1. The drafter proposes a block of candidate tokens.
  2. The target model scores the full block in parallel.
  3. The verifier accepts the longest valid prefix.
  4. After a rejection, decoding resumes from a target-model token.

In this context, “lossless” means the verifier preserves the target model’s output distribution under the same sampling settings. Greedy decoding produces the same result as ordinary target-model decoding when the verifier is implemented correctly.

Drafter Block generation Token dependencies Primary cost
Autoregressive, including EAGLE-style models Sequential Each token sees earlier draft tokens Proposal latency grows with block length
DFlash-style diffusion Parallel Positions are predicted independently under the paper’s formulation Acceptance declines at later block positions
FlashMap Parallel flow map The block is modeled jointly