Skip to main content
Back

Inco AI Ships DFlash 2 Draft Model to Triple GLM-5.3-Flash Token Throughput

Inco AI's DFlash 2 drafter for GLM-5.3-Flash now ships under Apache 2.0, delivering near 3x speculative decoding speedups with lossless output.

  • Inco AI released a DFlash 2 drafter for GLM-5.3-Flash under Apache 2.0.
  • Block diffusion drafter predicts entire token blocks in parallel, output stays lossless versus the target model.
  • Delivers close to 3x throughput of autoregressive decoding with roughly 1% added cycle latency.
  • Two tricks: top-16 candidate path selector plus two-tap dynamic convolutions to fight suffix decay.
  • Gains 21% more accepted tokens over DFlash and 0.48 over DSpark on Qwen3.5-4B benchmarks.
  • Runs in SGLang today via the DFLASH speculative algorithm; full writeup here.

Inco AI has released a 1-billion-parameter DFlash 2 draft model for Zhipu AI’s GLM-5.3-Flash. The model checkpoint integrates with SGLang and uses speculative decoding to accelerate generation while preserving the target model’s output. Its repository now carries an Apache 2.0 license, allowing commercial use, modification, and distribution under the license terms.

The checkpoint is a specialized drafter, so requests still run through GLM-5.3-Flash. It proposes blocks of likely tokens, and the target model verifies those proposals before returning output. Accepted tokens reduce the number of target-model decoding steps; rejected tokens are replaced through the normal verification process.

Parallel blocks change the latency math

Conventional speculative decoding uses a smaller autoregressive model to propose tokens sequentially, then verifies several proposals with one target-model pass. The approach saves target-model work, though sequential drafting adds latency before each verification step.

DFlash predicts an entire draft block in parallel and retains several candidates at each position. DFlash 2 adds a lightweight path selector that connects those candidates into a coherent sequence. Inco reports more than 20% additional accepted output per verification pass over DFlash, with about 1% more draft-and-verify cycle latency in its evaluated configurations.

Target verification provides the lossless guarantee. Greedy decoding returns the same token sequence as the target model, while sampled decoding preserves the target distribution when the speculative algorithm is implemented correctly. Speed comes from accepting multiple proposals after one target pass, with invalid branches discarded before output.