Skip to main content
Back

Unsloth's Qwen3.5-122B Runs 2x Faster on Local Hardware Without a Draft Model

Unsloth ships GGUF quants of Qwen3.5-122B-A10B with built-in Multi-Token Prediction, delivering 1.5-2x faster local inference via llama.cpp.

  • Unsloth released GGUF quants of Qwen3.5-122B-A10B preserving Multi-Token Prediction heads for speculative decoding.
  • MTP delivers 1.5-2x faster generation once llama.cpp is built from the MTP PR branch, merged May 16.
  • Model is 122B total, 10B activated MoE with 256 experts, hybrid Gated DeltaNet plus attention, 262K native context.
  • Scores 86.7 MMLU-Pro, 72.0 SWE-bench Verified, 49.4 Terminal Bench 2, beating GPT-5-mini on terminal tasks.
  • Quants range from 38.7 GB (1-bit) to 249 GB (BF16); UD-Q4_K_XL at 78.6 GB is the recommended sweet spot.
  • MTP mode currently disables multimodal input and parallel batching; use standard Qwen3.5 checkpoints for vision workloads.

Running a 122B-parameter multimodal model on your own hardware just got noticeably faster. Unsloth's new MTP-enabled GGUF release of Qwen3.5-122B-A10B pairs Alibaba's mid-tier flagship with Multi-Token Prediction speculative decoding that llama.cpp only started supporting a few days earlier, and the combination is producing 1.5-2x throughput gains for local users.

The headline change is baked into the filename. Standard GGUF quants of Qwen3.5 already exist, but this variant preserves the model's MTP layers so llama.cpp can use them as a self-drafting speculator. The upstream PR that made this possible was merged into llama.cpp on May 16, and Unsloth notes that MTP speculative decoding delivers roughly 1.5-2x faster generation once you build llama.cpp from the MTP PR branch.

Self-drafting, no second model required

Speculative decoding normally requires a smaller draft model running alongside the main one. MTP folds that draft into the base model itself: extra heads predict several tokens ahead in a single forward pass, and the main model verifies them in parallel. If the guesses match, you emit multiple tokens per step instead of one. Reports from NVFP4 deployments on DGX Spark hardware suggest MTP acceptance rates can approach 100% on suitable workloads, which is where the throughput multiplier comes from.