Skip to main content
Back

MTPLX Doubles Local AI Speed on Apple Silicon Without Losing Quality

A new MLX runtime uses a model's built-in multi-token prediction heads to speed up local LLM inference on Macs by roughly 2x, without changing outputs.

  • MTPLX is a native Mac app and CLI running local LLMs roughly 2x faster on Apple Silicon.
  • Uses the target model's built-in MTP heads as the drafter, no second model loaded into RAM.
  • Exact Leviathan-Chen rejection sampling preserves the output distribution at temperature 0.6.
  • Serves OpenAI and Anthropic-compatible APIs at 127.0.0.1:8000 for Claude Code, Cline, Continue.
  • Forge converts arbitrary Hugging Face repos into verified MTP models with measured before/after speedups.
  • Apache-2.0, requires Apple Silicon M1+ and macOS 14+, install via Homebrew, pip, or DMG.

Running large language models on a Mac has always been bandwidth-bound. On Apple Silicon, every token has to drag the entire quantized weight tensor through unified memory, which caps a 27B Q4 model somewhere around 7 tokens per second on an M4 Pro regardless of how fast the GPU actually is. MTPLX is a new open-source Mac app and CLI that attacks that ceiling by using a feature modern Qwen and Gemma checkpoints already ship with but almost no runtime touches: native multi-token prediction heads.

MTPLX runs local LLMs at roughly twice the speed using native MTP speculative decoding on Apple Silicon, with OpenAI and Anthropic compatible serving. The repo is Apache-2.0, sits at around 1,500 stars, and ships as a DMG, a Homebrew formula, and a PyPI package.

The model drafts for itself

Speculative decoding usually means running a small "draft" model to guess several tokens ahead, then verifying them with the big model in one batched forward pass. The catch is you have to load and maintain a second model, and its distribution has to line up with the target's or you burn accuracy.

MTPLX skips the second model entirely. It runs the model's own built-in MTP heads as a speculative drafter, with exact probability-ratio acceptance plus residual correction, rather than the greedy-argmax shortcut most fast-decode tools use at temperatures above zero. Real coding settings (temperature=0.6, top_p=0.95, top_k=20) actually get the speculative speedup and keep the target model's distribution.