Skip to main content
Back

Rapid-MLX Beats Ollama by 4.2x, Turning Apple Silicon Into a Free AI Server

A drop-in OpenAI-compatible server built on Apple's MLX framework, claiming 4.2x Ollama throughput and 0.08s cached time-to-first-token.

Repo, Full Access
  • Rapid-MLX is a local AI inference server for Apple Silicon Macs built on MLX.
  • Benchmarks show 4.2x Ollama throughput, up to 261 tok/s aggregate, and 0.08s cached TTFT.
  • Drop-in OpenAI and Anthropic API compatibility works with Cursor, Claude Code, Aider, LangChain out of the box.
  • Ships 17 tool-call parsers, prompt cache with sub-100ms TTFT, reasoning separation, and cloud routing.
  • DFlash speculative decoding uses a block-diffusion drafter to predict and verify tokens in parallel.
  • Apache 2.0 licensed, installable via pip, Homebrew, or uv, requires M1 or newer.

If you own an M-series Mac and want to stop paying for API tokens, Rapid-MLX is worth a look. It's an open-source inference server that turns any Apple Silicon machine into a local endpoint that speaks the OpenAI and Anthropic wire formats, and it's clocking numbers that put the usual local-LLM stacks on notice.

Rapid-MLX is the fastest local AI engine on Apple Silicon, built on Apple's MLX framework rather than the C++-centric stacks (llama.cpp, Ollama) that dominated 2024 to 2025. The performance advantage is structural: MLX is designed around unified memory with native Metal compute kernels, so it avoids the CPU-to-GPU copy overhead that slows down engines originally built for discrete NVIDIA hardware.

The numbers behind the pitch

The project uses Apple's MLX framework, designed for unified memory with native Metal compute kernels, to deliver 4.2x the throughput of Ollama and ranks first on 16 of 18 benchmarked models. On the homepage benchmark, Rapid-MLX reaches up to 261 tokens per second of aggregate throughput with a 0.08-second time to first token (cached) on Apple Silicon, using continuous batching, prompt caching, and speculative decoding.

The cached TTFT number is the interesting one for agent workloads. Prompt caching with KV trimming gives sub-100ms time to first token on transformers, and RNN state snapshots bring the same to hybrid architectures. In practice, follow-up turns in a Cursor or Claude Code session feel nearly instant instead of eating a fresh prefill on every message.