MLX-Audio Brings 646-Language Voice Cloning to Apple Silicon Without the Cloud
MLX-Audio brings a full on-device audio stack to Apple Silicon, covering TTS, STT, and speech enhancement with 30+ models and no cloud dependency.
Takeaways
- Full audio stack on Apple Silicon: MLX-Audio covers TTS, STT, speech enhancement, diarization, and source separation — all running locally on M-series Macs.
- 30+ models supported: Includes Kokoro, Qwen3-TTS, Voxtral TTS, Whisper, Parakeet, VibeVoice-ASR, OmniVoice (646 languages), and Kyutai's Multilingual Pocket TTS with day-0 support.
- OmniVoice is the standout addition: Zero-shot voice cloning across 646+ languages with nonverbal tags like [laughter], running entirely on-device.
- MLX advantage: Apple's unified memory architecture lets the GPU use all system RAM; MLX runs 25–30x faster than PyTorch MPS for inference on the same hardware.
- OpenAI-compatible API: Drop-in replacement for OpenAI's audio endpoints, plus a web UI with 3D waveform visualization and a Swift SDK for iOS/macOS apps.
- Free and MIT-licensed: Install via
pip install mlx-audio; requires Python 3.10+ and an Apple Silicon Mac (M1 or later).
MLX-Audio has become the most comprehensive audio library for Apple Silicon. Built by Prince Canuma on top of Apple's MLX framework, it covers text-to-speech, speech-to-text, voice activity detection, speaker diarization, speech enhancement, and source separation, all running locally on M-series Macs with no cloud account required.
Why Apple Silicon changes the equation
Apple's M1 through M5 chips use a unified memory architecture: one pool accessible to the CPU, GPU, and Neural Engine without transfers across a bus. Most ML frameworks weren't built with this in mind. MLX was designed for unified memory from scratch, which is why it runs 25–30x faster than PyTorch MPS for LLM inference on the same hardware. Audio workloads are often memory-bandwidth-bound, so this gap matters. A 32GB Mac effectively has 32GB of VRAM, because the GPU can use all system RAM directly.
What's in the box
The model catalog spans three major categories:
- Text-to-Speech: Kokoro (82M parameters, 54 voice presets, 8 languages), Qwen3-TTS (voice cloning and emotion control), Voxtral TTS (Mistral, 4B, 20 voices, 9 languages), Chatterbox (16 languages), CSM (voice cloning), Dia, Spark, MeloTTS, KugelAudio (7B, 24 European languages), LongCat-AudioDiT (diffusion-based), Higgs Audio v2 (3B, real-time cloning), MOSS-TTS, and OmniVoice.
- Speech-to-Text: Whisper and Distil-Whisper, Qwen3-ASR, Parakeet v3 (NVIDIA, 25 EU languages), Voxtral Realtime (4B streaming), VibeVoice-ASR (Microsoft, 9B, diarization and timestamps), Canary, Moonshine, MMS (Meta, 1000+ languages), Granite Speech, and Qwen2-Audio.