Skip to main content
Back

MusicGen Now Runs Faster Than Realtime on a MacBook With MLX

An independent developer ported Meta's MusicGen to Apple's MLX framework, making text-to-music generation run faster than realtime on M-series Macs.

  • Independent developer released musicgen-mlx, an open source port of Meta's MusicGen to Apple's MLX framework.
  • The 300M small model generates 8 seconds of audio in about 6.3 seconds on M4 Max, faster than realtime.
  • The 3.3B stereo-large model runs at roughly 0.3x realtime, still practical for non-interactive use.
  • Full transformer LM and EnCodec decoder run on MLX; T5 text encoder still uses PyTorch on CPU.
  • Melody and style-conditioned variants are not yet supported, but on the roadmap.
  • MIT licensed code; Meta's model weights remain CC-BY-NC 4.0.

Running Meta's MusicGen on a Mac has always been a chore. The official checkpoints ship as PyTorch, which on Apple Silicon falls back to CPU inference for large chunks of the pipeline and turns an eight second clip into a coffee break. An independent developer named Andrade Olivier just released musicgen-mlx, an open source port of the entire inference stack to MLX, Apple's array framework built around unified memory on M-series chips.

The headline number: the small 300M parameter model generates 8 seconds of audio in about 6 seconds on an M4 Max, faster than realtime, and the large stereo 3.3B parameter model also runs, producing high-quality stereo music. MusicGen goes from a batch-job curiosity on Mac laptops to something you can iterate with interactively.

What actually got ported

The project covers the full inference path for the text-to-music variants. The transformer language model, including 48 layers of streaming multi-head attention with RoPE positional embeddings, KV caching, and cross-attention on the text conditioning, now runs natively on MLX. The EnCodec decoder that turns discrete audio tokens back into a waveform also runs on MLX, along with automatic conversion of Meta's PyTorch checkpoints on first load.

Two pieces still lean on PyTorch. The T5 text encoder that turns your prompt into embeddings runs on CPU and hands its output over to MLX at the boundary, which adds startup overhead but avoided porting the entire T5 stack. The stereo variants also wrap HuggingFace's EnCodec with torch-to-MLX conversion at the edges.