Open-Source LuxTTS Clones Voices 150x Faster Than Realtime on One GPU
LuxTTS is a free, open-source voice cloning model that runs at 150x realtime speed and fits inside just 1GB of VRAM.
Takeaways
- 150x realtime speed: LuxTTS clones voices faster than any comparable open-source TTS model, and runs above realtime even on CPU.
- 1GB VRAM footprint: Fits on any consumer GPU or runs entirely on CPU, no cloud required.
- 48kHz audio output: Double the 24kHz standard of most TTS models, with a custom vocoder.
- 3-second reference audio: Zero-shot voice cloning from a short WAV or MP3 clip, no fine-tuning needed.
- Built on ZipVoice + flow distillation: 4-step distilled flow-matching model; Apache 2.0 licensed on GitHub with a free HuggingFace Space to try instantly.
- Active community ecosystem: Gradio apps, ComfyUI nodes, and Pinokio launchers already available; float16 support on the roadmap for ~2x additional speedup.
LuxTTS is a new open-source text-to-speech model that clones voices at speeds most production systems can only dream of: 150x faster than realtime on a single GPU, and still faster than realtime on CPU alone. It fits entirely within 1GB of VRAM, outputs 48kHz audio, and needs as little as 3 seconds of reference audio to clone a voice. With nearly 4,000 GitHub stars since launch, the community has clearly taken notice.
The speed problem TTS has always had
Most high-quality TTS and voice cloning systems are slow by design. Large diffusion-based or autoregressive models need dozens of inference steps to generate audio, and they typically require 4-8GB of VRAM just to load. That makes them unsuitable for real-time applications, edge deployment, or anything running on modest hardware. ZipVoice, the architecture LuxTTS builds on, was designed specifically to address this: a high-quality flow-matching-based zero-shot TTS model with a compact size and fast inference speed.
Flow matching is a generative modeling technique that learns to transport samples from random noise to realistic speech features along a smooth path. The key insight is that you can distill this process, training a smaller model to mimic the behavior of a larger one in far fewer steps, without a meaningful quality drop.
What LuxTTS actually is
LuxTTS is a lightweight, high-quality text-to-speech voice cloning model based on the ZipVoice architecture, distilled to 4 inference steps with an improved sampling technique. The original ZipVoice paper from the k2-fsa team showed that ZipVoice (123M parameters) achieves speech quality comparable to or better than existing SOTA zero-shot TTS models with significantly more parameters, such as MaskGCT at 1048M and F5-TTS at 336M.
LuxTTS takes that already-efficient base and pushes it further. From the repo's FAQ: it uses the same architecture as ZipVoice but distilled to 4 steps with an improved sampling technique, and swaps in a custom 48kHz vocoder instead of the default 24kHz version. A vocoder is the component that converts the model's internal acoustic features into actual audio waveforms, and running it at 48kHz doubles the output resolution compared to most TTS models.