Skip to main content
Back

ZipVoice-Dialog Beats Billion-Scale Voice AI at 15x Faster Speed

A small 123M-parameter flow-matching model outperforms billion-scale autoregressive rivals on dialogue synthesis, shipping with a 6.8k-hour open dataset.

  • The k2-fsa research team released ZipVoice-Dialog, a 123M-parameter non-autoregressive flow-matching model for two-speaker dialogue TTS.
  • Two tricks make flow matching work for dialogue: monologue-to-dialogue curriculum learning and learnable speaker-turn embeddings added to text features.
  • Beats MoonCast (2.67B) and Dia (1.61B) on WER, cpWER, speaker similarity, and UTMOS while running roughly 15 times faster.
  • OpenDialog, a 6.8k-hour open spoken dialogue dataset (1759h Chinese, 5074h English), released alongside the model.
  • Stereo variant available, rendering each speaker on a separate channel for full-duplex training or media production.
  • Code, weights, dataset, and benchmark are on Hugging Face under Apache 2.0.

Multi-speaker spoken dialogue has been one of the messier corners of text-to-speech. Most systems that can convincingly juggle two voices, natural turn-taking, and back-channels are large autoregressive models that generate token-by-token, which makes them slow and prone to word skipping or endless repetition. A research team from the k2-fsa project (the open-source speech group behind next-gen Kaldi, with Daniel Povey among the authors) has now published a paper and released code, checkpoints, and a dataset for ZipVoice-Dialog, a non-autoregressive alternative built on flow matching.

The headline claim: a 123M-parameter model beats billion-scale dialogue systems like MoonCast (2.67B) and Nari Labs' Dia (1.61B) on every objective metric they tested, while running roughly 15 times faster.

Why dialogue breaks vanilla flow matching

Flow matching has become a popular backbone for non-autoregressive TTS because it trains with a simple objective and needs few sampling steps. Applied directly to dialogue, though, vanilla flow-matching architectures produce unintelligible speech with unstable turn-taking. The team traces this to two problems: learning speech-to-text alignment across two very different voices is much harder than for a single speaker, and the model has no clean way to know which speaker owns which part of the text.