Skip to main content
Back

Meta's MMS-300M Beats Whisper Across 11x More Languages

Meta's 300M-parameter wav2vec 2.0 checkpoint pretrained on 500,000 hours of speech across 1,400+ languages is trending as a fine-tuning base for ultra-low-resource ASR.

  • Meta's mms-300m is a 300M wav2vec 2.0 backbone pretrained on 500K hours across 1,400+ languages.
  • Part of the MMS project covering ASR and TTS for 1,107 languages, language ID for 4,000+.
  • Fine-tuned MMS more than halves Whisper's word error rate on 54 FLEURS languages.
  • Training data built from New Testament audio readings, roughly 32 hours per language.
  • Released under CC-BY-NC 4.0, so no commercial use without a separate Meta agreement.
  • Meant as a base for fine-tuning, not out-of-the-box inference; use mms-1b-all for that.

Meta's facebook/mms-300m checkpoint is climbing the Hugging Face trending charts again. It's the compact pretrained backbone from Meta's Massively Multilingual Speech project: a 300 million parameter wav2vec 2.0 encoder pretrained on roughly 500,000 hours of speech across more than 1,400 languages, designed as a base you fine-tune for automatic speech recognition, translation, or classification in languages that most other models don't touch.

A backbone built for 1,100+ languages

Pretrained with the self-supervised wav2vec 2.0 objective on 16 kHz mono audio, mms-300m sits inside the broader MMS release, where Meta expanded speech technology from roughly 100 languages to over 1,000. The model card is explicit: this checkpoint produces nothing useful out of the box and requires fine-tuning before inference. If you want ASR on day one, the fine-tuned heads like mms-1b-all, mms-1b-l1107, and mms-1b-fl102 are the right starting points. The 300M variant is purely a pretrained base.

Where the training data came from

Collecting labeled audio for thousands of languages was the core obstacle, since the largest existing speech datasets cover at most 100 languages. Meta's solution was to mine religious texts, primarily Bible translations, which exist in hundreds of languages with publicly available audio recordings of people reading them aloud. New Testament readings alone cover over 1,100 languages, averaging 32 hours of data per language. Unlabeled audio from other Christian religious sources pushed pretraining coverage past 4,000 languages. The 300M and 1B wav2vec 2.0 models learn general speech representations from this mixture, then a thin adapter and CTC head gets fine-tuned per language for actual ASR.