Skip to main content
Back

pyannote.audio 4.0 Ships community-1 With 50% Fewer Speaker Errors

pyannote.audio 4.0 ships community-1, a new open-source diarization model with 50% fewer speaker errors and native Whisper integration

Model, Full Access
  • pyannote.audio 4.0 released with new open-source community-1 model, the biggest update in nearly two years. GitHub
  • 50% fewer speaker confusion errors compared to the 3.1 legacy pipeline, with major improvements in speaker counting and assignment.
  • New exclusive diarization mode makes aligning pyannote timestamps with Whisper or other STT outputs dramatically simpler.
  • 15x faster training via metadata caching and optimized dataloaders, benefiting researchers fine-tuning on custom datasets.
  • Two-tier model structure: free open-source community-1 (CC-BY-4.0) and premium precision-2 API, switchable with one line of code. HuggingFace
  • Hosted pricing starts at €19/month (Developer Plan, 125 hours); open-source self-hosting remains free.

pyannote.audio has been the de facto open-source standard for speaker diarization since its debut at ICASSP 2020. Version 4.0 is now out, and it ships community-1 -- the biggest open-source model upgrade in nearly two years. The headline improvement: dramatically fewer cases where speech gets assigned to the wrong speaker, plus a new mode that makes combining diarization with Whisper or any other speech-to-text model far less painful.

The problem it solves

Speaker diarization answers one question other speech tools skip: not what was said, but who said it. It automatically partitions an audio recording to answer "who spoke when," transforming raw multi-speaker audio into structured segments, each labeled with a unique speaker identity. Without it, you are left manually timestamping every speaker turn in a recording -- a task that scales terribly for long meetings or large call center datasets.

The pipeline works in three stages:

  • Segmentation: A neural model scans the audio in sliding windows and classifies every frame -- detecting speech activity, speaker changes, and overlapping voices simultaneously.
  • Embedding: Each speech segment gets encoded into a compact voice fingerprint (a speaker embedding) that captures the acoustic identity of the speaker.
  • Clustering: Embeddings from across the recording are grouped together, so all segments from the same person get the same label.

The key technical innovation powering the segmentation step is powerset classification. Older diarization systems treated each speaker as an independent yes/no switch, which made overlapping voices a fragile special case. Pyannote's modern segmentation model instead uses powerset classification: it has a dedicated category not just for "speaker 1" and "speaker 2" but for "speakers 1 and 2 talking at once." Concretely, the model reads ten seconds of audio and classifies every frame into one of seven categories: non-speech, speaker 1 only, speaker 2 only, speaker 3 only, speakers 1+2, speakers 1+3, or speakers 2+3.