Skip to main content
Back

WhisperLiveKit Hits 10K Stars Bringing Real-Time Transcription On-Device

An open-source stack combining Whisper, streaming diarization, and 200-language translation into a self-hosted, ultra-low-latency transcription server.

  • WhisperLiveKit passes 10k stars as a self-hosted, ultra-low-latency streaming Whisper server.
  • Combines SimulStreaming, Streaming Sortformer diarization, and NLLB-based translation across 200 languages.
  • Exposes OpenAI-compatible REST, Deepgram-compatible WebSocket, and native WebSocket endpoints for drop-in use.
  • Backends span Faster-Whisper, MLX, Voxtral Mini, and Qwen3-ASR via vLLM, with Silero VAD gating.
  • Ships CLI tools for transcription, subtitle generation, model management, and reproducible RTF-vs-WER benchmarks.
  • Includes a Chrome extension to transcribe audio from any browser tab locally.

Running Whisper on live audio is deceptively hard. The model was trained on complete utterances, so slicing a microphone stream into chunks and feeding them through produces cut-off words, lost context, and noticeably poor output. WhisperLiveKit (WLK) is an Apache-2.0 project that has quietly crossed 10,000 GitHub stars by packaging current research on simultaneous speech-to-text into a single self-hosted server, complete with speaker diarization and real-time translation across 200 languages.

The project bills itself as an ultra-low-latency, self-hosted pipeline. Because Whisper expects complete utterances rather than real-time chunks, naive segmentation loses context, cuts words mid-syllable, and produces poor transcription. WLK instead borrows from simultaneous speech research to buffer intelligently and process incrementally.

The research stack under the hood

WLK is essentially an orchestration layer over a stack of recent papers and models. The transcription path defaults to Simul-Whisper / SimulStreaming, a 2025 approach that achieves ultra-low latency using the AlignAtt policy, which decides when to emit a token based on how attention aligns with the audio frames seen so far. As a fallback, it can use WhisperStreaming, the 2023 low-latency method built on the LocalAgreement policy, where a word is only committed once successive decoding passes agree on it.