Skip to main content
Back

Argmax's WhisperKit Beats OpenAI and Deepgram With 11M Downloads on Apple Silicon

Argmax's on-device Whisper for Apple Silicon crossed 11M downloads, hit v1.0, and beats cloud ASR APIs on both latency and accuracy.

  • WhisperKit CoreML crossed 11M downloads on Hugging Face, first Apple Silicon-only model to hit that scale
  • ICML 2025 paper shows 0.46s latency and 2.2% WER, beating OpenAI, Deepgram, and Fireworks
  • New OD-MBP compression shrinks Whisper Large v3 Turbo from 1.6 GB to 0.6 GB with under 1% WER loss
  • Stateful Core ML models cut decoder energy per pass by 75% (1.5W to 0.3W) on M3 Neural Engine
  • Repo renamed to argmax-oss-swift, bundling WhisperKit, SpeakerKit, and TTSKit with Swift 6 support
  • Install with brew install whisperkit-cli or add the Swift package in Xcode, MIT license

A pure-Swift speech recognition package running Whisper entirely on Apple Silicon has quietly become one of the most-downloaded models on Hugging Face, and its authors just published benchmarks showing it beats OpenAI, Deepgram, and Fireworks on real-time transcription. WhisperKit CoreML from Argmax now sits at over 11 million all-time downloads, and the underlying framework has graduated from a single kit into a full open-source SDK that bundles speech-to-text, speaker diarization, and text-to-speech in one MIT-licensed Swift package.

Argmax reports that WhisperKit is the first Apple Silicon-only model to cross 10 million monthly downloads on Hugging Face, with usage growing roughly tenfold in about 100 days. That trajectory, combined with a fresh ICML paper and a v1.0 SDK release, makes this worth a closer look for anyone shipping voice features on Mac or iOS.

Beating cloud APIs on their own turf

The headline claim from the WhisperKit paper: an on-device system that matches or exceeds server-side APIs on both speed and word error rate. The team benchmarked against a frontier model (OpenAI gpt-4o-transcribe), a proprietary ASR service (Deepgram nova-3), and a hosted open-source model (Fireworks large-v3-turbo).

WhisperKit ties for lowest latency at 0.46s while posting the best accuracy at 2.2% WER on TIMIT. Fireworks matched raw latency but issued an order of magnitude more corrections, which the authors argue degrades user experience even when average speed looks fast. OpenAI's gpt-4o-transcribe doesn't support hypothesis text streams at all, so its confirmed-text latency lags far behind.

How a 1B-parameter Transformer fits on your phone

Getting Whisper Large v3 Turbo, a 1-billion-parameter encoder-decoder, to run in real time on a phone required rethinking three things: the streaming architecture, the compression scheme, and how the model talks to the Apple Neural Engine.