Skip to main content
Back

Kyutai's MuScriptor Doubles Open-Source Music Transcription Accuracy to 48.2 F1

Kyutai and Mirelo's open transcription model now exports editable sheet music and guitar tabs, on top of streaming per-instrument MIDI.

  • MuScriptor now exports sheet music and guitar tabs as PDF and editable MusicXML, generated from MIDI via MuseScore.
  • Open-weight decoder-only Transformer from Kyutai and Mirelo for multi-instrument music transcription.
  • Three sizes: 103M, 307M, and 1.4B parameters, auto-downloaded from Hugging Face on first use.
  • Trained on 1.45M synthetic MIDI plus 170k real recordings (11k+ hours) with RL post-training.
  • Large variant scores Multi F1 48.2 versus 21.9 for YourMT3+ baseline; try it at muscriptor.kyutai.org.
  • Code is MIT-licensed, weights are CC BY-NC 4.0 (non-commercial); works best on steady-tempo music.

Automatic music transcription has been chronically half-solved. Drop a full band mix into most tools and you get back a smeared piano roll that fuses drums, bass, guitar, and vocals into one unreadable blob. MuScriptor, a joint model from Kyutai and Mirelo, was released to fix that, and it just got a practical upgrade: sheet music and guitar tabs, not only MIDI.

The pipeline exports both PDFs and editable MusicXML, so output drops straight into notation software without hand re-engraving. Under the hood, the team generates scores from MIDI using MuseScore plus custom preprocessing to clean up rhythm and layout. Everything runs from the same open-source repo, and you can try it in the browser at muscriptor.kyutai.org or self-host locally.

Multi-instrument score showing electric guitar, bass, drums, and vocals parts transcribed by MuScriptor

What's actually under the hood

MuScriptor is a decoder-only Transformer for music transcription. It reads a mel-spectrogram of a short audio segment, then autoregressively predicts MIDI-like tokens for pitch, timing, and instrument, turning transcription into a language-modeling task following the MT3 tokenization scheme. Feed it a recording in any genre (pop, classical, metal, jazz) and it transcribes the notes played by every instrument into MIDI, without needing to know in advance what instruments are present.