Kyutai's MuScriptor Doubles Open-Source Music Transcription Accuracy to 48.2 F1
Kyutai and Mirelo's open transcription model now exports editable sheet music and guitar tabs, on top of streaming per-instrument MIDI.
Takeaways
- MuScriptor now exports sheet music and guitar tabs as PDF and editable MusicXML, generated from MIDI via MuseScore.
- Open-weight decoder-only Transformer from Kyutai and Mirelo for multi-instrument music transcription.
- Three sizes: 103M, 307M, and 1.4B parameters, auto-downloaded from Hugging Face on first use.
- Trained on 1.45M synthetic MIDI plus 170k real recordings (11k+ hours) with RL post-training.
- Large variant scores Multi F1 48.2 versus 21.9 for YourMT3+ baseline; try it at muscriptor.kyutai.org.
- Code is MIT-licensed, weights are CC BY-NC 4.0 (non-commercial); works best on steady-tempo music.
Automatic music transcription has been chronically half-solved. Drop a full band mix into most tools and you get back a smeared piano roll that fuses drums, bass, guitar, and vocals into one unreadable blob. MuScriptor, a joint model from Kyutai and Mirelo, was released to fix that, and it just got a practical upgrade: sheet music and guitar tabs, not only MIDI.
The pipeline exports both PDFs and editable MusicXML, so output drops straight into notation software without hand re-engraving. Under the hood, the team generates scores from MIDI using MuseScore plus custom preprocessing to clean up rhythm and layout. Everything runs from the same open-source repo, and you can try it in the browser at muscriptor.kyutai.org or self-host locally.
What's actually under the hood
MuScriptor is a decoder-only Transformer for music transcription. It reads a mel-spectrogram of a short audio segment, then autoregressively predicts MIDI-like tokens for pitch, timing, and instrument, turning transcription into a language-modeling task following the MT3 tokenization scheme. Feed it a recording in any genre (pop, classical, metal, jazz) and it transcribes the notes played by every instrument into MIDI, without needing to know in advance what instruments are present.