LAION's CLAP2 Beats Audio AI Rivals With 365 Million Training Clips
LAION releases a 365M-clip audio dataset with 707M captions and a family of open CLAP2 models that set new state of the art on MAEB-30.
Takeaways
- LAION releases LAION-Audio-XL: 365M audio clips, 707M captions, 1.39M hours, curated from 1.48B raw clips.
- Five CLAP2 models (27M to 340M params) trained from scratch using the NaFlex variable-length audio encoder.
- CLAP2-L hits 53.6% on MAEB-30, the best score of any CLAP model trained from scratch.
- Scaling law: error drops 3.9% per compute doubling; optimal data scales 1.4x faster than model size.
- Environmental sound performance saturates around 1.5e10 GFLOPs, while speech and music keep improving.
- Dataset, captions, checkpoints, and training code all released; models work with open_clip.
LAION and researchers from TU Munich, Forschungszentrum Jülich, and the University of Tübingen have released LAION-Audio-XL, a web-scale corpus containing 365 million audio clips, 707 million captions, and 1.39 million hours of audio. They also published five CLAP2 models trained from scratch on the corpus.
CLAP2-L, the largest checkpoint at 340 million parameters, scores 53.6% on the 30-task Massive Audio Embedding Benchmark, or MAEB-30. The release includes dataset configurations, captions, checkpoints, training code, and compute-scaling results that developers can use to plan audio-text training runs.
Audio-text training gets its missing dataset
Contrastive Language-Audio Pretraining, commonly shortened to CLAP, learns a shared embedding space for audio and text. An audio encoder maps a clip to a vector, while a text encoder does the same for a caption or label. Training pulls matching vectors closer and separates unrelated pairs.
The resulting embeddings support audio search, prompt-based classification, automatic tagging, and downstream classifiers. CLAP follows the same broad approach as CLIP for images, but public audio-text corpora have remained much smaller than image-text datasets. Many available audio clips also lack captions or carry video descriptions that say little about the soundtrack.
Earlier public CLAP datasets contained a few million pairs. LAION-Audio-XL expands that training pool by roughly two orders of magnitude, giving the accompanying paper enough model sizes and training budgets to estimate how audio-text performance changes with compute.
From 1.48 billion samples to 365 million clips
The researchers gathered 1.48 billion audio samples from YouTube, Common Crawl, other web collections, and 316 academic datasets. Their curation pipeline removed unusable or redundant material, balanced the remaining categories, generated missing captions, and limited clips to 30 seconds.