Google DeepMind's SURF Separates Audio Without Ever Hearing Clean Sounds
Google DeepMind's SURF learns to separate mixed audio without ever seeing clean isolated sources, setting a new unsupervised state-of-the-art at ICML 2026
Takeaways
- SURF is a new Google DeepMind model that separates mixed audio into individual sources without ever training on clean, isolated audio.
- It combines flow matching (a generative modeling technique) with a teacher-student remixing loop, training entirely on raw mixtures.
- On speech separation (Libri2Mix), SURF achieves 16.54 dB SI-SDR vs. 13.22 dB for the best prior unsupervised baseline, nearly matching supervised performance.
- The method is general: it was validated on both audio and image separation benchmarks, suggesting broad applicability.
- SURF sets a new unsupervised state-of-the-art but struggles slightly on mixtures with 3+ simultaneous sources compared to some baselines.
- The paper was published at ICML 2026 with a live audio demo at the Google booth in Seoul.
Sound separation has always had a dirty secret: the best models need clean, isolated audio tracks to train on. If you want a model to separate speech from background noise, you typically need thousands of hours of pristine, individually-recorded voices. That data is expensive to collect, and models trained on it often break when they hit real-world audio that doesn't match their training set. Google DeepMind's new SURF (Separation via Unsupervised Remixing Flow) paper, presented at ICML 2026 in Seoul, attacks this problem head-on: it trains entirely on raw, mixed audio with no clean sources required.
The core problem SURF solves
Single-channel source separation is genuinely hard. Given a single audio recording of two people talking over music, you want to recover each individual stream. In supervised settings where vast amounts of clean source data are available, this problem has been addressed successfully by generative diffusion and flow-based prior models. However, access to such clean source samples is often limited, and even when available, supervised models are vulnerable to domain shifts.
This matters enormously in practice. Reliance on synthetic training data is problematic because good performance depends upon the degree of match between the training data and real-world audio, especially in terms of the acoustic conditions and distribution of sources. The acoustic properties can be challenging to accurately simulate, and the distribution of sound types may be hard to replicate. In domains like bio-acoustics, hyperspectral imaging, or gravitational wave detection, obtaining clean in-domain source data is practically impossible.
How SURF actually works
Flow matching is a generative modeling technique that learns to transform one probability distribution into another by training a neural network to predict a smooth "velocity field" , essentially, a direction of travel through the space of possible signals. SURF applies this idea to source separation, but with a twist: it never sees clean sources during training.
SURF is an unsupervised flow matching approach for source separation that learns directly from observed mixtures. The method relies on a novel combination of state-of-the-art supervised flow matching and regression-based self-supervised techniques.
The training loop works in three phases:
- Teacher estimates: A teacher model (initially trained with MixIT, an existing unsupervised baseline) separates a batch of real-world mixtures into estimated sources.
- Remixing: Starting from a teacher model, SURF utilizes a "remixing" step to bootstrap the learning of a student flow model from the teacher's estimates. The teacher's source estimates are randomly shuffled across the batch and re-summed to create new synthetic mixtures with known pseudo-targets.
- Student update + EMA: The student flow model trains on these synthetic mixtures, then the teacher is updated via exponential moving average (EMA) , a slow-tracking mechanism that stabilizes training.
The key mathematical bridge is connecting the flow model's velocity field to the regression-style estimates that the remixing algorithms expect. This is non-trivial because flow models estimate a velocity term at an intermediate noisy state, while regression models directly estimate the clean source. SURF resolves this with an identity that lets you recover a clean-source estimate from any point along the flow path.
The paper provides insights into the objectives optimized by this approach and draws a novel connection to the Wake-Sleep algorithm , a classical method for training latent-variable generative models from the 1990s. The "sleep" phase trains the student on synthetic mixtures generated by the teacher; the "wake" phase updates the teacher toward the student via EMA. This framing gives SURF a principled probabilistic interpretation.
What the numbers look like
Empirical evaluations on image and audio benchmarks demonstrate that SURF establishes a new state-of-the-art, significantly outperforming existing unsupervised methods. Here's a summary of key results:
- Speech separation (Libri2Mix): SURF (ReMixIT) achieves SI-SDR of 16.54 dB vs. 13.22 dB for the best prior unsupervised method (ReMixIT regression), closing most of the gap to the supervised flow matching baseline at 17.89 dB.
- Universal sound separation (AudioSet): On single-source FUSS evaluation, SURF (ReMixIT) scores 32.67 vs. 19.83 for ReMixIT regression , a jump of nearly 13 dB.
- Image separation (CIFAR-10): SURF achieves PSNR of 19.73 and FID of 14.83, compared to 17.40 PSNR and 28.44 FID for regression-based ReMixIT. The supervised flow baseline scores 20.38 PSNR.
- Perceptual quality: SURF's generative nature means it produces outputs that look and sound more natural, not just outputs that are numerically close to the ground truth.
Where it struggles
SURF is not uniformly better across all conditions. On the FUSS benchmark for mixtures with 3 or 4 simultaneous sources, SURF does slightly worse than the MixIT teacher on the 3-source case. The paper hypothesizes this is because MixIT is trained on mixtures-of-mixtures with a higher average source count, giving it an edge in crowded scenes. Complex mixtures with many overlapping sources remain a hard open problem.
There's also a practical consideration: the generative approach introduces the possibility of hallucination. The model can produce plausible-sounding outputs that weren't actually in the original mixture, which complicates provenance tracking and makes traditional signal-fidelity metrics like SI-SDR less meaningful as the sole evaluation criterion.
Why this matters beyond audio
SURF was validated on both audio and image separation benchmarks, which signals that the underlying framework is general. The same teacher-student remixing loop with flow matching could apply to any domain where you have access to mixtures but not clean components.
The broader implication is a shift in how the field thinks about what data is required to train separation models. Supervised approaches to single-channel speech separation rely on synthetic mixtures, so that the individual sources can be used as targets. Good performance depends upon how well the synthetic mixture data match real mixtures. However, matching synthetic data to the acoustic properties and distribution of sounds in a target domain can be challenging. Instead, unsupervised methods require only single-channel acoustic mixtures, without ground-truth source signals. SURF now shows you can do this with a generative model that also captures the distribution of natural signals , something prior unsupervised methods couldn't do.
For practitioners, this opens up domains that were previously off-limits: wildlife audio recordings, medical acoustic signals, archival recordings, or any setting where you have plenty of real mixed audio but no way to obtain isolated tracks. The live demo at the Google booth at ICML 2026 lets attendees hear the separation outputs directly , a fitting way to evaluate a model whose outputs are meant to be listened to, not just measured.
The SURF paper is authored by Henry Li and Robin Scheibler (equal contribution), Efthymios Tzinis, Matt Shannon, Arnaud Doucet, and John R. Hershey from Google and Google DeepMind. Code and audio samples are available via the paper's demo page.