Skip to main content
Back

Fondazione Bruno Kessler's Audio Trick Pulls Apart Voices 2.6 dB Better

A signal processing approach uses tunable beta-divergence factorization to pull out overlapping voices from reverberant multichannel recordings, beating two blind separation baselines.

Paper, Full Access
  • New method estimates Gaussian source separation parameters via beta-divergence nonnegative tensor factorization with source priors
  • Sparsity of factorization, not the specific beta value used in training, drives separation quality
  • Beta between 0.3 and 0.6 works best at the estimation step across most tested conditions
  • Beats Nesta and Cho baselines on SISEC2013 live-recorded mixtures with 2.6 dB average SDR gain
  • Method struggles with male voices in reverberant rooms above 250 ms reverberation time
  • Published in IEEE/ACM Transactions on Audio Speech and Language Processing

Separating overlapping voices captured by a couple of microphones in a real, echoey room is one of the oldest headaches in audio processing. A paper from the SHINE unit at Fondazione Bruno Kessler tackles it with a purely statistical recipe, showing that how sparse you force your factorization to be matters more than which specific divergence you optimize.

The authors work in the classic underdetermined convolutive mixing regime: N speakers observed by M microphones where M is less than N, with reverberation smearing each voice across time. In Gaussian model-based multichannel separation, the likelihood of the observed signals is parametrized by source spectral variances and their associated spatial covariance matrices. Those parameters get estimated by maximizing the likelihood through an Expectation-Maximization loop, then plugged into a multichannel Wiener filter that pulls the sources apart. Plain EM tends to accumulate errors and leak crosstalk between the recovered voices.

Prior knowledge as a lever

The proposal is to inject prior information about what the sources actually sound like. The method estimates model parameters by applying nonnegative factorization constrained by spectral basis matrices, which encode that prior. Those matrices can be extracted directly from the mixture or drawn from a redundant library trained in advance on clean speech.