Skip to main content
Back

Exeter's Forgive or Forget Framework Strips Hate Speech From Audio Search Without Retraining

A new post-hoc framework uses causal inference to scrub toxic audio from retrieval results without retraining the underlying model or mangling the original query.

Paper, Full Access
  • New framework: Forgive or Forget uses causal inference to remove toxic audio from text-to-audio retrieval results post-hoc.
  • Two strategies: "Forget" averages logits across LLM-generated toxic query variants to wash out bias; "Forgive" transcribes retrieved audio and re-ranks toxic clips down.
  • Model-agnostic: Works as a drop-in post-processing layer on any retrieval pipeline -- no retraining or weight access needed.
  • Results: Consistent toxicity reduction with over 90% audio quality similarity preserved vs. ground truth on AudioCaps and Clotho benchmarks.
  • Six harm categories: Hate, Profanities, Pornographic Language, Slurs, Violence, and Bullying are all targeted by the mediator generation step.
  • No code released: No public checkpoint or codebase is available yet, limiting immediate reproducibility.

Text-to-audio retrieval systems -- the kind that let you type "man arguing with woman" and get back matching audio clips -- have a hate speech problem. And it turns out the naive fixes make things worse. Researchers from the University of Exeter and collaborators have published a paper introducing a two-pronged framework called Forgive or Forget that tackles toxic audio retrieval using causal inference, without touching the underlying retrieval model at all.

The problem with "just filter it"

Handling sensitive topics such as hate and sarcasm in text-to-audio retrieval systems presents unique challenges due to their contextual dependencies. The phrase "man arguing with woman" is not inherently hateful -- but a retrieval system trained on internet-scraped audio can easily surface clips laced with slurs, threats, or profanity in response to it. Existing strategies such as rephrasing or summarization risk altering intent or omitting details. If you rephrase the query to be "safer," you change what the user actually asked for. If you summarize it, you lose specificity. Neither approach is satisfying.

The deeper issue is a causal one. In a text-to-audio retrieval pipeline, there are hidden (unobserved) variables -- things like the cultural context of a phrase, the speaker's tone, or the implicit demographic framing -- that simultaneously influence the query and the retrieved audio. These are called confounders, and they make it impossible to simply condition on the query text to remove bias. Standard backdoor adjustment, the usual causal fix, doesn't work here because these confounders can't be directly observed or measured.

Causal surgery on the retrieval pipeline

The authors introduce a post hoc causal debiasing framework that applies front-door adjustment via a sentiment-controlled mediator to ensure that retrieved audio is not biased towards harmful speech while maintaining semantic relevance.