AI Safety Filters Are Silencing Transgender and Marginalized Voices at Alarming Rates
A new audit of 4 pretraining filters and 3 guardrails finds they systematically erase transgender, women's, and Central American content — while humans would keep over 88% of what these systems flag for removal.
Takeaways
- Researchers audited 4 pretraining filters and 3 guardrails on Common Crawl text and found they systematically over-flag marginalized identities.
- Transgender mentions are flagged at 1.5–1.8x the rate of cisgender mentions across nearly every system tested.
- Central American content is flagged at up to 99.3% by some systems — nearly 70 percentage points above baseline — the largest single deviation in the audit.
- Human annotators would keep 88.5% of filter-flagged and 91.3% of guardrail-flagged content, showing a massive gap between automated and human judgment.
- All three guardrails correlate most strongly with a simple keyword blocklist (Shutterstock), suggesting LLM-based guardrails are largely replicating crude lexical matching.
- Code and data are available via the paper's repository; future work will extend to religion, politics, and non-English corpora.
Every large language model you use today has been shaped by two invisible gatekeepers: pretraining filters that scrub text before the model ever sees it, and inference-time guardrails that block outputs at serving time. A new paper from researchers at the University of Turin, IT University of Copenhagen, and the University of Hamburg takes a hard look at both , and finds they are quietly erasing the voices of the very communities they are supposed to protect.
The problem hiding in plain sight
Modern language models rely on pretraining filters to remove undesirable content from training corpora and inference-time guardrails to suppress undesirable outputs during deployment , and how these filtering and moderation decisions produce forms of epistemic erasure has gone largely unexamined. Epistemic erasure here means the systematic removal of knowledge, perspectives, and identities from the information space a model can draw on. The concern is not just that harmful content slips through, but that the process of removing it disproportionately silences specific groups.
Prior work showed that evaluating and removing "toxicity" had a disparate impact on topic coverage and dialect coverage, and that detoxification introduces gender bias where it had not appeared in prior measurements. But no one had run a systematic, cross-system audit comparing filters and guardrails head-to-head on identity-specific content , until now.
What they audited and how
The researchers audited four pretraining filters and three inference-time guardrails on Common Crawl sentences containing gender and regional-origin mentions, together with a manually annotated subset of 500 sentences. The four filters span the main approaches used in practice:
- Hatebase , a slur lexicon focused on racial and misogynist terms
- Shutterstock , the sexual-vocabulary blocklist used in C4 and T5 pretraining
- DOLMA , a FastText classifier trained on the Jigsaw dataset, used in the DOLMA 3-trillion-token corpus
- HateBERT , BERT retrained on posts from Reddit communities banned for abusive content
Three guardrail models of comparable size (8B parameters) were selected from organizations on three different continents, so that any divergence between them cannot be attributed to scale alone and can plausibly be related to the harm taxonomies their developers encode. These were Meta's