Stanford AIMI's deidentifier-base Beats Human Labelers at Stripping Medical Records
Stanford AIMI's PubMedBERT-based token classifier scrubs protected health information from radiology reports at near-perfect F1, and has crossed 1.9M downloads.
Takeaways
- Stanford AIMI's PubMedBERT-based deidentifier has crossed 1.9M downloads on Hugging Face under MIT license.
- Hits 97.9 to 99.6 F1 on radiology reports and i2b2 2006/2014 PHI benchmarks.
- Beats human labelers on i2b2 2014, per the JAMIA 2022 paper.
- Trained on 6,193 multi-institutional documents with synthetic PHI augmentation.
- Uses hide-in-plain-sight surrogates instead of visible redaction placeholders.
- Full pipeline available at MIDRC/Stanford_Penn_Deidentifier; a v2 checkpoint is now published.
A quietly dominant model in the medical NLP corner of Hugging Face has been racking up downloads at a pace most general-purpose LLMs would envy. Stanford AIMI's deidentifier-base is a small BERT-family token classifier built to strip protected health information (PHI) from radiology reports and other clinical text, and it has crossed 1.9 million downloads on the Hub while sitting under an MIT license.
The weights are not new, but they have become the default choice for anyone trying to ship a HIPAA-friendly pipeline over free-text medical documents. It is worth understanding what the model does, where it fails, and why the field keeps reaching for it.
Under the hood
This is a token classification head fine-tuned on top of PubMedBERT (uncased), the biomedical BERT variant pretrained on PubMed abstracts. The full system pairs the transformer with a rule-based layer for post-processing. It tags spans like patient names, dates, phone numbers, doctor names, MRNs, and institution identifiers so they can be replaced downstream.
The replacement strategy is where things get interesting. Rather than masking PHI with placeholders like [NAME] or <DATE>, the associated pipeline uses a hide-in-plain-sight approach: detected PHI is swapped for realistic-looking surrogates. A downstream reader (or model) sees a coherent report, and any adversary trying to spot residual PHI cannot simply diff for redaction markers.