LocalAI's Privacy Filter Nemotron Scans 55 PII Categories Without Python
A local-first GGUF port turns OpenAI's Privacy Filter into a 1B PII detector that tags 55 categories in one pass, no Python required.
Takeaways
- GGUF port of OpenMed/privacy-filter-nemotron, a 1B PII token classifier with 55 categories.
- Runs locally via privacy-filter.cpp on CPU, CUDA, or Vulkan with no Python.
- Reports macro B-F1 of 0.9533 and token accuracy of 0.9910 on Nemotron-PII test split.
- Ships F16 (2.82 GB) and Q8_0 (1.64 GB); Q8 flipped one token in a 1,360-token doc.
- Custom architecture will not load in stock llama.cpp, Ollama, or LM Studio.
- English-only, Apache-2.0, aimed at pre-LLM redaction pipelines.
One of the most-downloaded models on Hugging Face this cycle is a token classifier, not a chatbot. The privacy-filter-nemotron-GGUF scans text for personally identifiable information, returns clean entity spans ready for redaction, and runs locally on CPU, CUDA, or Vulkan with no Python required.
55 categories, one forward pass
Built on OpenAI's Privacy Filter architecture, the model labels every token with a BIOES tag across 55 PII categories (221 classes) in a single forward pass, then decodes coherent spans using a constrained Viterbi procedure. The base openai/privacy-filter covers 8 coarse categories with multilingual breadth; this fine-tune goes the other direction, trading language coverage for category depth. The 55 English categories include first and last name, government IDs, financial data, healthcare identifiers, vehicle info, and digital artifacts like ipv4, mac_address, api_key, and http_cookie. LocalAI drives it through a dedicated privacy-filter backend rather than the standard chat/completion path: other models opt into PII detection via a pii.detectors list.
Why stock llama.cpp won't load it
This GGUF uses a custom architecture, openai-privacy-filter, not yet merged into upstream llama.cpp. Stock llama.cpp, llama-cpp-python, Ollama, and LM Studio all fail with unknown model architecture: 'openai-privacy-filter'. The supported path is privacy-filter.cpp, a small standalone GGML engine built on stock upstream ggml with no patches.