Skip to main content
Back

Meta's Llama Prompt Guard 2 Jumps from 21% to 97% Attack Detection

Meta shipped Llama Prompt Guard 2 in 86M and 22M sizes with a new loss function that pushes jailbreak recall from 21% to 97%.

  • Meta released Llama Prompt Guard 2 in 86M and 22M parameter versions for prompt injection and jailbreak detection.
  • Recall at 1% false positive rate jumped from 21.2% in v1 to 97.5% in the new 86M model.
  • The 22M variant runs classifications in 19 ms on A100 while beating all non-Meta competitors on AgentDojo.
  • A new energy-based loss function drives the accuracy gains by suppressing false positives on benign prompts.
  • Tokenizer was hardened against whitespace and fragmented-token bypass tricks, and the output is now binary benign or malicious.
  • Weights are open under the Llama 4 license, with a fine-tuning cookbook for domain-specific traffic.

Meta shipped an updated pair of small classifier models built to sit in front of large language models and block prompt injection and jailbreak attempts before they reach the main model. Llama Prompt Guard 2 comes in two sizes, an 86M parameter version and a new 22M variant, and it ships as part of the broader Llama 4 protection tooling.

The headline change is a new training objective that fixes a specific failure mode in the original Prompt Guard, turning what used to be a leaky filter into something closer to a hard gate on known attack patterns.

What actually shipped

Meta announced Prompt Guard 2 alongside Llama Guard 4 as part of its open source AI defender tooling. The 86M model updates the original Llama Prompt Guard classifier for jailbreak and prompt injection detection, and the 22M variant is a smaller, faster sibling that cuts latency and compute costs by up to 75% with minimal performance trade-offs.

Both are open weight, small enough to run on CPU, and structured as a plain text classification pipeline. The 86M model uses mDeBERTa-base as its backbone, and the 22M model uses DeBERTa-xsmall, both open-source MIT-licensed models from Microsoft. Usage follows the standard Hugging Face pattern:

from transformers import pipeline
classifier = pipeline(
    "text-classification",
    model="meta-llama/Llama-Prompt-Guard-2-86M"
)
classifier("Ignore your previous instructions.")
# -> MALICIOUS

Why v1 needed a rewrite

The v1 model had a decent AUC but a brutal recall problem at low false positive rates. In practice, if you tuned it tight enough not to annoy real users, it caught almost nothing. The internal numbers make this concrete.