KAIST's Membrane Cuts AI Jailbreaks to 0% Without Blocking Legitimate Requests
Membrane's contrastive memory pairs every blocked attack with a permitted benign twin, slashing false refusals while topping safety benchmarks across six jailbreak types
Takeaways
- Membrane is a new LLM guardrail that stores safety knowledge as contrastive pairs: each cell encodes both what to block and what similar-looking benign requests to allow. [Paper]
- It achieves the highest F1 on all six jailbreak attacks across HarmBench (model safety) and AgentHarm (agent safety) benchmarks, with no retraining required.
- False refusal rate on AgentHarm is 7-14%, versus 28-85% for prior memory-based and prompt-level guards, solving the over-refusal problem.
- Memory cells trained on one attack type transfer to unseen attacks at 87-88% F1 retention, because cells are indexed by attack mechanism, not topic.
- A lite mode (Retrieval Critic disabled) runs at ~2.4 seconds per query while still outperforming all baselines, making real-time deployment feasible.
- The system is robust to memory poisoning at 20% injection rates, with ASR drift below 0.4 points, due to low retrieval rates for poisoned cells and critic filtering.
Every safety team building on top of an LLM faces the same uncomfortable tradeoff: tighten the guardrail and watch it start refusing legitimate requests; loosen it and watch jailbreaks slip through. A new paper from KAIST AI and KakaoBank's Financial Tech Lab argues this tradeoff is not fundamental. It is a consequence of how safety knowledge has been stored, and a different storage format can break the deadlock.
The system is called Membrane, and its core insight is deceptively simple. Most existing guardrails store only one side of the safety boundary: what to block. Membrane stores both sides simultaneously, pairing every blocked attack pattern with an explicit description of the superficially similar benign requests that must still be allowed through. The authors call this a Contrastive Safety Memory (CSM).
The problem no one was solving
LLM jailbreaks have evolved from simple prompt tricks into a sophisticated, continuously shifting attack surface. Modern attacks span feedback-driven prompt-refinement search, persuasion-based rewriting, and self-adapting attack policies. The standard response has been to fine-tune dedicated safety classifiers like LlamaGuard or WildGuard. But traditional defenses encode safety behavior in fixed model parameters, making their decision boundaries difficult to revise after deployment.
Memory-based guardrails tried to fix the rigidity problem by storing safety knowledge in an external pool that grows without retraining. But they introduced a new failure mode. Existing memory designs store safety knowledge in a one-sided form, capturing attacks or refusal rules without paired conditions for permitting benign queries. This creates two coupled failure modes: coverage remains tied to the surface forms already stored, while expanding the memory can push the guard toward rejecting benign queries that resemble known attacks.

What a memory cell actually looks like
The fundamental unit in Membrane is a CSM cell, a structured record with three components:
- Attack Profile: classifies the attack by its underlying mechanism (persuasion, role injection, payload splitting, obfuscation, etc.) rather than by topic. A request for chemical synthesis routes and one for financial fraud can both be tagged as
persuasionif they use the same rhetorical framing. - Decision Criteria: a contrastive pair. The
d_unsafeside lists conditions sufficient to block a query. Thed_safeside explicitly describes superficially similar benign requests that must be allowed through, the anti-over-refusal channel. - Tool-Action Context: activated only in agent settings, this field anchors the boundary to specific tool calls and adds a