Skip to main content
Back

Harvard Researchers' RBI-Eval Catches AI Agents Leaking Sensitive User History

RBI-Eval is a new benchmark that measures when memory-augmented AI agents inappropriately surface sensitive user history in unrelated conversations

Paper, Full Access
  • New benchmark: RBI-Eval measures when memory-augmented LLM agents inappropriately surface sensitive user history in unrelated conversations.
  • The gap it fills: Existing memory benchmarks (LoCoMo, LongMemEval) only measure recall accuracy, not whether surfacing a memory was appropriate.
  • Key finding: Separation scores collapse by 51--83% for Claude-Sonnet-4.6, DeepSeek-V4-Flash, and Qwen3.5-9B when sensitive memory is present.
  • GPT-5.4-mini is notably more restrained, dropping only 8.9--26.6%, suggesting generation-time policy plays a major role.
  • Retrieval helps but doesn't fix it: Mem0, A-Mem, and MemU reduce exposure but cannot prevent integration once sensitive memory reaches the generator.
  • Implication: Safe personalization requires memory-aware decisions at both retrieval and generation time, not just better retrieval filtering.

Memory-augmented AI agents are getting better at remembering you -- but a new benchmark asks a harder question: should they? Researchers from Hefei University of Technology and Harvard Medical School have introduced RBI-Eval (Relevant-But-Inappropriate Evaluation), a controlled measurement framework designed to probe exactly when LLM agents cross the line from helpful personalization into inappropriate disclosure of sensitive user history.

The gap nobody was measuring

The memory benchmark landscape has been almost entirely focused on one question: can the agent remember? Long-term memory enables language model agents to support personalized interactions, but existing memory evaluations emphasize retrieval accuracy and downstream task utility, while overlooking whether retrieved sensitive memory content is warranted in the current turn. Benchmarks like LoCoMo, LongMemEval, and BEAM measure recall quality and factual accuracy across sessions. None of them ask whether surfacing a piece of memory was the right call in the first place.

Existing benchmarks primarily focus on whether agents can recall and apply user information, while overlooking whether such personalization is used appropriately. In fact, agents may overuse personal information, producing responses that feel forced, intrusive, or socially inappropriate to users. RBI-Eval is specifically designed to fill this gap -- not by measuring what an agent remembers, but by measuring when it should stay quiet.

Diagram showing appropriate vs inappropriate memory use: an AI assistant handling a user's eating disorder history in a benign conversation

How RBI-Eval works

RBI-Eval is a controlled probe set that measures memory-use behavior by comparing responses with and without access to sensitive history. Each item contains a benign current prompt paired with prior history containing sensitive disclosures. Think of it as a controlled experiment: the same innocuous question ("what should I have for lunch?") is sent to the agent twice -- once with no memory, and once with a memory store that includes sensitive personal history like an eating disorder recovery. The key signal is whether the memory-equipped agent brings up the sensitive context unprompted.