Skip to main content
Back

SharedRequest Hides Your Sensitive Prompts in a Crowd, Cutting API Costs 5x

SharedRequest hides your prompts by mixing them into noisy batches, cutting query costs by 5x while outperforming differential privacy baselines by 20%

Paper, Full Access
  • New framework: SharedRequest protects LLM prompt privacy by mixing real queries with noisy decoys in a shared batch, not by scrambling individual prompts.
  • 20% utility gain: Outperforms differential privacy baselines by over 20% on utility benchmarks at equivalent privacy levels.
  • 5x cost reduction: Shared-prompt batching amortizes API costs across users with similar instructions, cutting query cost by up to 5x vs. non-batched inference.
  • Model-agnostic: Requires no access to model weights or architecture -- works with any LLM API including GPT-4o, Claude, and open-source models.
  • Threat model: Designed to protect against a curious API provider observing raw prompts; not a defense against compromised user devices.
  • No code released yet: The paper is available on arXiv but no public implementation has been published alongside it.

Every time you type a sensitive question into ChatGPT or any cloud-hosted LLM, your raw prompt travels to a server you don't control. That server can log it, analyze it, or expose it. For enterprises handling medical records, legal documents, or proprietary business data, this is not a theoretical concern -- it's a compliance nightmare. A new paper introduces SharedRequest, a framework that tackles prompt privacy in a fundamentally different way: instead of scrambling individual prompts, it hides them inside a crowd.

The wall every privacy method hits

The standard playbook for private LLM inference leans on differential privacy (DP) -- a mathematical framework that adds carefully calibrated random noise to data, making it statistically impossible to identify any single person's contribution. In practice, this means replacing words in your prompt with semantically similar but randomly chosen alternatives before sending it to the model.

The problem is that noise is the enemy of coherence. As privacy protection increases, utility metrics like BERTScore and semantic similarity decline -- higher privacy means significant utility loss, while lower privacy improves utility but compromises protection. This is the so-called "no free lunch" theorem of private LLM inference, and it has plagued every prior approach.

The other main alternative, Secure Multi-Party Computation (SMPC), encrypts the computation itself. Although SMPC can ensure the confidentiality of client data, it requires the LLM to collaborate tightly with the client, which limits its application to LLMs accessible only through commercial APIs. And the latency cost is brutal: under a secure three-party protocol, generating a 64-word sentence can take up to 20 minutes.