Skip to main content
Back

Meta and Google DeepMind Prove LLMs Memorize Exactly 3.6 Bits per Parameter

Researchers from Meta, Google DeepMind, Cornell, and NVIDIA put a hard number on LLM memorization: ~3.6 bits per parameter, with a new framework that cleanly separates memorization from generalization.

Paper, Full Access
  • Researchers from Meta, Google DeepMind, Cornell, and NVIDIA published an ICML paper establishing that GPT-style models memorize approximately 3.6 bits per parameter.
  • The paper introduces a new information-theoretic framework that formally separates unintended memorization (sample-specific storage) from generalization (learning reusable patterns).
  • Memorization is measured via compression: a model is said to have memorized data if it can help encode that data in fewer bits than a baseline model.
  • Models fill their memorization budget first, then switch to generalization — directly explaining the grokking phenomenon and double descent in training curves.
  • Scaling laws derived from the work predict that most modern LLMs trained on large datasets have near-zero membership inference risk on average data points, but rare/unique strings remain disproportionately memorized.
  • The 3.6 bits/parameter figure enables practical privacy auditing, dataset sizing decisions, and copyright risk assessment before training begins.

How much does a language model actually remember? It sounds like a simple question, but it has resisted clean answers for years. Extraction attacks, membership inference, differential privacy bounds , all useful tools, but none of them directly answer the core question: how many bits of specific training data are locked inside a model's weights? A new ICML paper from researchers at Meta, Google DeepMind, Cornell, and NVIDIA finally puts a number on it.

The measurement problem nobody solved

The field has long struggled with a fundamental confusion: when a model outputs something that was in its training set, is that memorization or generalization? Language models can be coerced to output almost any string, so the fact that a model outputs something is not necessarily a sign of memorization. A model that correctly answers a math problem it was trained on might simply be doing math , not recalling a specific example.

Prior studies of language model memorization struggled to disentangle memorization from generalization. The paper formally separates memorization into two components: unintended memorization (the information a model contains about a specific dataset) and generalization (the information a model contains about the true data-generation process). This distinction is what makes the measurement possible.

Compression as the measuring stick

The key insight is borrowed from information theory: if a model helps you compress a dataset, it must contain information about that dataset. The authors use Kolmogorov complexity , the theoretical minimum description length of a piece of data , as the formal backbone, then approximate it in practice using the model's own log-likelihoods. Concretely, unintended memorization of a sample is measured as how many fewer bits you need to encode it when the trained model is available as a reference, compared to using only a baseline model trained on the full distribution.

To isolate pure memorization capacity, the team first eliminated generalization entirely by training on datasets of uniformly random bitstrings , sequences with no structure to generalize from. Models were trained using the GPT-2 architecture initialized from scratch, ranging from 1 to 8 layers with hidden dimensions from 32 to 512, covering 100K to 20M parameters, trained for 10⁶ steps with a batch size of 2048 using the Adam optimizer on a single A100 GPU.

3.6 bits per parameter, consistently

By gradually increasing model size and training each variant to saturation, across hundreds of experiments on models ranging from 500K to 1.5 billion parameters, they observed consistent results: 3.6 bits memorized per parameter, which they report as a fundamental measure of LLM memory capacity.

To put that number in perspective: storing 3.6 bits allows for approximately 12.13 distinct values, as calculated by 2^3.6 , about the amount of information needed to choose one of 12 options, similar to selecting a month of the year or the outcome of a roll of a 12-sided die. Per parameter. Summed across billions of parameters, that's a meaningful but bounded budget.

One practical note: a prior estimate by Allen-Zhu and Li established that transformer models store approximately 2 bits of knowledge per parameter, derived from synthetic factual tuples. The new figure of 3.6 bits is higher because it measures raw memorization capacity rather than structured knowledge storage , and because the methodology is more rigorous about stripping out generalization.

Grokking, explained

One of the paper's most elegant findings is that the 3.6 bits/parameter limit directly predicts a well-known but poorly understood training phenomenon called grokking , the sudden jump in generalization that happens late in training, long after a model has already fit the training set.

The measurements estimate that models in the GPT family have an approximate capacity of 3.6 bits per parameter. Models memorize until their capacity fills, at which point "grokking" begins, and unintended memorization decreases as models begin to generalize. In other words: the model first fills its memory budget with specific examples, and only once that budget is saturated does it start compressing knowledge into reusable patterns. This transition aligns with the well-known double descent curve in machine learning.

What this means for real text

The experiments on synthetic random data establish the capacity ceiling. The paper then repeats the analysis on real text using the FineWeb dataset, where generalization is possible and beneficial. The results show that models trained on text memorize up to a point, then trade sample-level memorization for generalizable patterns as the dataset grows beyond the model's capacity.

The paper also produces scaling laws for membership inference , the task of guessing whether a specific example was in the training set. The key finding: the team trained hundreds of transformer language models ranging from 500K to 1.5B parameters and produced a series of scaling laws relating model capacity and data size to membership inference. The scaling law predicts that most modern LLMs, trained on hundreds of billions to trillions of tokens, are in a regime where loss-based membership inference on the average data point is essentially impossible , the dataset is simply too large relative to the model's memorization budget.

What it's good for, and where it falls short

This framework is most useful for:

  • Privacy auditing: Given a model size and dataset size, you can now estimate the expected membership inference risk before training, rather than running expensive post-hoc attacks.
  • Dataset sizing: If you want to ensure a model generalizes rather than memorizes, the 3.6 bits/parameter figure gives you a concrete data-to-capacity ratio to target.
  • Scaling law reasoning: The capacity number connects directly to double descent and grokking, giving practitioners a principled explanation for training dynamics they may have observed empirically.
  • Copyright and data governance: Rare or unique strings , like non-English text with unusual tokens , are disproportionately memorized. Rare or unique strings still stick. This has direct implications for copyright risk assessment.

The limitations are worth noting. The 3.6 bits figure is measured on GPT-style transformers; other architectures may differ. The experiments used models up to 1.5B parameters, and the scaling laws are extrapolated beyond that range. The framework also requires access to model likelihoods, so it cannot be applied to black-box APIs without modification. And as the authors note, gradient descent doesn't guarantee optimal memorization , the measured capacity is a lower bound on what the architecture could theoretically store.

The practical punchline

The paper's most actionable result is also its most reassuring one for most practitioners: as datasets grow and model capacity saturates, memorization drops , models are forced to generalize. A Llama 3 8B model trained on 15 trillion tokens has a memorization budget of roughly 3.6 × 8 × 10⁹ ≈ 29 billion bits, or about 3.4 GB of raw information. Against 15 trillion tokens of training data, the average data point receives a vanishingly small slice of that budget. The model simply cannot remember most of what it saw.

That said, the tail risk is real. Documents with rare tokens , technical jargon, non-English text, unique identifiers , are memorized at much higher rates than average. Measuring memorization solely by average extraction rates is insufficient. It should focus on identifying specific pieces of copyrighted or private text that are most likely to be memorized. The 3.6 bits/parameter number gives you the budget; this paper's framework now lets you figure out how that budget is allocated.