|
For years, deep learning has relied on a simple assumption: a model is one big optimization problem that learns from data all at once. But that simplicity has a price, catastrophic forgetting.
Train a model on new data, and it often forgets what it already knew. Google Research wants to fix that with a new paradigm called Nested Learning, unveiled with a proof-of-concept architecture named Hope.
The problem: memory that doesn’t last Machine learning models struggle to retain knowledge over time. Even large language models, with billions of parameters, lose context when fine-tuned or retrained. This is the same issue that prevents continual learning, the ability to learn new skills without erasing old ones.
The insight: models are more like brains than we thought Nested Learning reimagines a model not as one giant learner but as a collection of smaller learners nested inside each other.
Each one runs its own optimization loop and updates at its own speed, just like different brain regions processing information at distinct rhythms.
The breakthrough: self-modifying and context-aware learning Google’s Hope model uses Nested Learning to structure memory across multiple timescales. Instead of treating architecture and training as separate, it unifies them into one system of layered optimization problems.
This means the model can decide which parts to update quickly and which to keep stable.
Key components
-
Deep Optimizers: Treats standard algorithms like Adam as memory systems that compress gradient updates.
-
Continuum Memory System (CMS): Extends Transformer memory into layers that update at different frequencies.
-
Self-Modifying Architecture: The Hope model learns to adjust its own learning rules over time.
Results Hope outperforms Titans, Samba, and Transformers on language modeling and reasoning tasks. It achieves lower perplexity, higher accuracy, and handles long-context reasoning better, solving “Needle-in-a-Haystack” tasks with more stability and precision.
How you can use it You can simulate nested optimization today in PyTorch or JAX by assigning different update rates to model components or optimizers. This lets you fine-tune large models while preserving old knowledge, a practical first step toward continual learning.
|