TWS Lets One Brain Decoder Read Any Animal's Neurons Without Retraining
A new tokenizer segments neural recordings by behavioral events instead of neuron identity, letting one model transfer from mice to macaques with just a linear probe.
Takeaways
- TWS tokenizes neural recordings by population state between behavioral events, not by neuron identity.
- Fixes the core failure of per-neuron foundation models: new sessions have new neurons with no embedding.
- Pretrained on mouse IBL Neuropixels, transfers frozen to macaque Utah arrays with a linear probe.
- Beats a per-neuron foundation model that decodes at chance on held-out IBL sessions.
- Reach-direction MCC of 0.23 on macaques versus 0.01 from event-time baseline alone.
- Core claim: for cross-session transfer, the token matters more than the model on top.
Each neural recording typically samples a different set of neurons, which creates an out-of-vocabulary problem for models that assign learned embeddings to individual cells. The TWS preprint, Neural Data Needs Semantic Tokenization, proposes a session-independent representation built from population activity between task events. The approach aims to let one encoder process recordings from new animals, laboratories, and electrode systems without learning embeddings for every new neuron.
| Input | Trialized population activity and task-event timestamps |
|---|---|
| Segmentation | Three behavioral states defined by event boundaries and two shared, trainable time offsets |
| Output | Six 256-dimensional tokens per trial |
| Identity features | No neuron or session embeddings |
| Transfer test | Mouse Neuropixels pretraining followed by evaluation on held-out mice and macaque Utah arrays |
| Status | Preprint; no code repository linked in the submission |
Neuron IDs expire between sessions
Spike-level models such as POYO represent each spike with its timestamp and the identity of the neural unit that produced it. POYO demonstrated that this design can scale across 158 sessions, seven nonhuman primates, and more than 27,000 recorded units.
A learned unit embedding still binds the representation to neurons seen during training. New recordings introduce new unit IDs, so the model must initialize or fit additional embeddings before using them. That adaptation can work, but it requires session-specific data and prevents direct ingestion of an unseen recording.
States provide a shared coordinate system
Although a recording may contain hundreds of neurons, correlated firing patterns can often be summarized with fewer latent dimensions. As behavior unfolds, population activity traces a path through this reduced space, commonly called a neural manifold. The preprint builds on evidence that aspects of this population-level geometry can remain comparable across sessions and animals after alignment.