Skip to main content
Back

Alec Radford's Open-Source talkie Freezes an AI's Knowledge in 1930

A 13B open-weight LLM trained exclusively on pre-1931 text opens a new front in AI research: contamination-free generalization and historical forecasting.

  • Talkie is a 13B open-weight LLM trained on 260B tokens of exclusively pre-1931 English text, the largest vintage language model released to date. [GitHub]
  • Built by Nick Levine, David Duvenaud, and Alec Radford (of GPT/GPT-2/Whisper fame), released under Apache 2.0 with weights on Hugging Face.
  • Because it predates all modern benchmarks, it is contamination-free by construction, enabling clean generalization experiments impossible with web-trained models.
  • OCR noise from physical source documents is a major bottleneck: standard OCR yields only 30% of the performance of human-transcribed text; the team is building a custom vintage OCR pipeline.
  • The instruction-tuned variant (talkie-1930-13b-it) was post-trained entirely from pre-1931 reference works (etiquette manuals, encyclopedias) to avoid injecting modern knowledge.
  • A GPT-3-scale vintage model is planned for summer 2026, with a corpus estimated to grow beyond one trillion historical tokens. Blog post

What does a language model think the world looks like in 2026, if it has never heard of World War II, the internet, or even Franklin D. Roosevelt's presidency? Asked about 2026, talkie offers a vision straight out of a Victorian futurist novel: Europe with a billion inhabitants, iron railroads crisscrossing the continent, and steamships connecting London and New York in ten days. That is not a bug. It is the entire point.

A team led by Nick Levine, David Duvenaud, and Alec Radford has built talkie, described as the most historically disciplined large language model ever released to the public: a 13-billion parameter open-weight model trained exclusively on pre-1931 English text. Co-author Alec Radford is one of the most influential AI researchers of recent years, having been lead author of the seminal 2018 GPT paper at OpenAI, where he also worked on Whisper and DALL-E.

A model frozen in 1930

A vintage language model is deliberately trained only on historical data so that its worldview is frozen at a particular point in the past. For talkie, that cutoff is December 31, 1930, chosen precisely because that is the date when works enter the public domain in the United States, making pre-1931 text legally usable for training.

The model was trained on 260 billion tokens of historical pre-1931 English text, including books, newspapers, periodicals, scientific journals, patents, and case law. The team also ships two variants:

  • talkie-1930-13b-base: the raw pre-trained model, great for completion and research experiments.
  • talkie-1930-13b-it: instruction-tuned using a novel dataset built from pre-1931 reference works including etiquette manuals, letter-writing manuals, encyclopedias, and poetry collections, and further trained with reinforcement learning using online DPO to improve instruction-following.
  • talkie-web-13b-base: a modern twin with the same architecture and training FLOPs but trained on FineWeb, allowing controlled comparisons between modern and vintage models.