Karpathy's autoresearch Lets AI Run 100 Experiments Overnight Without Human Help
Karpathy's autoresearch lets an AI agent run ~100 LLM training experiments overnight on a single GPU, replacing the human research loop entirely
Takeaways
- What it is: autoresearch is Karpathy's open-source framework where an AI agent autonomously runs LLM training experiments overnight on a single GPU.
- The loop: The agent edits
train.py, trains for a fixed 5-minute budget, measures val_bpb, keeps improvements, reverts failures, and repeats -- ~100 experiments per night. - Real results: Karpathy's own 2-day run produced 700 experiments and 20 stacked optimizations, cutting GPT-2 training time from 2.02 to 1.80 hours with zero human input.
- How to use it: Requires a single NVIDIA GPU and PyTorch; you write
program.mdin plain English, point any coding agent (Claude, Codex) at the repo, and walk away. - Key limits: NVIDIA-only (community forks exist for Mac/Windows/AMD), results are hardware-specific and not cross-platform comparable, and scope is intentionally narrow.
- Industry signal: The repo hit 92k+ stars; Fortune called the pattern "The Karpathy Loop" and funded companies are now building enterprise versions of the same idea.
autoresearch is Andrej Karpathy's open-source framework that hands the entire ML experimentation loop to an AI agent. The idea is simple: give an AI agent a small but real LLM training setup and let it experiment autonomously overnight. It modifies the code, trains for 5 minutes, checks if the result improved, keeps or discards, and repeats. You wake up in the morning to a log of experiments and, hopefully, a better model. The repo launched in March 2026 and picked up 21,000+ GitHub stars and 8.6 million views on Karpathy's announcement within days. It now sits at over 92,000 stars.
The experimenter, not the experiment
The key framing here is not that autoresearch automates experiments. Karpathy isn't automating the experiments. He's automating the experimenter. The traditional ML research workflow is: human reads papers, forms a hypothesis, modifies training code, runs an experiment, analyzes results, updates their mental model, and repeats. autoresearch collapses that entire loop into an agent running unattended.
Karpathy frames this as a natural progression in how engineers work with AI. In February 2026, he coined the term "agentic engineering": "You are not writing the code directly 99% of the time. You are orchestrating agents who do and acting as oversight." AutoResearch takes the next step. The human doesn't even orchestrate. They describe what good research looks like in a markdown file and walk away.
Three files, one loop
The codebase it operates on is a stripped-down single-GPU LLM training implementation called nanochat -- the key file, train.py, is ~630 lines of Python. The whole repo has exactly three files that matter:
prepare.py-- the evaluation harness that scores the model. The agent cannot touch it -- if it could, it would just make the test easier instead of making the model better.train.py-- contains the model implementation and training loop. This is the file the agent edits when proposing new experiments. Everything is fair game: architecture, hyperparameters, optimizer, batch size.