Westlake University's Free RL Textbook Hits 14,000 Stars by Fixing Math Nobody Taught
Shiyu Zhao's free, math-first RL textbook hits 16K GitHub stars and 2.1M video views, making it the clearest on-ramp to reinforcement learning theory
Takeaways
- Shiyu Zhao's Mathematical Foundations of Reinforcement Learning has crossed 16K GitHub stars and 2.1M+ lecture video views.
- The full book PDF and all lecture slides are free to download from the GitHub repo; a Springer hardcover (~$66-86) is also available.
- Covers 10 chapters from basic MDP concepts through actor-critic methods, all grounded in rigorous but accessible math.
- Every example uses a single grid world environment, keeping focus on the math rather than problem-specific complexity.
- A 54-video English YouTube playlist accompanies the book, covering every chapter in detail.
- Community has contributed Python, MATLAB, R, and C++ implementations of the book's algorithms, plus study notes in multiple languages.
Reinforcement learning has a textbook problem. Sutton & Barto's classic is thorough but sprawling at 500+ pages. Most online courses skip the math entirely and jump straight to code. Mathematical Foundations of Reinforcement Learning by Shiyu Zhao threads the needle: a tight, 275-page treatment that builds every algorithm from first principles without losing the reader in abstraction.
The book has quietly become one of the most-starred educational resources in the AI space. The lecture videos accompanying the book have received over 2,100,000 views across the internet. The textbook has sold over 15,000 copies, with its GitHub repository receiving over 14,000 stars. That kind of traction for a math-heavy RL textbook is unusual, and it signals something real about the gap it fills.
Who built it and why
Shiyu Zhao is an Associate Professor and Director of the Intelligent Unmanned Systems Laboratory in the School of Engineering at Westlake University, Hangzhou, China. He has been teaching a graduate-level course on reinforcement learning since 2019, and prepared this book as the lecture notes for his students. The result is a textbook that reads like a well-organized course rather than a reference manual.
Before joining Westlake University in 2019, he was a Lecturer in the Department of Automatic Control and Systems Engineering at the University of Sheffield, UK. His background in control systems shows in the book's structure: it is precise, sequential, and obsessed with explaining