What rebuilding AlphaGo teaches us about self-play, RL, and future of LLMs - Eric Jang
Eric Jang discusses his experience rebuilding AlphaGo from scratch, detailing the intricacies of Monte Carlo Tree Search (MCTS) and neural network architectures. He explores AlphaGo's unique self-play reinforcement learning approach, contrasting it with LLM training methods, and delves into the philosophical implications of AI solving NP-hard problems. The episode concludes with insights into the current capabilities and limitations of using large language models for automating AI research.
- AlphaGo's 2014-2016 breakthroughs showed deep learning could solve problems "long understood to be intractable for search," like the game of Go, which had baffled traditional AI methods for decades. Read →
- Eric Jang’s experience rebuilding AlphaGo showed how small neural networks can “amortize” complex, seemingly intractable search problems, compressing vast simulation into minimal compute. Read →
- AlphaGo's Monte Carlo Tree Search (MCTS) generates a "strictly better action" for every single move, offering immediate, local feedback, a stark contrast to the sparse rewards common in LLM reinforcement learning. Read →
- AlphaGo made the combinatorially complex game of Go tractable by using neural networks to guide a Monte Carlo Tree Search (MCTS) algorithm, a core breakthrough in AI decision-making. Read →
- AlphaGo relies on two distinct neural networks: a value network to predict win/loss probability from a given board state, and a policy network to suggest optimal next moves. Read →