AlphaGo
AlphaGo was the first Go system to cross the professional-human threshold and then defeat one of the strongest human players in the world, showing that Deep Reinforcement Learning plus Monte Carlo Tree Search could master a domain long treated as a benchmark for intuitive, decades-away AI. Its core contribution was not a single neural network, but an engineered loop between learned move priors, learned position evaluation, and search. The AlphaGo line—AlphaGo, AlphaGo Master, AlphaGo Zero, and AlphaZero—became a reference template for Search and Learning systems, while also exposing how much that template depends on closed rules, cheap simulation, and verifiable outcomes.
Coverage note: verified through May 18, 2026.
Why AlphaGo mattered
Go had been a stubborn benchmark for AI because it combines a simple rule system with an enormous search space and weakly decomposable strategic structure. Unlike chess, where hand-engineered evaluation functions and alpha-beta search had become extremely strong by the late twentieth century, Go resisted brute-force tree search: a 19×19 board gives players hundreds of plausible moves per position, and good play often depends on long-range influence, shape, sente, ko threats, territory, and life-and-death judgment that are hard to encode manually.
The 2016 AlphaGo paper framed Go as “the most challenging of classic games for artificial intelligence” because the state space is vast and position evaluation is difficult. AlphaGo’s novelty was to combine two learned components—policy networks for move selection and value networks for outcome prediction—with Monte Carlo tree search, thereby replacing much of the hand-coded judgment that earlier Go engines lacked. Silver et al. reported that AlphaGo beat other Go programs 99.8% of the time and defeated Fan Hui, the European Go champion, 5–0 in full-board games without handicap. storage.googleapis.com
The later Lee Sedol match made that technical result historically visible. In March 2016, AlphaGo defeated Lee Sedol 4–1 in Seoul; DeepMind described Lee as an 18-time world-title winner and said more than 200 million people watched the match. Google DeepMind The match was not merely a product demonstration. It was evidence that deep learning could supply the perceptual and evaluative machinery that search-based AI had lacked in domains where handcrafted heuristics were brittle.
Minimal system summary
| Component | Role in AlphaGo | Training source | Why it mattered |
|---|---|---|---|
| Supervised policy network | Proposed plausible expert-like moves | Human expert games from KGS | Narrowed the search tree to human-plausible candidates |
| Fast rollout policy | Played cheap simulations to terminal states | Human move prediction features | Supplied fast Monte Carlo estimates during search |
| Reinforcement-learning policy network | Improved move policy by self-play | Self-play against earlier policy versions | Produced stronger play than human imitation alone |
| Value network | Estimated probability of winning from a position | Self-play games generated by the RL policy | Replaced expensive rollouts with learned evaluation |
| MCTS | Combined priors, values, and rollouts into a move decision | Online search at inference time | Turned imperfect neural predictions into stronger decisions |
This design is best understood as Policy-Value Search: neural networks supplied compressed strategic intuition, and tree search turned that intuition into deliberate choice. The system was not end-to-end in the later AlphaGo Zero sense; it was an asymmetric ensemble trained from human games, self-play, supervised learning, reinforcement learning, and engineered search.
Historical position: before AlphaGo, AlphaGo, and after
AlphaGo sits between two eras. Before it, game-playing AI success was associated with search plus handcrafted evaluation; after it, the dominant research story shifted toward learned evaluation, self-play, and search guided by neural networks.
The transition can be summarized as follows:
| Era | Representative system | Main recipe | Limitation or lesson |
|---|---|---|---|
| Pre-AlphaGo computer Go | Monte Carlo Go programs such as Pachi, Crazy Stone, Zen | Rollouts, MCTS, domain heuristics | Strong amateur/professional-adjacent play, but not clearly superhuman |
| AlphaGo Fan / AlphaGo Lee | Silver et al. 2016 AlphaGo | Human imitation + self-play RL + value network + MCTS | Breakthrough, but still used human games, handcrafted features, and rollouts |
| AlphaGo Master | Improved AlphaGo lineage | Stronger neural search system, still preceding the public Zero simplification | Demonstrated a leap beyond top human professionals |
| AlphaGo Zero | Silver et al. 2017 AlphaGo Zero | Self-play from random initialization, single policy-value network, no human data, no rollouts | Cleaner demonstration that tabula-rasa RL plus search could exceed human-derived systems |
| AlphaZero | Silver et al. 2017/2018 AlphaZero | Generalized AlphaGo Zero-style method across Go, chess, and shogi | Showed transfer across board games, but still within perfect-information games with exact rules |
The AlphaGo Zero paper explicitly characterized AlphaGo as the first program to achieve superhuman performance in Go and summarized the original system as two networks—a policy network and a value network—trained first from human expert data and then refined through reinforcement learning and self-play. UCL Discovery AlphaGo Zero then removed the most human-dependent pieces: no human game data, no supervised imitation, no separate rollout policy, and no handcrafted domain features beyond the basic board representation and legal game rules. UCL Discovery
The 2016 AlphaGo architecture
Policy networks: learning what moves are plausible
The first major part of AlphaGo was the policy network, a deep convolutional neural network that mapped a Go position to a probability distribution over legal moves. In the supervised phase, AlphaGo was trained to predict expert human moves from a database of about 30 million KGS positions. The 2016 paper reports that its 13-layer supervised policy network achieved 57.0% move-prediction accuracy using the full feature set, while a faster rollout policy achieved only 24.2% accuracy but could evaluate moves far more cheaply. storage.googleapis.com
This distinction between a strong policy network and a fast rollout policy is central. The strong policy network was accurate enough to guide search toward plausible expert moves, but too expensive to use for every step of every rollout. The rollout policy was less accurate but fast enough to simulate games to the end. AlphaGo therefore did not rely on a single monolithic model; it separated quality from speed and used each model where its tradeoff was useful.
The supervised policy was then copied into a reinforcement-learning policy network. That policy played games against earlier versions of itself, received a terminal win/loss reward, and was optimized by policy gradient. Silver et al. report that this RL policy won more than 80% of games against the supervised policy, showing that imitation of human moves was not the ceiling: self-play could push beyond the human-derived distribution. storage.googleapis.com
Value network: learning to evaluate positions
The second major part was the value network, which estimated the probability that the current player would win from a given position. This was the critical substitute for human positional judgment. Instead of relying on thousands of stochastic rollouts from a position, AlphaGo could ask a trained neural network for a direct evaluation.
Training the value network was not as simple as labeling every position in human games with the final game outcome. The 2016 paper notes that positions from the same game are strongly correlated, which caused overfitting when using complete games. DeepMind addressed this by generating a large dataset of distinct self-play positions, each sampled from a separate game played by the reinforcement-learning policy, and then training the value network to regress from positions to outcomes. storage.googleapis.com
This design choice is one of AlphaGo’s deepest methodological contributions. The value network learned from positions generated by the system’s own stronger policy, not merely from human games. That made it a bootstrapped evaluator for the agent’s current style of play. In effect, AlphaGo used self-play to create the distribution of states on which its evaluator had to be competent.
MCTS: turning neural intuition into deliberate play
The third major part was Monte Carlo tree search. AlphaGo’s MCTS maintained action statistics at tree edges: visit counts, action values, and prior probabilities supplied by the policy network. During search, it traversed the tree by balancing high-value actions against actions with strong policy priors and low visit counts; at the leaf, it evaluated the position using a mixture of the value network and rollout results; then it backed up those evaluations through the search tree. storage.googleapis.com
This is where AlphaGo’s architecture becomes more than “deep learning for Go.” The policy network alone could suggest expert-like moves, and the value network alone could evaluate positions, but MCTS let the system revise first impressions. Search provided tactical correction, local consistency checks, and a way to discover whether a plausible-looking move led to a bad continuation.
After search completed, AlphaGo chose the move with the highest visit count at the root. storage.googleapis.com That detail matters because the system’s final decision was not simply the move with the largest raw neural-network probability or the highest immediate value estimate. It was the move that survived an explicit deliberative process.
Asymmetric training and the non-obvious role of human data
AlphaGo’s training was asymmetric. The supervised policy, reinforcement-learning policy, rollout policy, and value network were trained with different objectives, data distributions, and computational constraints. This asymmetry was not incidental; it produced one of the paper’s more subtle findings.
The stronger reinforcement-learning policy was not necessarily the best policy prior for MCTS. Silver et al. found that the supervised policy performed better inside AlphaGo’s search than the stronger RL policy, apparently because human expert data provided a more diverse beam of plausible candidate moves. Conversely, the value network trained from the RL policy performed better than one trained from the supervised policy. storage.googleapis.com
That result complicates the simple story that “self-play beats humans.” In AlphaGo 2016, human games still supplied a powerful prior over the move space. The system surpassed humans not by discarding human data immediately, but by combining human-derived breadth with self-play-derived strength and search-time verification.
Engineering and compute
AlphaGo was also a large engineering system. The Nature paper reports a single-machine version with 40 search threads, 48 CPUs, and 8 GPUs, and a distributed version with 40 search threads, 1,202 CPUs, and 176 GPUs. Neural network evaluations were much more expensive than traditional search heuristics, so the system used asynchronous multithreaded CPU search while evaluating neural networks on GPUs. storage.googleapis.com
This matters for interpreting AlphaGo’s lesson. The breakthrough was algorithmic, but it was also computational. AlphaGo was not a small, elegant theorem about reinforcement learning; it was a demonstration that modern neural networks, large-scale data generation, and search could be engineered into a system that crossed a historic performance threshold.
Empirical results that defined the era
Fan Hui and the professional threshold
Before the Lee Sedol match, AlphaGo defeated Fan Hui, the European Go champion, 5–0 in October 2015. Silver et al. emphasized that this was the first time a computer Go program defeated a professional human player in a full-sized game without handicap, a result that many observers had expected to be at least a decade away. storage.googleapis.com
The paper also reported internal evaluation against other Go programs. The single-machine AlphaGo won 494 of 495 games against a set of contemporary Go engines, a 99.8% win rate. The distributed version was stronger still, winning 77% of games against the single-machine version and 100% against the other programs in the reported tournament. storage.googleapis.com
These results established that AlphaGo was not merely tuned to a human exhibition setting. It had crossed the previous computer-Go frontier by a large margin.
Lee Sedol: the public turning point
The Lee Sedol match was the cultural and scientific inflection point. AlphaGo won 4–1, but the match is remembered not only for the score. Game two’s Move 37 became famous because it was evaluated as highly unlikely by human expectations yet proved strategically powerful; game four’s Move 78 by Lee Sedol, often called the “God’s Touch,” exposed a weakness and produced Lee’s single win. DeepMind’s official AlphaGo history highlights both moves as emblematic moments in the match. Google DeepMind
The methodological lesson of the Lee match was that superhuman play could look alien rather than merely more precise. AlphaGo’s strongest moves were not always legible as better versions of standard human judgment. This became a recurring theme in later discussions of AI Creativity, Alien Move Selection, and Human-AI Collaboration: a system trained by self-play and search may discover policies that humans can learn from but would not naturally invent.
Master and the post-Lee leap
After the Lee Sedol match, DeepMind continued improving AlphaGo. In early 2017, an online system using the names “Master” and “Magister” played a series of games against top professional players. The AlphaGo Zero paper states that AlphaGo Master defeated the strongest human professional players by 60–0 in online games in January 2017. UCL Discovery
DeepMind later organized the Future of Go Summit in Wuzhen, China, including matches involving Ke Jie, then widely described as the world number one player. DeepMind described the Ke Jie match as the headline three-game one-on-one event of the summit and framed the event as AlphaGo’s final competitive appearance. Google DeepMind
The Master period matters because it showed that AlphaGo’s 2016 victory was not a narrow achievement against one opponent. The system had entered a regime where the best human players were no longer the relevant upper bound.
AlphaGo Zero: removing human imitation
AlphaGo Zero was the cleaner scientific result. It began from random play, used no human game data, and trained only by self-play reinforcement learning. It also collapsed AlphaGo’s separate policy and value networks into a single residual neural network with two heads: one outputting move probabilities and one outputting a scalar value estimate. UCL Discovery
The training loop made MCTS part of learning rather than merely inference. For each self-play position, MCTS produced an improved search policy; the final game outcome supplied the value target. The network was trained to match the search policy and predict the outcome, thereby amortizing search into the neural network. UCL Discovery
AlphaGo Zero surpassed the Lee version of AlphaGo after 36 hours of training and, after 72 hours, defeated AlphaGo Lee 100–0 under match conditions. After a longer run, it defeated AlphaGo Master 89–11 in a 100-game match. UCL Discovery
This was the point at which the “human knowledge” story changed. AlphaGo 2016 showed that human imitation plus self-play plus search could defeat humans. AlphaGo Zero showed that, in Go, human imitation was not necessary; it could even constrain the system relative to pure self-play.
Methodological contribution
AlphaGo’s central methodological contribution was the demonstration that three ingredients could be composed into a system stronger than any one of them:
-
A learned policy that makes search tractable by proposing plausible actions.
-
A learned value function that evaluates nonterminal states without exhaustive rollout.
-
A search procedure that improves decisions at inference time and supplies better training targets.
This composition is now a canonical pattern in Reinforcement Learning with Search. It is not just “deep RL,” because plain deep reinforcement learning would struggle with Go’s sparse terminal rewards and long horizons. It is not just MCTS, because unguided MCTS wastes enormous effort in a branching space as large as Go’s. It is not just supervised learning, because predicting human moves does not by itself produce superhuman play.
AlphaGo’s practical insight was that search and learning can compensate for each other’s weaknesses. Learning compresses past computation into a policy and value function; search spends fresh computation on the current position; self-play generates an adaptive curriculum of hard examples. AlphaGo Zero made this loop especially explicit by treating MCTS as a policy-improvement operator and the neural network as the policy/value approximation that absorbs that improvement. UCL Discovery
This is why AlphaGo became more important than a Go engine. It provided a working example of a general research pattern: use learning to guide search, use search to improve learning, and repeat.
AlphaGo versus AlphaGo Zero
The difference between AlphaGo and AlphaGo Zero is often summarized as “human data versus no human data,” but the deeper differences are architectural and methodological.
| Dimension | AlphaGo 2016 | AlphaGo Zero |
|---|---|---|
| Human game data | Yes: supervised policy trained from expert games | No human games |
| Initial policy | Human move prediction | Random initialization |
| Network structure | Separate policy and value networks; separate rollout policy | Single residual policy-value network |
| Search | MCTS with policy prior, value network, and rollouts | MCTS guided by policy-value network, no rollouts |
| Domain features | Multiple feature planes, including Go-specific inputs | Primarily board stones and rules |
| Training loop | Human imitation → self-play policy RL → value learning → search | Self-play MCTS → train network to match search and outcome → repeat |
| Scientific message | Deep networks plus search can defeat professionals | Tabula-rasa self-play plus search can exceed human-derived systems |
The AlphaGo Zero paper explicitly lists the simplifications: training solely from self-play, no human supervision, no human data, no handcrafted features beyond the rules, one network instead of separate policy and value networks, and no Monte Carlo rollouts. UCL Discovery
Zero also sharpened the Bitter Lesson interpretation. AlphaGo 2016 still used human examples and Go-specific engineering; Zero made the result look more like a pure scaling-and-search story. Yet even Zero was not free of structure: it still had the game rules, exact legal move generation, a perfect simulator, board symmetries, and a closed win/loss objective. The absence of human games did not mean the absence of domain structure.
AlphaZero and general game-playing
AlphaZero generalized the AlphaGo Zero approach to chess, shogi, and Go. The AlphaZero paper describes a single reinforcement learning algorithm that starts from random play, uses no domain knowledge beyond the game rules, and learns a policy and value function through self-play. Within 24 hours, AlphaZero achieved superhuman performance in chess, shogi, and Go, defeating a world-champion program in each game. arXiv
This was an important generalization because chess and shogi are less naturally suited to convolutional board-learning than Go. The AlphaZero paper itself notes that Go has helpful properties such as translational invariance, local board structure, symmetries, and a simpler win/loss outcome, while chess and shogi have asymmetries, long-range piece interactions, larger or more irregular action spaces, and draws. arXiv
AlphaZero’s result therefore strengthened the claim that AlphaGo’s recipe was not Go-only. But it did not prove that the recipe transfers automatically to arbitrary real-world domains. Chess, shogi, and Go are all deterministic, perfect-information, two-player games with exact rules and cheap self-play. The leap from Go to chess and shogi is meaningful; the leap from board games to open-ended engineering, science, or social decision-making is much larger.
Relation to modern reasoning models
AlphaGo is not a direct ancestor of large language models in the architectural sense. It did not use transformers, natural language pretraining, instruction tuning, or autoregressive token prediction. Its environment was a closed game, its reward was a win/loss outcome, and its inference-time search was explicit MCTS over legal moves.
The conceptual relationship is nevertheless important. Modern reasoning models increasingly combine learned priors with additional test-time computation. OpenAI describes its o1 line as using large-scale reinforcement learning to produce more effective chain-of-thought reasoning, with performance improving as both train-time RL compute and test-time “thinking” compute increase. OpenAI DeepSeek-R1 similarly reports that reasoning abilities such as self-reflection, verification, and dynamic strategy adaptation can emerge from reinforcement learning, including variants that do not begin with human-labeled reasoning trajectories. arXiv
The analogy is strongest at the level of Test-Time Compute and Search Over Thoughts. AlphaGo used MCTS to search over game continuations; reasoning models may sample, revise, verify, decompose, call tools, or search over candidate derivations. In both cases, a learned model proposes possibilities and additional computation selects or improves them.
The analogy is weakest where language tasks lack Go’s clean structure. Go has exact rules, legal moves, perfect state observability, and a definitive terminal outcome. Many reasoning tasks have ambiguous goals, incomplete information, unreliable reward signals, and no cheap simulator. For those tasks, AlphaGo is a design inspiration, not a solved transfer recipe.
The Bitter Lesson reading
Rich Sutton’s “The Bitter Lesson” argues that the biggest long-run gains in AI come from methods that leverage computation through search and learning rather than from encoding human domain knowledge. AlphaGo is one of the most frequently cited examples of this lens because it combined large-scale neural learning, self-play, and search to surpass human expertise in a domain where handcrafted knowledge had been insufficient. incompleteideas.net
But AlphaGo 2016 is not a pure bitter-lesson system. It used human expert games, engineered input features, a rollout policy, handcrafted search machinery, and a large distributed inference system. It is better read as a transitional system: enough human knowledge to bootstrap the search, enough learning to exceed human priors, and enough computation to make the combination work.
AlphaGo Zero is the cleaner bitter-lesson exhibit. It removed human game data, learned from self-play, and beat the human-trained AlphaGo Lee and AlphaGo Master systems. UCL Discovery Even there, however, the system benefited from a domain where self-play is meaningful, simulation is exact, and the reward is unambiguous. The bitter lesson does not imply that domain structure disappears; it says that scalable learning and search tend to dominate hand-coded expertise when the domain can be made computationally tractable.
What transferred beyond Go
AlphaGo’s most durable transferable ideas were not Go tactics. They were structural patterns.
1. Learned priors make search tractable
A policy network can convert a huge action space into a ranked set of plausible candidates. This idea transfers to any domain where a model can propose actions, proof steps, code edits, molecular modifications, plans, or tool calls.
2. Learned values reduce rollout cost
A value function can replace or shorten expensive simulation. In Go, the value network estimated win probability; in other settings, analogous evaluators estimate proof success, reward, preference, correctness, safety, or downstream utility.
3. Search can improve the policy that guides it
AlphaGo Zero made this feedback loop explicit. Search produced an improved policy target; the neural network learned to predict that target; the improved network then guided stronger future searches. UCL Discovery This pattern appears in later systems that use self-generated data, verifier-guided training, expert iteration, or search-distillation loops.
4. Self-play can generate an automatic curriculum
In two-player games, self-play naturally creates problems of the right difficulty. As the agent improves, its opponent improves too. This avoids dependence on a fixed human dataset and can discover strategies outside the human distribution.
5. Test-time compute can be a capability multiplier
AlphaGo’s move quality depended not only on network weights but also on search effort. AlphaZero similarly compensated for searching far fewer positions than classical engines by using a neural network to focus search on promising lines; the paper reports that AlphaZero searched far fewer positions per second than Stockfish or Elmo yet achieved stronger results in the evaluated matches. arXiv
What remained Go-specific
The strongest caution about AlphaGo is that Go has unusually favorable properties for self-play search.
Exact simulation
Go permits perfect, cheap simulation. The system can generate millions of games without physical cost, safety risk, or simulator mismatch. Many real-world domains do not have such simulators.
Perfect information
Both players see the entire board. There is no hidden state, deception outside the rules, partial observability, or uncertain environment dynamics. Domains with hidden information require belief-state modeling or different algorithms.
Clear reward
A Go game ends in a win or loss. The reward is sparse but unambiguous. Many real tasks have multi-objective, delayed, contested, or subjective rewards.
Closed action space
Legal moves are enumerable. The policy network chooses from board intersections plus pass. Open-ended tasks such as designing an experiment, writing a program, or making a business decision have much less clean action spaces.
Self-play symmetry
Self-play is meaningful because Go is adversarial and symmetric. In many tasks there is no natural opponent who improves in lockstep.
Useful board geometry
AlphaGo’s convolutional architecture exploited the spatial regularity of the Go board. AlphaZero showed that the recipe could extend to chess and shogi despite less ideal geometry, but the original Go setting was unusually compatible with convolutional neural networks. arXiv
These constraints do not diminish AlphaGo’s achievement. They define the boundary of the lesson. AlphaGo proved that deep RL plus search could master a previously resistant intellectual domain; it did not prove that the same recipe automatically solves open-world reasoning.
The open transfer question
The central open question is not whether AlphaGo’s ideas transferred at all. They clearly transferred within games: AlphaGo Zero improved on AlphaGo, and AlphaZero generalized the approach to chess and shogi. UCL Discovery DeepMind also argues that AlphaGo-like search, planning, and reinforcement learning ideas influenced later systems, including AlphaZero, MuZero, AlphaProof, AlphaGeometry, AlphaDev, and aspects of Gemini-era reasoning and tool-use research. Those claims are plausible as a lineage claim, but they come from DeepMind’s own retrospective and should be read as institutional framing rather than independent proof of broad transfer. Google DeepMind
The harder question is what problem structure is required for the recipe to work. AlphaGo-style systems need at least some substitute for Go’s rules, simulator, value target, and search space. In formal mathematics, a proof checker can supply part of that structure. In code, tests or verifiers can supply part of it. In robotics, learned world models may approximate it, but errors compound. In scientific discovery, surrogate models and laboratory feedback can help, but the cost and uncertainty of evaluation are much higher.
Thus the honest conclusion is mixed. AlphaGo’s design choices transferred powerfully to deterministic games and influenced the broader search-plus-learning paradigm. The evidence is thinner for domains without exact simulators, crisp verifiers, or cheap self-play. The transfer problem remains one of the central questions in General-Purpose Reinforcement Learning and AI Reasoning Systems.
Primary source map
| Source | What it establishes |
|---|---|
| Silver et al. 2016, “Mastering the game of Go with deep neural networks and tree search” | Original AlphaGo system: supervised policy, RL policy, value network, MCTS, Fan Hui result, computer-Go benchmark results |
| Silver et al. 2017, “Mastering the game of Go without human knowledge” | AlphaGo Zero: no human data, single policy-value network, no rollouts, self-play from random initialization, 100–0 vs AlphaGo Lee, 89–11 vs AlphaGo Master |
| Silver et al. 2017/2018, “Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm” | AlphaZero: generalization of AlphaGo Zero-style learning to chess, shogi, and Go |
| DeepMind AlphaGo official history | Public match context: Fan Hui, Lee Sedol 4–1, Move 37, Move 78, and legacy framing |
| DeepMind Future of Go Summit posts | Master-era context and Ke Jie summit framing |
| Sutton, “The Bitter Lesson” | Conceptual framing: scalable search and learning as long-run drivers of AI progress |
| OpenAI o1 and DeepSeek-R1 technical sources | Modern reasoning-model analogy: RL-trained reasoning and increased test-time computation |
Why AlphaGo still matters
AlphaGo’s importance is not that it solved Go. It matters because it changed what seemed strategically plausible in AI research. Before AlphaGo, many researchers treated high-level Go play as requiring a kind of human strategic intuition that would resist machine learning for many years. AlphaGo showed that such intuition could be approximated by neural networks and sharpened by search.
The system also changed the interpretation of search. Search was no longer only a brute-force enumeration procedure. In AlphaGo, search became a way to refine learned judgment, generate better policy targets, and allocate computation adaptively. This reframing continues to matter for reasoning systems: the question is not whether to use learning or search, but how to make each improve the other.
The strongest version of AlphaGo’s legacy is therefore methodological rather than game-specific: learn a prior, search with it, learn from the search, and repeat. The strongest caveat is equally important: this loop works best when the world can be made game-like enough for search and feedback to be reliable.
Companion entries
Core theory: Monte Carlo Tree Search, Policy Networks, Value Functions, Policy-Value Search, Self-Play Reinforcement Learning, Expert Iteration, Test-Time Compute, Search and Learning
AlphaGo lineage: AlphaGo Zero, AlphaZero, MuZero, AlphaProof, AlphaGeometry, AlphaDev
Historical context: Computer Go, Lee Sedol Match, Move 37, DeepMind, Reinforcement Learning Milestones
Research strategy: Bitter Lesson, Scaling Laws, General-Purpose Reinforcement Learning, Simulator-Based Learning, Verifier-Guided Learning
Modern reasoning systems: Reasoning Models, Search Over Thoughts, Chain-of-Thought Reinforcement Learning, Tool-Augmented Reasoning, Inference-Time Scaling
Limits and counterarguments: Go-Specific Inductive Bias, Simulator Gap, Reward Specification, Self-Play Limitations, Transfer Beyond Games