AlphaGo’s 37th Move Still Unmatched by LLMs

A decade after the historic match, Thore Graepel — co-architect of AlphaGo — argues that today’s large language models still lack true reasoning capability.

🌐 The Legendary Move: March 2016, Seoul

In March 2016, at the Four Seasons Hotel in Seoul, history was made during Game 2 of the AlphaGo vs. Lee Sedol match. Midway through, AlphaGo placed a black stone on the fifth line — an unconventional, seemingly illogical move.

AlphaGo vs. Lee Sedol — Game 2, Move 37

Commentators were stunned. Some even suspected a bug — because, to professional Go players, the move appeared almost sacrificial. Yet it proved decisive. AlphaGo won that game — and the series 4–1.

Lee Sedol later reflected: “I thought AlphaGo was just a probability-calculating machine. But seeing that move, I changed my mind. AlphaGo must possess creativity.”

That was Move 37 — now immortalized as the “divine move.”

⚙️ What Made Move 37 Possible?

Thore Graepel — AlphaGo’s core architect and longtime DeepMind researcher — recently left Google DeepMind to found a new venture focused on building machines with genuine reasoning. His critique, published in MIT Technology Review, is stark: “Don’t Be Fooled — LLMs Don’t Reason.”

Thore Graepel's MIT Technology Review article cover

Graepel dismantles the myth that Move 37 was pure “intuition.” Instead, it emerged from a hybrid architecture:

Policy Network (System 1): Learned human intuition — predicted what strong players would do. For Move 37, this network assigned it a mere ~0.01% probability.
Monte Carlo Tree Search (System 2): A deliberate, step-by-step search mechanism that explored thousands of future board states — and revealed Move 37 led to superior long-term outcomes.

✅ True reasoning = intuition + verifiable search

As Graepel explains using Daniel Kahneman’s dual-process framework:
– System 1: Fast, associative, pattern-driven (LLMs excel here)
– System 2: Slow, sequential, self-monitoring, belief-updating (LLMs lack this)

AlphaGo architecture diagram

Deep Blue beat Kasparov via brute-force search — but AlphaGo succeeded where brute force failed: it combined neural intuition with guided search to navigate围棋’s astronomical complexity (~10¹⁷⁰ possible positions).

❌ Why LLMs Aren’t Reasoning Machines (Yet)

Graepel identifies three foundational deficits in current LLMs:

Shortcoming	Explanation	Implication
No auditable cognitive state	LLMs maintain no explicit, inspectable record of hypotheses, confidence levels, or open questions. Everything is buried in weights.	Cannot trace why a conclusion was reached — only what was said.
No separation of knowledge & reasoning	Knowledge and inference are inseparable — both encoded in static weights. There’s no modular “belief engine” or “reasoning scheduler.”	No capacity for structured doubt, revision, or meta-cognition.
Post-hoc rationalization, not real-time reasoning	Chain-of-thought outputs are retrofitted narratives, not live deliberation traces. Studies show models often generate plausible-sounding but fictional reasoning paths.	Outputs may be convincing — but not trustworthy in high-stakes domains.

Comparison: AlphaGo vs. LLM reasoning architecture

🧠 Toward Auditable Machine Reasoning

Graepel proposes a new paradigm — inspired by AlphaGo’s design:

🔹 Maintain an explicit, evolving cognitive state — like a dynamic “belief tree” tracking known facts, uncertainties, hypotheses, and evidence.

🔹 Treat reasoning as actions on that state: e.g., “query PubMed,” “run simulation X,” “reject hypothesis Y given Z.”

🔹 Introduce a validator module: Assess each action by how much uncertainty it resolves — not just linguistic coherence.

This vision — dubbed “the amped-up scientific method” — aims to produce certified, auditable knowledge, not just fluent text.

Conceptual diagram: Cognitive state + reasoning actions

💬 Debate Ignites: Is This Fair?

The article sparked intense discussion:

David Duvenaud (UofT, NeurIPS Best Paper) challenged: “By your criteria, do humans truly reason?” — noting we also lack perfect cognitive transparency.
Graepel replied: “Just as paper & pencil enhance human reasoning, external tools can scaffold LLMs — but only if they’re integrated into a structured, accountable process.”

Duvenaud-Graepel exchange on X (Twitter)

Other critiques included:
– KingBroken: “Paper is working memory — not new reasoning capacity.”
– Daniel Grant: “Are you building a stronger internal model, or just better tool orchestration?”
– Katherine Moore: “Language itself is the wrong substrate — formal representations may be essential.”

Graepel acknowledged these — and left the final question open:

🌐 What would a non-linguistic, formal representation language for real-world reasoning look like?

Community responses and counterpoints

🎯 Why It Matters: Beyond Go Boards

In medicine, climate modeling, drug discovery, or engineering — where lives and systems depend on traceable justification — fluency without reasoning is dangerous.

✨ Move 37 mattered not because it was surprising — but because it was justified, searched, and verified.

Ten years later, Graepel reminds us: Scaling System 1 doesn’t birth System 2.

📌 The next leap isn’t bigger models — it’s new architectures that make reasoning explicit, inspectable, and accountable.*

🔗 Source: MIT Technology Review — “Don’t Be Fooled: LLMs Don’t Reason” (Oct 2026)

Article adapted from “Ji Qi Zhi Xin” (Machine Heart), republished by AITNT.