Extracting Search Trees from LLM Reasoning Traces Reveals Myopic Planning

Researchers developed a method to extract and analyze search trees from LLM reasoning traces, revealing that large language models use shallower, more myopic planning strategies compared to humans. While LLMs generate extended chain-of-thought reasoning, their actual decision-making is driven primarily by shallow search rather than deep lookahead, contrasting sharply with human expert planning.

This research addresses a critical gap in understanding how reasoning-focused LLMs actually plan and make decisions during complex problem-solving tasks. The study uses four-in-a-row as a controlled domain to extract explicit search trees from model reasoning traces, then applies computational modeling to determine which planning components genuinely influence move selection. The findings reveal a fundamental architectural limitation: LLMs generate deep reasoning chains that appear deliberative, but their actual choices depend on breadth of exploration at shallow depths rather than genuine long-term lookahead.

This work builds on growing evidence that LLM reasoning processes differ substantially from human cognition. While humans leverage deep search expertise through extensive training and pattern recognition across vast experience, LLMs appear to rely on surface-level pattern matching within their generated tokens. The causal intervention study—selectively removing deep reasoning paragraphs to test their causal impact—provides compelling evidence that deep nodes in reasoning traces are largely decorative rather than functionally integrated into decision-making.

The implications extend beyond game-playing benchmarks. If LLMs fundamentally struggle with genuine long-horizon planning, this constrains their reliability for strategic domains including financial modeling, code optimization, scientific research planning, and policy analysis. The research suggests that scaling model size or reasoning token length alone may not address this myopia. Organizations deploying LLMs for planning-heavy tasks should recognize that explicit chain-of-thought outputs may overstate actual planning capability, while the framework itself offers researchers a generalizable approach for auditing planning behavior across different domains and model architectures.

• →LLM planning is structurally shallower than human planning, with performance driven by breadth rather than depth of search.
• →LLMs generate deep reasoning chains that don't causally influence final decisions, suggesting decorative rather than functional deliberation.
• →Move selection in reasoning models is best predicted by myopic algorithms ignoring deep lookahead nodes entirely.
• →The dissociation between reasoning output and actual decision drivers raises questions about LLM reliability in strategic domains.
• →A new generalizable framework enables researchers to audit and interpret LLM planning structure across different problem domains.