How Transformers Learn to Plan via Multi-Token Prediction

Researchers demonstrate that multi-token prediction (MTP) outperforms standard next-token prediction (NTP) for training language models on reasoning tasks like planning and pathfinding. Through theoretical analysis of simplified Transformers, they reveal that MTP enables a reverse reasoning process where models first identify end states then reconstruct paths backward, suggesting MTP induces more interpretable and robust reasoning circuits.

This research addresses a fundamental challenge in language model training: capturing global reasoning structure rather than optimizing for local token-by-token accuracy. While next-token prediction has dominated the field, its myopic focus on immediate predictions often fails on tasks requiring planning or multi-step reasoning. The shift toward multi-token prediction represents a meaningful evolution in how we design objective functions for neural networks.

The empirical results span both controlled synthetic environments and realistic benchmarks, demonstrating MTP's consistent advantages on graph pathfinding, Countdown problems, and boolean satisfiability tasks. The theoretical contribution proves particularly valuable: by analyzing a two-layer Transformer on star graphs, the authors reveal MTP's mechanism—a gradient decoupling property that creates cleaner learning signals. This allows models to develop two-stage reasoning: first attending to goals, then reconstructing intermediate steps backward.

For the AI development community, this work has significant implications. It suggests that training objectives fundamentally shape how models reason, with MTP naturally biasing toward interpretable circuit development rather than opaque pattern matching. This aligns with broader efforts to understand and control AI behavior through architectural and objective function design choices.

The findings indicate that future language model improvements may depend less on scaling alone and more on intelligently designed training objectives that match task structure. As models tackle increasingly complex reasoning, this research provides both theoretical grounding and practical evidence that multi-token objectives deserve wider adoption in training pipelines.

• →Multi-token prediction outperforms next-token prediction on reasoning tasks by capturing global structure rather than local token dependencies
• →MTP induces a gradient decoupling property that enables cleaner training signals and more interpretable model reasoning circuits
• →Theoretical analysis reveals MTP enables reverse reasoning: models identify end states first, then reconstruct paths backward through intermediate nodes
• →Results hold across synthetic benchmarks and realistic problems like Countdown and boolean satisfiability, suggesting broad applicability
• →Training objective design may be as critical as model scale for developing robust reasoning capabilities in language models