Finding the Evidence: Discovering Decision-Supporting Tokens for On-Policy Reasoning Distillation

Researchers introduce DEAR, a novel on-policy distillation method that improves AI model training by distinguishing between decision tokens (where models branch) and evidence tokens (supporting intermediate steps). The technique achieves significant performance gains of up to 5.7% on code generation and 2.5% on math benchmarks compared to standard distillation approaches.

This research addresses a fundamental challenge in knowledge distillation: effectively transferring reasoning capabilities from larger teacher models to smaller student models. Traditional on-policy distillation focuses primarily on capturing decision points—moments where the model must choose between different reasoning paths—but overlooks the intermediate evidence that justifies those decisions. DEAR's innovation lies in dual discovery mechanisms that identify both components separately, recognizing that they require different detection strategies based on student confidence patterns.

The methodology builds on established machine learning principles but applies them with new insight. Decision points emerge where student models express highest uncertainty (high entropy), making them naturally discoverable. Evidence tokens, conversely, hide in regions where students display false confidence—positions where they assign high probability to incorrect answers. By measuring hidden-state similarity to decision anchors and leveraging teacher-student divergence, DEAR prioritizes the most significant knowledge gaps, ensuring efficient transfer of reasoning capability.

For the AI development community, this work has implications for model efficiency and deployment. Smaller, faster models trained with DEAR could maintain competitive reasoning performance on mathematical and programming tasks while reducing computational overhead. This directly benefits applications requiring real-time inference or edge deployment. The consistent improvements across multiple student-teacher configurations suggest the approach generalizes beyond specific architectures.

The research opens pathways for enhanced distillation in specialized domains. Future work might explore whether evidence discovery mechanisms transfer to other reasoning-heavy tasks like planning, multi-step problem solving, or scientific reasoning. Organizations developing efficient AI systems should monitor whether these techniques prove practical for production-scale implementations.

• →DEAR distinguishes between decision tokens (uncertainty-driven) and evidence tokens (confidence-based failures) requiring separate discovery mechanisms.
• →The method achieves +2.5% improvement on competition math and +5.7% on code generation across multiple model configurations.
• →Evidence tokens represent substantive knowledge that previous distillation methods fail to transfer, creating optimization opportunities.
• →Hidden-state cosine similarity combined with teacher-student divergence metrics effectively identifies supporting reasoning steps.
• →Smaller student models trained with DEAR maintain competitive reasoning performance while reducing computational requirements for deployment.