#student-teacher News & Analysis

3 articles tagged with #student-teacher. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

3 articles

AIBullisharXiv – CS AI · Jun 237/10

🧠

Finding the Evidence: Discovering Decision-Supporting Tokens for On-Policy Reasoning Distillation

Researchers introduce DEAR, a novel on-policy distillation method that improves AI model training by distinguishing between decision tokens (where models branch) and evidence tokens (supporting intermediate steps). The technique achieves significant performance gains of up to 5.7% on code generation and 2.5% on math benchmarks compared to standard distillation approaches.

AIBullisharXiv – CS AI · Mar 177/10

🧠

Explain in Your Own Words: Improving Reasoning via Token-Selective Dual Knowledge Distillation

Researchers developed Token-Selective Dual Knowledge Distillation (TSD-KD), a new framework that improves AI reasoning by allowing smaller models to learn from larger ones more effectively. The method achieved up to 54.4% better accuracy than baseline models on reasoning benchmarks, with student models sometimes outperforming their teachers by up to 20.3%.

AINeutralarXiv – CS AI · Jun 26/10

🧠

OPD+: Rethinking the Advantage Design for On-Policy Distillation

Researchers propose OPD+, an improved on-policy distillation framework that corrects mathematical flaws in existing knowledge transfer methods between language models. The work proves that stop-gradient operations in current approaches produce biased reward estimates and introduces a corrected optimization framework supporting multiple f-divergence functions, with validation on reasoning and tool-use tasks.