y0news
AnalyticsDigestsSourcesTopicsRSSAICrypto

#student-teacher News & Analysis

2 articles tagged with #student-teacher. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

2 articles
AIBullisharXiv – CS AI · Mar 177/10
🧠

Explain in Your Own Words: Improving Reasoning via Token-Selective Dual Knowledge Distillation

Researchers developed Token-Selective Dual Knowledge Distillation (TSD-KD), a new framework that improves AI reasoning by allowing smaller models to learn from larger ones more effectively. The method achieved up to 54.4% better accuracy than baseline models on reasoning benchmarks, with student models sometimes outperforming their teachers by up to 20.3%.

AINeutralarXiv – CS AI · Jun 26/10
🧠

OPD+: Rethinking the Advantage Design for On-Policy Distillation

Researchers propose OPD+, an improved on-policy distillation framework that corrects mathematical flaws in existing knowledge transfer methods between language models. The work proves that stop-gradient operations in current approaches produce biased reward estimates and introduces a corrected optimization framework supporting multiple f-divergence functions, with validation on reasoning and tool-use tasks.