AIBullisharXiv – CS AI · 5h ago7/10
🧠
Data-Efficient Autoregressive-to-Diffusion Language Models via On-Policy Distillation
Researchers introduce On-Policy Diffusion Language Models (OPDLM), a technique that converts autoregressive language models into diffusion models using 15-7,000x fewer training tokens. The method addresses fundamental efficiency problems by eliminating train-inference mismatches and preserving knowledge from the original model through on-policy distillation.