Reasoning Compression with Mixed-Policy Distillation

Researchers introduce Mixed-Policy Distillation (MPD), a technique that compresses reasoning in smaller language models by having larger teacher models rewrite student-generated reasoning traces into more concise versions. The method reduces token usage by up to 27.1% while maintaining or improving performance, addressing critical deployment constraints around memory, latency, and serving costs.

Mixed-Policy Distillation represents a pragmatic solution to a growing tension in AI deployment: reasoning-capable language models excel at problem-solving through verbose intermediate steps, but this capability becomes economically unfeasible at scale. The research directly addresses why smaller models struggle with efficiency—they naturally generate longer, more redundant reasoning traces than larger counterparts, making them unsuitable for resource-constrained environments despite their lower computational footprint.

The technical innovation bridges two existing distillation approaches. Traditional on-policy distillation trains students to mimic teacher outputs, perpetuating verbose patterns. Off-policy approaches use pure teacher trajectories but risk distribution mismatch when students diverge. MPD combines benefits by having teachers compress student-sampled trajectories, preserving the student's exploration while injecting efficiency guidance. This hybrid approach maintains policy alignment while improving token efficiency.

For the AI infrastructure industry, this development has significant implications. Deployment costs scale directly with token generation; reducing tokens by 27% translates to proportional improvements in latency, memory consumption, and serving expenses. The 1.7B parameter experiments suggest the technique scales effectively to practical model sizes, potentially accelerating adoption of smaller models in production environments. Organizations can now deploy compact reasoning models without sacrificing performance quality.

The research also signals momentum in making reasoning-capable models more commercially viable. As inference costs drive adoption decisions, techniques enabling efficient small-model reasoning reshape the competitive landscape away from larger models. Future work will likely explore how MPD scales to larger student models and whether similar compression transfers across different model architectures.

• →Mixed-Policy Distillation enables 27.1% token reduction in small language models while maintaining reasoning performance
• →Larger teacher models can compress student reasoning by rewriting trajectories into more concise forms
• →The technique preserves student policy exploration while injecting teacher-guided compression, balancing multiple objectives
• →Smaller models with efficient reasoning become more viable for resource-constrained deployments
• →Hybrid distillation combining on-policy and off-policy approaches outperforms either method alone