Implicit Compression Regularization: Concise Reasoning via Internal Shorter Distributions in RL Post-Training

Researchers introduce Implicit Compression Regularization (ICR), a novel training method that reduces unnecessary verbosity in AI reasoning models without sacrificing accuracy. By leveraging the shortest correct responses within training batches as natural compression targets, ICR maintains performance while producing more concise outputs—addressing a key limitation of existing length-penalty approaches.

This research addresses a fundamental challenge in reinforcement learning-based language model training: models optimized for reasoning accuracy often generate excessively long responses, a phenomenon termed 'overthinking.' Traditional solutions like length penalties risk degrading accuracy by incentivizing underthinking, while early-exit strategies make risky assumptions about which reasoning steps are dispensable.

The key innovation lies in the researchers' observation that the relationship between response length and correctness evolves predictably during training. Initially, shorter responses correlate with higher accuracy, but this relationship inverts as training progresses and models drift toward underthinking. ICR exploits this dynamic by creating a virtual distribution based on the shortest correct responses observed in on-policy rollouts, providing a compression signal grounded in empirically validated shorter trajectories.

This approach carries significant implications for the broader AI reasoning landscape. As language models increasingly tackle complex mathematical and knowledge-intensive tasks, controlling response length while maintaining accuracy becomes economically critical—shorter outputs reduce computational costs and latency. The method's demonstrated improvements across multiple backbones and benchmarks suggest it could become a standard component in production reasoning systems.

For developers and organizations deploying reasoning-capable LLMs, ICR represents a pathways toward more efficient models without quality degradation. The technique's agnostic nature relative to specific model architectures enhances its practical applicability. Future work should explore ICR's performance on frontier models and its interaction with other recent post-training innovations, particularly in scaling behaviors and transfer learning scenarios.

• →ICR uses the shortest correct responses in training batches as natural compression targets, eliminating need for manual length penalties
• →The method maintains or improves accuracy while consistently reducing response length across multiple reasoning benchmarks
• →Training dynamics reveal ICR preserves the length-accuracy correlation better than existing methods, preventing drift toward underthinking
• →The approach is model-agnostic and shows gains across three different reasoning backbones and mathematical/knowledge-intensive tasks
• →Shorter, efficient reasoning outputs have direct computational and cost implications for deploying reasoning-capable language models