InfoDensity: Rewarding Information-Dense Traces for Efficient Reasoning

Researchers propose InfoDensity, a reinforcement learning reward framework that optimizes Large Language Models for efficient reasoning by measuring information density rather than just output length. The method tracks entropy trajectories to identify high-quality intermediate reasoning steps, achieving better accuracy-efficiency trade-offs on mathematical and general reasoning benchmarks.

The research addresses a fundamental inefficiency in extended-reasoning LLMs: their tendency to generate verbose, redundant reasoning traces that consume computational resources without improving output quality. Rather than treating verbosity as a simple length optimization problem, the researchers identify a deeper issue—poor intermediate reasoning quality that manifests as redundancy. By analyzing per-token predictive entropy across reasoning trajectories, they discovered that superior reasoning exhibits two measurable properties: low uncertainty convergence (reaching confident conclusions) and fast uncertainty descent (achieving confidence efficiently). This insight reframes reasoning quality as information density, where each token meaningfully contributes to reducing model uncertainty about the final answer.

The InfoDensity framework captures both properties through a single entropy-based metric, weighted to reward achieving equivalent quality with fewer tokens. This approach sidesteps reward hacking vulnerabilities present in length-only optimizations, where models might simply truncate reasoning without improving logical coherence.

For the AI industry, this research has practical implications for deploying reasoning models in production environments where computational cost directly impacts feasibility and profitability. As organizations increasingly adopt extended-reasoning models for complex problem-solving, efficiency gains translate directly to reduced inference costs and faster response times. The framework demonstrates measurable improvements on competitive benchmarks, suggesting adoption potential across research institutions and AI service providers developing reasoning-capable systems.

• →InfoDensity uses entropy trajectory analysis to reward information-dense reasoning rather than penalizing length alone.
• →High-quality reasoning exhibits fast uncertainty descent and low uncertainty convergence, measurable through per-token predictive entropy.
• →The framework outperforms state-of-the-art baselines on accuracy-efficiency trade-offs for mathematical and general reasoning tasks.
• →This approach prevents reward hacking vulnerabilities inherent in simple length-based optimization methods.
• →The research provides computational efficiency gains relevant to production deployment of extended-reasoning LLMs.