LC-ERD: Mining Latent Logic for Self-Evolving Reasoning via Consistency-Regulated Reward Decomposition

Researchers introduce LC-ERD, a framework for improving Large Language Model reasoning by mining high-quality supervision signals through consistency-regulated reward decomposition. The method addresses critical challenges in self-aligned LLM training by reducing label noise, providing granular step-level guidance, and preventing distributional collapse, demonstrating potential improvements in reasoning quality and generalization.

LC-ERD addresses a fundamental bottleneck in LLM development: the scarcity of high-quality training data for reasoning tasks. Current self-alignment approaches suffer from inherent flaws—reward signals often reinforce statistical patterns rather than logical correctness, creating a veneer of accuracy that masks cascading errors deeper in reasoning chains. The framework's innovation lies in treating reward decomposition as a latent structure mining problem, using consensus from multiple logical pathways within the model to denoise training signals.

This research emerges from the broader trend of moving beyond supervised fine-tuning toward self-improvement mechanisms. Process-level training data remains expensive and limited, making endogenous reward systems increasingly attractive. However, previous approaches like GRPO treat entire reasoning chains atomically, missing opportunities to identify which individual steps contribute value or introduce errors. LC-ERD's Multi-Agent Value Decomposition protocol, grounded in game-theoretic principles, enables granular attribution of contribution at each reasoning step.

For the AI development community, this work suggests a path toward more efficient self-evolution of reasoning capabilities without relying on extensive human annotation. The framework's ability to expose trade-offs between logic consistency and accuracy provides valuable insights for practitioners choosing model behaviors. Developers building reasoning-heavy applications could benefit from models trained with such methods, achieving more robust generalization across diverse problem domains.

The immediate impact remains within academic and research circles, though successful implementation could accelerate deployment of more reliable reasoning systems. The released codebase enables reproducibility and further iteration, positioning this as a reference point for future self-alignment research.

• →LC-ERD mitigates label noise from mimetic bias by aggregating consensus from the model's latent logical pathways rather than relying on single reward signals.
• →Multi-Agent Value Decomposition enables step-level supervision instead of treating entire reasoning chains as monolithic units, improving feedback granularity.
• →The framework reveals trade-offs between logic consistency and raw accuracy, helping practitioners make informed model selection decisions.
• →Addressing distributional collapse prevents reward signals from merely amplifying pre-training biases, improving generalization across unseen problems.
• →Open-sourced implementation democratizes access to advanced self-alignment techniques for the broader AI research community.