Can RL Teach Long-Horizon Reasoning to LLMs? Expressiveness Is Key

Researchers introduce ScaleLogic, a synthetic reasoning framework that systematically studies how reinforcement learning improves LLM reasoning across varying task difficulty and logical complexity. The study reveals that RL training compute follows a power law with reasoning depth, with scaling efficiency improving when models train on more expressively complex logic, suggesting that training content quality matters as much as training volume.

This research addresses a fundamental gap in AI development: understanding how to systematically improve LLM reasoning through reinforcement learning. Rather than relying on benchmark chasing, the authors created a controlled environment that isolates and measures two critical variables—proof depth and logical expressiveness—enabling reproducible, scalable analysis of AI training dynamics.

The power-law relationship discovered here ($T \propto D^{\gamma}$) provides quantifiable insights into training efficiency. The scaling exponent increasing from 1.04 to 2.60 as logical complexity grows reveals that more sophisticated reasoning demands exponentially more compute. This isn't merely academic; it informs practical decisions about training resource allocation and curriculum design for production AI systems.

The transfer learning results demonstrate meaningful real-world impact: models trained on more expressive logic achieve up to 10.66-point improvements on downstream mathematics and reasoning benchmarks, while requiring fewer computational resources than training on simpler logics. This inverts conventional training wisdom—it's not just about scale, but about training on qualitatively richer problems. The finding that curriculum-based training substantially improves scaling efficiency suggests that the order and structure of training data significantly impacts learning trajectories.

For AI developers and organizations building reasoning-capable systems, this research provides empirical grounding for training strategy decisions. It clarifies why training on diverse, complex reasoning tasks yields better transfer than simply scaling basic reasoning tasks. The framework itself offers a reusable tool for future researchers studying reasoning capabilities across different logical domains and architectures.

• →RL training compute follows a power law with reasoning depth, enabling predictable scaling estimates across task difficulty levels
• →Scaling exponent increases monotonically with logical expressiveness (1.04 to 2.60), showing more complex reasoning requires exponentially more compute
• →Models trained on expressive logic transfer more effectively to downstream tasks, gaining up to 10.66 points on mathematics and reasoning benchmarks
• →Training content quality and diversity matter as much as training volume for developing robust reasoning capabilities
• →Curriculum-based training substantially improves scaling efficiency, making training strategy crucial alongside raw compute allocation