A Pre-Registered Causal Partition of Self-Consistency Elicitation and Reward Design in RLVR

Researchers present a pre-registered causal decomposition framework that reveals how reinforcement learning from verifiable rewards (RLVR) conflates self-consistency elicitation with genuine reward-design effects. Through controlled experiments, they demonstrate that naive performance metrics systematically overestimate reward-design impact by 50-95%, with elicitation dominating in weak-prior regimes. The work provides diagnostic tools to audit published alignment research and expose methodological confounds.

This paper addresses a fundamental measurement problem in reinforcement learning from verifiable rewards, a technique increasingly used to improve AI reasoning capabilities. The authors demonstrate that standard evaluation metrics—computing accuracy gains from true rewards minus random baselines—systematically misattribute performance improvements to reward design when much of the gain actually stems from self-consistency elicitation, a policy-sharpening effect unrelated to genuine signal quality.

The research employs rigorous causal decomposition methodology through a controlled tabular simulator, deriving an exact three-component partition: null effects, elicitation terms, and reward-design signal. Across varying prior strengths, reward-design fractions of the naive estimand range from 5-14%, suggesting published benchmarks substantially overstate the efficacy of reward-design interventions. Critical findings include sign-flipping elicitation effects at self-consistency crossover points and significant non-additive interactions, indicating the two mechanisms interact rather than combine linearly.

For AI alignment research, this work establishes methodological accountability through two re-audits of published results, one showing elicitation-dominated performance (98% of gains) and another appearing reward-design dominated (118%)—the latter suggesting additional confounds. The authors' pre-registration framework and commitment to publish regardless of outcome direction strengthens scientific credibility.

The broader implication centers on AI capability measurement integrity. As alignment techniques proliferate, proper causal attribution becomes essential for distinguishing genuinely effective methods from those producing spurious signal. The release of an open-source audit harness enables systematic re-examination of prior work, potentially reshaping how researchers evaluate reasoning improvements.

• →Standard RLVR metrics systematically overestimate reward-design impact by 50-95%, conflating it with self-consistency elicitation effects
• →Causal decomposition reveals reward-design contributes only 5-14% of naive performance gains at most parameter settings
• →Non-additive interactions between elicitation and reward-design mechanisms challenge linear interpretations of combined effects
• →Re-audits of published papers show either elicitation-dominated or potentially over-attributed reward-design results, indicating widespread measurement bias
• →Pre-registered audit framework with open-source tooling enables systematic re-examination of alignment research methodology