SPARE: Single-Pass Annotation with Reference-Guided Evaluation for Automatic Process Supervision and Reward Modelling

Researchers introduce SPARE, a new framework for automated process supervision in Large Language Models that improves multi-step reasoning capabilities. The method shows significant efficiency gains, using only 16% of training samples compared to human-labeled baselines while achieving competitive performance with 2.3x speedup.

• →SPARE enables efficient per-step annotation for LLM training by jointly aligning solution steps to reference solutions in a single generation.
• →The framework demonstrates consistent improvements across mathematical reasoning, multi-hop question answering, and spatial reasoning tasks.
• →SPARE achieves data-efficient out-of-distribution generalization using only ~16% of training samples compared to human-labeled baselines.
• →The method offers 2.3x speedup in token count while maintaining competitive performance with MCTS-based approaches.
• →Manual analysis reveals complementary precision-recall characteristics with existing methods, suggesting potential for ensemble approaches.