On the Step Length Confounding in LLM Reasoning Data Selection

Researchers identify a critical flaw in naturalness-based data selection methods for large language model reasoning datasets, where algorithms systematically favor longer reasoning steps rather than higher-quality reasoning. The study proposes two corrective methods (ASLEC-DROP and ASLEC-CASL) that successfully mitigate this 'step length confounding' bias across multiple LLM benchmarks.

The development of large reasoning models depends heavily on high-quality training datasets, with researchers traditionally relying on naturalness-based selection that ranks samples by average log probability. This research reveals a fundamental methodological problem: the selection mechanism inadvertently privileges verbosity over actual reasoning quality, distorting the composition of training data for advanced LLMs.

The root cause stems from low-probability first tokens in reasoning steps. When computing average log probabilities across entire reasoning chains, longer steps naturally dilute the negative impact of these problematic early tokens, artificially inflating overall probability scores. This creates a systematic bias that rewards quantity of tokens rather than reasoning validity. The phenomenon represents a subtle but pervasive issue in current dataset construction pipelines, affecting institutions across the AI industry.

This finding has significant implications for LLM development efficiency and performance. Training datasets contaminated by this confounding bias may optimize for the wrong signal, potentially leading to models that learn to generate unnecessarily verbose reasoning rather than concise, effective problem-solving approaches. The proposed solutions—removing first-token probabilities or applying causal debiasing regression—offer practical remedies that maintain dataset quality while preserving legitimate reasoning examples.

The research validates its approach across four different LLMs and five evaluation benchmarks, suggesting broad applicability. As competition intensifies for superior reasoning capabilities, teams constructing training datasets should audit existing pipelines for similar hidden biases. The study exemplifies how seemingly minor statistical quirks in data selection can systematically corrupt model development, warranting increased scrutiny of foundational dataset construction methods.

• →Naturalness-based data selection for LLM reasoning inadvertently favors longer reasoning steps over higher-quality ones due to low-probability first tokens
• →Longer steps dilute the negative influence of problematic initial tokens, artificially inflating average log probabilities and distorting training data composition
• →ASLEC-DROP and ASLEC-CASL methods effectively mitigate step length confounding across multiple LLM architectures and benchmarks
• →This bias could cause reasoning models to optimize for verbosity rather than concise, effective problem-solving capabilities
• →Dataset construction pipelines across the AI industry should audit for similar hidden statistical biases affecting model training quality