Understanding the Role of Training Data in Test-Time Scaling

Research paper analyzes test-time scaling in large language models, revealing that longer reasoning chains (CoTs) can reduce training data requirements but may harm performance if relevant skills aren't present in training data. The study provides theoretical framework showing that diverse, relevant, and challenging training tasks optimize test-time scaling performance.

• →Test-time scaling allows models to use extra compute for longer reasoning chains to solve complex problems through step-by-step breakdown.
• →Increased test-time compute can reduce the number of in-context examples needed during training.
• →Test-time scaling can actually harm performance when required skills are insufficiently represented in training data.
• →Task difficulty is characterized by the smallest eigenvalue of feature covariance matrix in the theoretical framework.
• →Training on diverse, relevant, and challenging tasks yields the best test-time scaling performance results.