AINeutralarXiv – CS AI · 6h ago6/10
🧠
Hard or Just Unreached? Diagnosing the Sampling Blind Spot in Math-Reasoning Difficulty Estimation
Researchers identify a critical blind spot in pass@k, the standard metric for evaluating math reasoning difficulty in large language models. Their analysis reveals that 10-23% of problems marked as unsolvable through sampling can actually be solved using deterministic inference with activation grafting perturbations, suggesting current difficulty assessments systematically underestimate model capabilities.