When Shallow Wins: Silent Failures and the Depth-Accuracy Paradox in Latent Reasoning

Research reveals that state-of-the-art AI mathematical reasoning models like Qwen2.5-Math-7B achieve 61% accuracy primarily through unreliable computational pathways, with only 18.4% using stable reasoning. The study exposes that 81.6% of correct predictions come from inconsistent methods and 8.8% are confident but incorrect outputs.

• →Only 18.4% of correct AI predictions use stable, faithful reasoning while 81.6% emerge through computationally inconsistent pathways.
• →8.8% of all AI model predictions are silent failures - confident yet incorrect outputs that mask reliability issues.
• →Scaling model parameters from 1.5B to 7B (4.7x increase) provided zero accuracy improvement on the evaluated subset.
• →Reasoning quality shows weak negative correlation with correctness, indicating benchmark accuracy can mask computational unreliability.
• →Current evaluation methods fail to measure AI reasoning stability beyond single-sample metrics, requiring evaluation reforms.