Self-Consistency Is Losing Its Edge: Diminishing Returns and Rising Costs in Modern LLMs

Researchers demonstrate that self-consistency—a technique where LLMs sample multiple reasoning paths to improve accuracy—delivers diminishing returns on modern models. Testing with Gemini 2.5 shows minimal accuracy gains (0.4-1.6%) while token costs scale linearly, suggesting the technique has become inefficient as model reliability improves.

The self-consistency technique emerged when large language models were prone to frequent reasoning errors, making multiple-path sampling a practical way to improve reliability through voting mechanisms. However, this research reveals a critical inflection point: as foundational models become more capable, the marginal value of additional sampling approaches zero while computational costs remain constant or increase.

The empirical findings are striking. On HotpotQA, sampling 20 reasoning paths yields only 0.4% accuracy improvement, translating to negligible gains per additional sample. MATH-500 performs slightly better at 1.6% improvement, but both benchmarks show performance plateaus and occasional degradation at high sample counts, indicating that weaker reasoning paths introduce noise rather than useful diversity in already-capable models.

This has profound implications for AI inference economics. As model providers scale up compute and charge by token consumption, indiscriminate self-consistency becomes increasingly difficult to justify for production systems. The research suggests a more targeted approach: reserve multi-path sampling exclusively for problem classes where single-pass reliability demonstrably falls below acceptable thresholds. This optimization could significantly reduce inference costs across the industry, benefiting cost-conscious enterprises and API providers alike.

Looking forward, this finding may drive development toward more sophisticated routing mechanisms—systems that dynamically select sampling strategies based on problem difficulty and model confidence scores. As competition intensifies in the LLM market, efficiency gains become competitive advantages, making this research particularly relevant for organizations optimizing their inference pipelines.

• →Self-consistency sampling delivers only 0.4% accuracy improvement on HotpotQA despite 5-20x higher token costs, indicating diminishing returns.
• →Performance plateaus and sometimes declines at high sample counts, suggesting additional reasoning paths introduce noise in already-reliable models.
• →The technique's efficiency depends on model capability level—it remains useful for problems exceeding single-pass reliability but wastes resources on problems models already solve consistently.
• →Selective multi-path sampling could significantly reduce enterprise inference costs and improve LLM provider margins.
• →Future approaches should use confidence-based routing to dynamically determine when self-consistency sampling provides genuine value.