Latent Self-Consistency for Reliable Majority-Set Selection in Short- and Long-Answer Reasoning

Researchers introduce Latent Self-Consistency (LSC), a new method for improving Large Language Model output reliability across both short and long-form reasoning tasks. LSC uses learnable token embeddings to select semantically consistent responses with only 0.9% computational overhead, outperforming existing consistency methods like Self-Consistency and Universal Self-Consistency.

• →LSC addresses inconsistent LLM outputs by selecting semantically consistent responses using learnable token embeddings.
• →The method adds negligible runtime overhead (maximum 0.9%) and requires no changes to model architecture.
• →LSC outperforms existing consistency methods across 11 benchmarks including MATH, MMLU, and TruthfulQA.
• →The approach works effectively for both short-form and long-form reasoning tasks, unlike previous methods that lose accuracy in one format.
• →LSC provides well-calibrated confidence estimates with low expected calibration error across answer formats.