Evolutionary Search for Automated Design of Uncertainty Quantification Methods

Researchers developed an LLM-powered evolutionary search method to automatically design uncertainty quantification systems for large language models, achieving up to 6.7% improvement in performance over manual designs. The study found that different AI models employ distinct evolutionary strategies, with some favoring complex linear estimators while others prefer simpler positional weighting approaches.

• →Automated evolutionary search outperformed manually-designed uncertainty quantification methods by up to 6.7% relative ROC-AUC improvement across 9 datasets.
• →Different LLMs showed distinct design preferences: Claude models favored high-feature-count linear estimators while Gpt-oss-120B preferred simpler positional weighting schemes.
• →Only Sonnet 4.5 and Opus 4.5 effectively leveraged increased method complexity to improve performance.
• →The evolved methods demonstrated robust generalization capabilities in out-of-distribution scenarios.
• →LLM-powered evolutionary search shows promise as a paradigm for automated hallucination detector design.