AINeutralarXiv – CS AI · 9h ago6/10
🧠
Correcting Prompt Dependence in LLM Benchmarks: A Bayesian Hierarchical Model with Embedding-Space Clustering
Researchers propose a Bayesian hierarchical model with embedding-space clustering to correct fundamental flaws in LLM benchmarking methodology. The approach addresses two critical issues—insufficient evaluation samples and non-independent test prompts—improving performance metric accuracy by 4-73% in mean absolute errors, particularly relevant for adversarial robustness evaluation.