Evaluation of Large Language Models via Coupled Token Generation

Researchers propose a new method called coupled autoregressive generation to evaluate large language models more efficiently by controlling for randomness in their responses. The study shows this approach can reduce evaluation samples by up to 75% while revealing that current model rankings may be confounded by inherent randomness in generation processes.