AIBullisharXiv – CS AI · 18h ago7/10
🧠
More Bang for the Buck: Improving the Inference of Large Language Models at a Fixed Budget using Reset and Discard (ReD)
Researchers propose Reset-and-Discard (ReD), a novel querying method that improves large language model inference efficiency by optimizing the coverage@cost metric—the number of unique questions answered within a fixed budget. The technique reduces computational attempts, tokens, and financial costs needed to achieve desired performance levels across coding, math, and reasoning tasks.