AINeutralarXiv – CS AI · 6h ago6/10
🧠
Is Escalation Worth It? A Decision-Theoretic Characterization of LLM Cascades
Researchers develop a decision-theoretic framework for optimizing LLM cascades, where cheaper models defer to expensive ones on low-confidence queries. Testing across five benchmarks reveals that cascade performance is fundamentally limited by structural costs rather than routing sophistication, with simpler router-based approaches often outperforming optimized cascade policies.