Compute Allocation for Reasoning-Intensive Retrieval Agents

Researchers studied computational resource allocation in AI retrieval systems for long-horizon agents, finding that re-ranking stages benefit more from powerful models and deeper candidate pools than query expansion stages. The study suggests concentrating compute power on re-ranking rather than distributing it uniformly across pipeline stages for better performance.

• →Re-ranking shows substantial improvements with stronger models (+7.5 NDCG@10) compared to minimal gains in query expansion (+1.1 NDCG@10).
• →Deeper candidate pools in re-ranking provide significant benefits (+21% improvement from k=10 to 100).
• →Query expansion reaches diminishing returns beyond lightweight models, making additional compute allocation inefficient.
• →Inference-time thinking provides minimal improvement at both pipeline stages.
• →Optimal compute allocation strategy concentrates resources on re-ranking rather than uniform distribution across stages.