Search, Do not Guess: Teaching Small Language Models to Be Effective Search Agents

Researchers developed a new training approach that makes small language models more effective search agents by teaching them to consistently use search tools rather than relying on internal knowledge. The method achieved significant performance improvements of 17.3 points on Bamboogle and 15.3 points on HotpotQA, reaching large language model-level results while maintaining lower computational costs.

• →Small language models are less likely to use search tools and more prone to hallucinations compared to large language models when acting as search agents.
• →A new lightweight fine-tuning approach teaches small models to reliably retrieve and generate evidence-grounded answers.
• →The method achieves 17.3 point improvement on Bamboogle and 15.3 point improvement on HotpotQA benchmarks.
• →Consistent search behavior is more effective than adaptive search strategies for small language models.
• →The approach enables small models to match large language model performance while maintaining computational efficiency.