Small Language Models for Efficient Agentic Tool Calling: Outperforming Large Models with Targeted Fine-tuning

Researchers demonstrated that a fine-tuned small language model (SLM) with 350M parameters can significantly outperform large language models like ChatGPT in tool-calling tasks, achieving a 77.55% pass rate versus ChatGPT's 26%. This breakthrough suggests organizations can reduce AI operational costs while maintaining or improving performance through targeted fine-tuning of smaller models.

• →A fine-tuned 350M parameter model achieved 77.55% pass rate on ToolBench evaluation, vastly outperforming ChatGPT-CoT at 26%.
• →Small language models can deliver comparable or superior performance to large models in targeted applications while drastically reducing infrastructure costs.
• →The research used Meta's OPT-350M model with Hugging Face TRL supervised fine-tuning for just a single epoch.
• →Results demonstrate that thoughtful design and targeted training can make enterprise AI adoption more cost-effective and accessible.
• →The findings challenge the assumption that larger models are always better for specialized enterprise tasks.