Switchcraft: AI Model Router for Agentic Tool Calling

Switchcraft is a new AI model router specifically designed for agentic tool calling that selects the lowest-cost model while maintaining correctness. The system achieves 82.9% accuracy matching top models while reducing inference costs by 84%, demonstrating that larger models don't consistently outperform smaller ones on function-calling tasks.

Switchcraft addresses a critical pain point in the AI infrastructure landscape: the economic inefficiency of defaulting to large language models for all inference tasks. Developers have traditionally chosen expensive flagship models as safe defaults, creating unnecessary cost bloat when smaller models could handle specific tasks equally well. This research reveals that tool-calling—a fundamental capability for agentic AI systems—doesn't always benefit from scale, upending common assumptions about model selection.

The broader context reflects growing maturity in the AI infrastructure stack. As agentic systems proliferate, the marginal cost of inference becomes a decisive factor in deployment viability. Previous routing approaches focused on chat completion, a different optimization problem with distinct efficiency patterns. Switchcraft fills this gap by training a DistilBERT-based classifier specifically calibrated for function-calling accuracy and latency constraints, enabling runtime decision-making at scale.

The market implications are substantial. At $3,600 saved per million queries, cost efficiency directly translates to margin improvement for AI application providers and accessibility gains for price-sensitive users. This creates competitive advantage for developers who implement intelligent routing versus those relying on default models. The finding that nominally cheaper models can incur higher total costs due to token-intensive reasoning patterns suggests that total cost of ownership metrics, not just per-token pricing, should drive procurement decisions.

Looking ahead, expect widespread adoption of routing layers across inference platforms as competition intensifies. The success of task-specific routers may fragment the market toward specialized models rather than consolidation around flagship systems, fundamentally reshaping how organizations approach model selection and budget allocation.

• →Switchcraft reduces AI inference costs by 84% while maintaining 82.9% accuracy on function-calling tasks, outperforming individual large models
• →Larger language models do not consistently outperform smaller ones on tool-use tasks, challenging assumptions about scale-driven performance
• →The router operates inline using a DistilBERT classifier, enabling real-time cost optimization without sacrificing latency budgets
• →Nominally cheaper models can incur higher total costs through token-intensive reasoning, requiring total cost of ownership analysis
• →Task-specific routing optimization indicates a shift toward specialized model selection rather than defaulting to flagship systems