AIBullisharXiv – CS AI · 7h ago6/10
🧠
OrcaRouter: A Production-Oriented LLM Router with Hybrid Offline-Online Learning
OrcaRouter is a production-ready LLM routing system that uses contextual bandits and hybrid offline-online learning to intelligently direct requests to the most appropriate language model. The system ranked second on the RouterArena leaderboard with 75.54% accuracy while maintaining low inference costs of $1.00 per 1,000 queries.