UniScale: Adaptive Unified Inference Scaling via Online Joint Optimization of Model Routing and Test-Time Scaling

UniScale introduces a unified framework that combines model routing and test-time scaling to optimize large language model inference, balancing quality and computational cost. The system uses online learning via contextual multi-armed bandits to adapt inference policies dynamically, achieving fine-grained performance improvements over existing decoupled approaches.

UniScale addresses a fundamental efficiency problem in LLM deployment: the suboptimal separation between model routing (switching between different-sized models) and test-time scaling (adjusting compute within a single model). Current systems treat these mechanisms independently, resulting in coarse-grained performance adjustments and diminishing returns. By unifying both dimensions into a single optimization space, UniScale enables more granular control over the quality-cost trade-off.

The technical innovation lies in framing this as a contextual multi-armed bandit problem solved via LinUCB, an online learning algorithm. This approach allows the system to learn optimal inference policies without requiring extensive offline training, making it suitable for dynamic, real-world deployment scenarios where request patterns fluctuate. The incorporation of efficiency-aware learning and cost modeling ensures the framework scales across high-dimensional action spaces without computational overhead.

For infrastructure operators and LLM providers, this has significant implications. Reduced inference costs directly improve profit margins while maintaining quality standards. Organizations running multiple model scales can now optimize routing decisions more precisely, potentially reducing unnecessary model switches and latency spikes. The adaptive nature suits cloud environments where demand varies across time periods and request complexities.

The framework's practical value depends on deployment validation at scale. Organizations should monitor whether the bandit-learning approach converges quickly enough for production environments and whether the cost savings justify implementation complexity. Broader adoption would signal industry movement toward more sophisticated inference optimization, potentially influencing future model architecture designs and deployment strategies.

• →UniScale unifies model routing and test-time scaling into a single optimization framework for more efficient LLM inference.
• →The system uses contextual multi-armed bandit learning to adaptively adjust inference policies without offline training.
• →Fine-grained quality-cost trade-offs improve upon existing discrete model-routing approaches with diminishing returns.
• →Online learning capability enables dynamic adaptation to varying request patterns in production environments.
• →Cost modeling integration ensures scalability across complex action spaces in real-world deployment scenarios.