Skill-Based Mixture-of-Experts: Adaptive Routing for Heterogeneous Reasoning via Inferred Skills

Researchers introduce Skill-MoE, a framework that improves AI reasoning by routing individual queries to specialized expert models based on inferred skills rather than broad task categories. The approach achieves 8.15% average improvement across multiple benchmarks while maintaining computational efficiency through intelligent batch processing.

Skill-MoE represents an important advancement in how multiple language models can be combined for complex reasoning tasks. Traditional mixture-of-experts approaches select a single expert per task, which fails to account for the reality that different problem instances within the same task category often require distinct expertise. This paper demonstrates that instance-level routing—analyzing each query individually to determine which specialized models should handle it—produces substantially better results than coarser-grained selection methods.

The innovation directly addresses a critical bottleneck in ensemble AI systems: computational overhead. Previous attempts at fine-grained expert selection required repeatedly loading and unloading models, creating prohibitive latency and resource consumption. Skill-MoE solves this through batch inference, grouping similar instances together so each expert loads only once. This efficiency gain is substantial—the system integrates 16 expert models on a single GPU with performance comparable to 4-GPU baseline systems.

The technical approach combines skill inference (identifying what capabilities a query demands) with expert selection and response aggregation. This three-stage pipeline generalizes effectively to unseen tasks and outperforms discussion-based multi-agent methods that require expensive iterative interactions. Performance across diverse benchmarks—MMLU-Pro, GPQA, AIME, and MedMCQA—demonstrates broad applicability rather than domain-specific optimization.

For the AI infrastructure and model serving industry, this work validates instance-level routing as a practical scaling strategy. Organizations deploying multiple specialized models can now achieve better performance with lower computational cost. The gradient-free, symbolic approach also suggests this framework integrates readily with existing model architectures without requiring retraining.

• →Instance-level expert routing achieves 8.15% average performance improvement over task-level selection methods
• →Batch inference strategy enables 16 expert models on a single GPU, reducing computational overhead versus prior approaches
• →The framework infers required skills from queries to determine which specialized models handle each instance
• →Skill-MoE generalizes to unseen tasks without expensive multi-round interactions like discussion-based methods require
• →Symbolic, gradient-free design allows seamless integration with existing pre-trained language models