MobileMoE: Scaling On-Device Mixture of Experts

Researchers present MobileMoE, a family of sub-billion parameter Mixture-of-Experts language models optimized for on-device deployment that achieve 2-4x efficiency gains over dense models while matching or exceeding performance. The work establishes new on-device scaling laws and delivers the first practical MoE inference implementation on smartphones, with 1.8-3.8x faster performance than existing mobile baselines.

MobileMoE represents a significant advancement in making large language models practical for resource-constrained mobile devices. The research challenges the prevailing assumption that Mixture-of-Experts architectures only benefit models at massive scales, revealing that moderate sparsity with fine-grained experts creates an efficiency sweet spot for mobile deployment. This discovery carries important implications for the accessibility of AI capabilities on personal devices.

The breakthrough stems from systematic investigation of how MoE architecture scales under mobile constraints, an area that received minimal attention despite rapid growth in on-device AI applications. Traditional dense models dominate mobile deployment due to inference optimization maturity, but MobileMoE's efficiency gains—requiring 60% fewer parameters than comparable MoE baselines—fundamentally alter that calculus. The four-stage training recipe using open-source datasets ensures reproducibility and lowers barriers to adoption.

The practical impact extends beyond academic benchmarks. Demonstrating efficient MoE inference on commodity smartphones with comprehensive profiling addresses the critical gap between theoretical improvements and real-world deployment. Mobile developers and device manufacturers now have viable options for deploying more capable models without proportional increases in computational overhead. The 2-4x FLOP reduction directly translates to lower latency, reduced power consumption, and improved user experience on mainstream hardware.

Industry observers should monitor adoption patterns across mobile platforms and whether competitive pressures push existing solutions toward MoE architectures. The combination of architectural innovation, rigorous scaling analysis, and demonstrated on-device implementation suggests this work will influence next-generation mobile AI development and potentially reshape resource allocation in edge computing infrastructure.

• →MobileMoE models with 0.3-0.9B active parameters achieve 2-4x inference efficiency gains over dense baselines while matching or exceeding performance across 14 benchmarks.
• →New on-device MoE scaling laws identify moderate sparsity with fine-grained experts as simultaneously memory and compute-optimal for mobile constraints.
• →First practical MoE inference implementation on smartphones delivers 1.8-3.8x faster prefill and 2.2-3.4x faster decode compared to dense mobile baselines.
• →Models match state-of-the-art MoE performance while using up to 60% fewer total parameters, significantly reducing deployment requirements.
• →Training uses open-source datasets across four stages including pre-training, mid-training, fine-tuning, and quantization-aware training for reproducibility.