Is Retraining-Free Enough? The Necessity of Router Calibration for Efficient MoE Compression

Researchers propose Router Knowledge Distillation (Router KD) to improve retraining-free compression of Mixture-of-Experts (MoE) models by calibrating routers while keeping expert parameters unchanged. The method addresses router-expert mismatch issues that cause performance degradation in compressed MoE models, showing particularly strong results in fine-grained MoE architectures.

• →MoE compression can be organized into three paradigms: Expert Pruning, Expert Editing, and Expert Merging.
• →Post-compression performance degradation mainly stems from router-expert mismatch when experts change but routers remain untouched.
• →Router Knowledge Distillation updates only router parameters (a tiny fraction) while keeping expert parameters unchanged.
• →The method shows consistent performance recovery across all three compression paradigms.
• →Fine-grained MoEs benefit more than coarse-grained MoEs due to their more complex routing decision boundaries.