Extracting Small Translation Specialists from LLMs by Aggressively Pruning Experts

Researchers present a method for aggressively pruning expert modules from mixture-of-experts large language models to create specialized translation systems. The approach removes up to 90% of experts with minimal performance degradation, demonstrating that translation tasks require only a fraction of a full LLM's parameters, enabling substantial model compression.

This research addresses a fundamental inefficiency in modern LLM deployment: using generalist models trained on diverse tasks for specialized applications like machine translation. The study reveals that mixture-of-experts architectures contain significant redundancy when applied to single-domain tasks, with researchers successfully removing half of all experts without noticeable quality loss and achieving 75% pruning after brief fine-tuning.

The modular design of MoE models enables this aggressive pruning without retraining from scratch, a critical advantage for computational efficiency. The finding that translation-specific experts can be identified and isolated reflects the emerging understanding that LLMs develop specialized internal structures despite their generalist training objectives. This has broader implications for model optimization beyond translation, suggesting similar compression techniques could apply to other specialized use cases.

For the AI industry, this research directly addresses deployment costs and accessibility barriers. Reducing parameter counts by 75-90% dramatically decreases memory requirements, inference latency, and energy consumption—making specialized translation systems viable for resource-constrained environments like mobile devices or edge computing. This efficiency gains align with industry pressure to make AI systems more practical and sustainable.

The work signals a shift toward task-specific optimization of foundation models rather than deploying monolithic architectures. As MoE models become standard infrastructure, systematic pruning methodologies will become increasingly valuable for enterprise deployments seeking cost-effective specialization without architectural redesign.

• →Researchers can prune up to 75% of MoE experts with full performance recovery using brief fine-tuning, and 90% while maintaining reasonable quality
• →Translation tasks exploit only a fraction of LLM parameters, enabling substantial compression of model weights
• →Modular MoE design permits pruning without retraining, reducing computational overhead of optimization
• →Expert specialization in multilingual LLMs can be systematically identified and removed for domain-specific applications
• →This approach has potential applications beyond translation to other specialized single-domain LLM deployments