AIBullisharXiv – CS AI · 7h ago7/10
🧠
Piper: Efficient Large-Scale MoE Training via Resource Modeling and Pipelined Hybrid Parallelism
Researchers introduce Piper, a framework for efficiently training Mixture-of-Experts (MoE) models on high-performance computing platforms through resource modeling and optimized pipeline parallelism. The approach achieves 2-3.5X higher computational efficiency than existing frameworks and introduces a novel all-to-all communication algorithm that delivers 1.2-9X bandwidth improvements over vendor implementations.