AINeutralarXiv – CS AI · May 296/10
🧠
A Minimal Bifurcation Model of Load Imbalance in a Softmax Mixture-of-Experts Router
Researchers propose a mathematical model explaining how Mixture-of-Experts (MoE) neural networks can suddenly shift from balanced to imbalanced expert utilization. The model reveals a bifurcation mechanism where increased feedback strength triggers abrupt transitions between stable states, providing theoretical insight into a practical problem affecting large language models and distributed AI systems.