AINeutralarXiv – CS AI · 10h ago6/10
🧠
SDG-MoE: Signed Debate Graph Mixture-of-Experts
Researchers introduce SDG-MoE, a novel mixture-of-experts architecture that enables deliberation among routed experts through signed graph communication before output aggregation. The model demonstrates 19.8% perplexity improvement over vanilla MoE and achieves state-of-the-art results on multiple language modeling benchmarks while maintaining computational efficiency.
🏢 Perplexity