How Far Can Disaggregation Go? A Design-Space Exploration of Attention-FFN Disaggregation for Efficient MoE LLM Serving

Researchers present a systematic study of Attention-FFN Disaggregation (AFD), a technique that separates attention and expert layers across different GPU groups to optimize inference serving for Mixture-of-Experts language models. The framework demonstrates that AFD enables 4k tokens/s throughput on DeepSeek-V3.2 under strict latency constraints where traditional disaggregation approaches fail, providing design principles for scaling LLM infrastructure.

The research addresses a critical bottleneck in modern LLM serving: efficiently balancing diverse computational demands within massive models. As language models grow exponentially, inference systems have progressively disaggregated workloads—from basic prefill-decode separation to operator-level splitting. This work systematizes the design space for Attention-FFN Disaggregation specifically within Mixture-of-Experts architectures, where different components have fundamentally different resource requirements.

The motivation is rooted in hardware efficiency constraints. Attention operations are memory-bound and latency-sensitive, while expert FFNs are compute-intensive; placing them on separate GPU clusters allows hardware specialization and reduces resource contention. The research moves beyond theoretical optimization by incorporating real kernel measurements and network simulation, validating that AFD achieves meaningful throughput gains—4k tokens/s—under strict Time-To-First-Token and Time-Per-Output-Token SLOs that conventional approaches cannot sustain.

For infrastructure teams and AI service providers, this work offers concrete guidance on GPU partitioning strategies tailored to specific workloads and model architectures. Organizations deploying DeepSeek-V3.2 or similar MoE models at scale can use these design principles to optimize rack and cluster configurations. The research clarifies when disaggregation overhead becomes worthwhile versus when it introduces unnecessary complexity, preventing over-engineering. This directly impacts operational costs and service reliability for companies running production LLM inference systems.

Looking forward, the framework provides a foundation for future disaggregated AI infrastructure design, potentially influencing hardware architectures and software scheduling frameworks as models continue growing in complexity.

• →AFD enables 4k tokens/s throughput on DeepSeek-V3.2 under strict latency SLOs where non-disaggregated approaches become infeasible
• →Systematic characterization of disaggregation trade-offs across input/output sequence lengths, prefix-KV reuse, and latency constraints provides data-driven design guidance
• →Separating attention and FFN operations across GPU groups reduces resource contention by exploiting their heterogeneous computational demands
• →Design principles for GPU partitioning vary significantly based on workload characteristics and model architecture, requiring careful optimization per deployment
• →Framework combining on-device kernel measurements with network simulation enables accurate prediction of system performance at scale