LLMServingSim 2.0: A Unified Simulator for Heterogeneous and Disaggregated LLM Serving Infrastructure

Researchers have released LLMServingSim 2.0, a unified simulator that models the complex interactions between heterogeneous hardware and disaggregated software in large language model serving infrastructures. The simulator achieves 0.97% average error compared to real deployments while maintaining 10-minute simulation times for complex configurations.

• →LLMServingSim 2.0 provides the first unified framework to simulate hardware-software interactions in modern heterogeneous LLM serving infrastructures.
• →The simulator captures dynamic serving behaviors including batching, routing, offloading, memory management, and power consumption in a single runtime loop.
• →Validation against real deployments shows highly accurate performance metrics with less than 1% average error.
• →The tool enables systematic exploration and co-design optimization for next-generation LLM serving systems.
• →Fast simulation times of around 10 minutes make it practical for complex configuration testing and hardware-software co-design.