Architecture-Aware LLM Inference Optimization on AMD Instinct GPUs: A Comprehensive Benchmark and Deployment Study

Researchers conducted comprehensive benchmarks of LLM inference on AMD Instinct MI325X GPUs, testing models from 235B to 1 trillion parameters. The study reveals that architecture-aware optimization is critical, with different model types requiring specific configurations for optimal performance on AMD hardware.

• →Architecture-aware optimization is essential for LLM inference, with MLA models requiring block size 1 while GQA models benefit from KV cache offloading.
• →AMD AITER runtime is necessary for competitive MLA inference throughput but must be selectively disabled for incompatible attention configurations.
• →Llama-405B and DeepSeek V3.2 achieved comparable peak throughput despite order-of-magnitude differences in active parameters.
• →All tested models exhibited throughput saturation at similar concurrent user levels, indicating memory-bandwidth bottlenecks.
• →The benchmark processed 18.9 million tokens across 17,406 requests with 100% HTTP-level success rates through 1,000 concurrent users.