AutoPass: Evidence-Guided LLM Agents for Compiler Performance Tuning

AutoPass is a multi-agent LLM framework that automatically tunes compiler performance by analyzing internal compiler states and runtime feedback, achieving 4.3% speedups on x86-64 and 11.7% on ARM64 compared to LLVM's standard optimization levels without requiring task-specific training.

AutoPass represents a significant advancement in applying large language models to systems-level optimization challenges that traditionally required deep expertise. The framework addresses a fundamental limitation of prior LLM-based approaches by treating the compiler as an open system rather than a black box, allowing the model to inspect intermediate representations and query compiler optimization states. This architectural choice enables more informed decision-making when selecting compiler flags and configurations.

The research builds on growing interest in using LLMs for code optimization, but tackles the notoriously difficult problem of runtime performance tuning. Compiler optimization has historically resisted automation because microarchitectural effects are complex and runtime measurements are inherently noisy. AutoPass handles this through an iterative refinement process that uses actual measured performance feedback to diagnose performance regressions and guide subsequent optimization attempts.

The practical impact extends across multiple domains. Developers working with embedded systems and server infrastructure could benefit from 10%+ performance improvements without manual tuning expertise. The training-free, inference-only approach makes AutoPass immediately applicable to new platforms and benchmarks, reducing deployment friction compared to models requiring fine-tuning. This democratizes access to performance optimization capabilities previously limited to compiler engineers and systems specialists.

The validation on both x86-64 server systems and ARM64 embedded platforms demonstrates broad applicability. As computational costs remain a concern for organizations running large-scale infrastructure, even modest percentage improvements compound to significant cost savings. Future developments might extend this approach to other compiler toolchains beyond LLVM or address memory optimization alongside latency improvements.

• →AutoPass achieves 11.7% speedup on ARM64 and 4.3% on x86-64 without offline training or task-specific fine-tuning
• →The framework enables LLMs to inspect compiler internals and intermediate representations for more informed optimization decisions
• →Iterative refinement using actual runtime feedback helps diagnose performance regressions and guide latency improvements
• →Training-free operation makes the system readily applicable to new benchmarks and hardware platforms
• →Results outperform both expert-tuned heuristics and classical autotuning methods across server and embedded systems