PassNet: Scaling Large Language Models for Graph Compiler Pass Generation

PassNet introduces the first large-scale ecosystem for using large language models to generate compiler passes—structured graph transformations that optimize tensor compiler performance. The framework includes 18K computational graphs and 200 curated benchmark tasks, revealing that while LLMs lag frontier models by 37% on average, they achieve up to 3x speedups on individual workloads, indicating consistency rather than capability is the limiting factor.

PassNet addresses a critical pain point in modern compiler optimization: 43% of real-world computational subgraphs experience performance degradation under default TorchInductor compilation. Rather than pursuing kernel generation as previous LLM-compiler efforts have done, this work targets pass generation—where LLMs create reusable graph transformation rules that integrate directly into compiler pipelines. This abstraction shift is significant because compiler passes operate at a higher level of abstraction and have broader applicability across diverse workloads.

The research demonstrates that LLM-driven optimization faces a consistency problem, not a capability ceiling. Frontier models trail TorchInductor by 37% in aggregate performance, yet achieve 3x speedups on individual subgraphs. This paradox reveals that LLMs can generate highly optimized solutions for specific problems but struggle with systematic coverage and robustness. The public release of PassNet-Dataset (18K graphs from 100K models) and PassBench (200 curated tasks with integrity defenses) creates infrastructure for systematic improvement.

The fine-tuning experiments validate this framework as practical training infrastructure: a small model trained on merely 4K trajectories achieves 2.67x improvement, approaching frontier-model performance. This suggests the bottleneck is data and training, not architectural limitations. For the AI infrastructure ecosystem, PassNet represents a scalable path toward AI-assisted compiler optimization that could significantly reduce latency on long-tail workloads affecting real production systems. The public tooling release enables reproducible research and competitive benchmarking in an area previously dominated by proprietary compiler teams.

• →PassNet is the first large-scale LLM ecosystem for compiler pass generation, featuring 18K computational graphs and 200 benchmark tasks with rigorous integrity defenses.
• →LLMs achieve up to 3x speedups on individual subgraphs but lag 37% on aggregate, indicating consistency rather than capability is the limiting factor.
• →Fine-tuning on 4K training trajectories yields 2.67x improvement, demonstrating substantial headroom and validating PassNet as live training infrastructure.
• →43% of real-world subgraphs experience slowdowns under default TorchInductor compilation, creating a market opportunity for LLM-driven optimization.
• →All data, benchmarks, and tooling are publicly available, enabling reproducible research and competitive advancement in AI-driven compiler optimization.