CuTeGen: An LLM-Based Agentic Framework for Generation and Optimization of High-Performance GPU Kernels using CuTe

CuTeGen is an AI-powered framework that automates GPU kernel generation and optimization using large language models and the CuTe abstraction layer. The system achieves 1.71× average speedup over PyTorch on standardized benchmarks by employing a generate-test-refine workflow with delayed performance profiling, significantly outperforming prior agentic approaches.

CuTeGen addresses a fundamental challenge in machine learning infrastructure: the manual, expertise-intensive process of GPU kernel development. The framework leverages large language models within an agentic architecture to automate what has traditionally required specialized knowledge in CUDA programming and performance optimization. By targeting CuTe—a higher-level abstraction over raw CUDA—the system maintains stability during iterative refinement while exposing critical performance structures like tiling and data movement patterns.

The delayed profiling approach represents a key architectural insight. Rather than providing constant low-level performance feedback during generation, the system allows high-level kernel structure to stabilize first. This prevents premature optimization decisions that could derail the refinement process. On KernelBench's 209 tasks, CuTeGen demonstrates tangible improvements: 1.71× speedup versus PyTorch and outperformance of the prior CudaForge baseline, all while maintaining comparable computational costs per task.

This development carries significant implications for the AI and systems infrastructure landscape. Automating kernel generation could democratize high-performance computing by reducing the skill barrier for optimization. Organizations building ML systems could decrease time-to-deployment while improving computational efficiency—a critical concern as model sizes and inference demands scale. The research validates that LLM-based code generation, when properly constrained to suitable abstraction layers, can produce production-competitive results.

The work establishes a blueprint for future AI-driven systems optimization. Practitioners should monitor whether similar agentic frameworks emerge for other infrastructure bottlenecks, and whether commercial systems begin adopting LLM-based kernel synthesis in production environments.

• →CuTeGen achieves 1.71× speedup over PyTorch using LLM-based GPU kernel generation targeting CuTe abstraction
• →Delayed profiling strategy prevents premature optimization and stabilizes high-level kernel structure during refinement cycles
• →Framework outperforms prior agentic baseline CudaForge while maintaining comparable per-task generation costs
• →CuTe abstraction layer balances performance visibility with iterative stability, contrasting with raw CUDA approaches
• →Automated kernel synthesis could reduce expertise barriers in GPU optimization and accelerate ML infrastructure deployment