From Prompts to Performance: Evaluating LLMs for Task-based Parallel Code Generation

Researchers evaluated Large Language Models' ability to generate parallel code across three programming frameworks (OpenMP, C++, HPX) using different input prompts. The study found LLMs show varying performance depending on problem complexity and framework, revealing both capabilities and limitations in high-performance computing applications.

• →LLMs demonstrate strong general code generation abilities but show mixed results for efficient parallel programming.
• →Three programming frameworks were tested: OpenMP Tasking, C++ standard parallelism, and HPX runtime system.
• →Performance varied significantly based on input prompt type: natural language, sequential code, or parallel pseudo code.
• →LLM-generated solutions were evaluated for both correctness and scalability in parallel execution.
• →Findings have implications for future AI-assisted development in scientific and high-performance computing.