Improving MPI Error Detection and Repair with Large Language Models and Bug References

Researchers developed enhanced techniques using Few-Shot Learning, Chain-of-Thought reasoning, and Retrieval Augmented Generation to improve large language models' ability to detect and repair errors in MPI programs. The approach increased error detection accuracy from 44% to 77% compared to using ChatGPT directly, addressing challenges in maintaining high-performance computing applications used in machine learning frameworks.

• →Direct application of large language models to MPI error detection yields suboptimal results due to lack of specialized knowledge.
• →Enhanced techniques combining FSL, CoT reasoning, and RAG improved error detection accuracy from 44% to 77%.
• →The bug referencing technique generalizes well across different large language models.
• →MPI programs are widely used in machine learning frameworks like PyTorch and TensorFlow for distributed training.
• →The research addresses critical challenges in maintaining complex high-performance computing applications.