Evaluating adaptive and generative AI-based feedback and recommendations in a knowledge-graph-integrated programming learning system

Researchers developed a framework integrating large language models with knowledge graphs to provide programming feedback and exercise recommendations. The hybrid GenAI-adaptive approach outperformed traditional adaptive learning and GenAI-only modes, producing more correct code submissions and fewer incomplete attempts across 4,956 code submissions.

• →Hybrid GenAI-adaptive learning systems achieved the highest number of correct programming submissions compared to adaptive-only or GenAI-only approaches.
• →Students receiving GenAI-generated feedback produced significantly more correct code and fewer submissions missing essential programming logic.
• →The framework successfully integrated large language models with retrieval-augmented generation using knowledge graphs and user interaction history.
• →Learners perceived GenAI-generated feedback as helpful while rating all modes positively for ease of use and usefulness.
• →The study analyzed 4,956 code submissions to demonstrate the effectiveness of combining generative AI with traditional adaptive learning methods.