Optimizing In-Context Demonstrations for LLM-based Automated Grading

Researchers introduce GUIDE, a new framework for improving automated grading of student responses using large language models. The system addresses key limitations in current LLM-based grading by optimizing the selection of training examples and generating better explanations for scoring decisions.

• →GUIDE framework reframes automated grading as a boundary-focused optimization problem to improve LLM performance.
• →The system uses contrastive operators to identify boundary pairs that are similar but have different grades.
• →Current retrieval methods based on semantic similarity often fail to capture subtle decision boundaries needed for accurate grading.
• →Experiments across physics, chemistry, and pedagogical datasets show significant improvements over standard baselines.
• →The approach shows particularly strong performance on borderline cases and improved adherence to grading rubrics.