Confusion-Aware Rubric Optimization for LLM-based Automated Grading

Researchers introduce CARO (Confusion-Aware Rubric Optimization), a new framework that improves LLM-based automated grading by using confusion matrices to separate and fix specific error patterns instead of aggregating all errors together. This approach prevents conflicting constraints and significantly outperforms existing methods in teacher education and STEM datasets.

• →CARO uses confusion matrices to decompose error signals into distinct modes for targeted repair of specific misclassification patterns.
• →The framework prevents 'rule dilution' where conflicting constraints weaken grading logic in traditional methods.
• →CARO eliminates resource-heavy nested refinement loops through diversity-aware selection mechanisms.
• →Empirical evaluations show CARO significantly outperforms existing state-of-the-art automated grading methods.
• →The surgical, mode-specific repair approach improves both scalability and precision in automated assessment systems.