A Finetuned SpeechLLM for Joint Multi-Granular L2 Assessment and Natural-Language Rationales

Researchers propose a fine-tuned speech language model that provides both multi-level L2 English proficiency assessment and natural-language explanations for its predictions. The model demonstrates competitive performance on standard benchmarks while offering improved interpretability, though generated rationales show lower reliability at granular word-level assessments.

This research addresses a critical gap in automated language assessment systems: the need for both accuracy and explainability. Traditional L2 proficiency evaluation relies on human raters, creating bottlenecks for global English learners seeking feedback. The proposed SpeechLLM leverages recent advances in large language models adapted for speech input, combining supervised fine-tuning with Bounded Direct Preference Optimization to balance performance across multiple prediction tasks simultaneously.

The multi-granular assessment approach represents a meaningful evolution in EdTech applications. By generating rationales alongside predictions, the system helps learners understand specific areas for improvement—sentence-level fluency, word-level accuracy, prosodic patterns—rather than receiving opaque scores. This transparency builds trust and provides actionable feedback that supports learning outcomes.

The research reveals important limitations: while sentence-level rationales remain plausible and consistent, word-level explanations lack grounding in the underlying linguistic references. This suggests that while the model's high-level assessments are reliable, fine-grained feedback may require architectural improvements or additional training signal. The gap between prediction accuracy and explanation faithfulness highlights a persistent challenge in deploying interpretable AI systems in educational contexts.

Future work should focus on tightening the alignment between token-level predictions and generated explanations, potentially through improved training objectives or additional supervision at the word-phoneme level. Such refinements could position automated L2 assessment as a credible complement to human evaluation, particularly for large-scale language proficiency screening where both speed and interpretability matter.

• →Multi-granular assessment approach provides sentence, word, and phoneme-level proficiency predictions in a single unified model
• →Generated natural-language rationales improve interpretability but show lower faithfulness at word-level granularity
• →Model matches or exceeds performance of single-task baselines while maintaining competitive results on SpeechOcean762 benchmark
• →Hybrid training combining supervised fine-tuning and preference optimization enables joint optimization across multiple assessment objectives
• →Research identifies specific reliability gaps that constrain deployment of fine-grained linguistic feedback in educational applications