Cross-lingual Retrieval-Augmented Classification for Dysarthria Severity Assessment

Researchers propose Cross-lingual Retrieval-Augmented Classification (CRAC), an AI method that improves dysarthria severity assessment by leveraging speech data from different languages to overcome the scarcity of labeled pathological speech datasets. The approach achieves significant accuracy improvements on Korean and Italian datasets, demonstrating the potential of cross-lingual transfer learning in medical speech analysis.

This research addresses a critical challenge in medical AI: the shortage of labeled training data for pathological speech conditions. Dysarthria severity assessment traditionally requires extensive labeled datasets in each target language, a resource that remains scarce and expensive to develop. CRAC solves this by innovating an align-retrieve-fuse architecture that allows models to borrow knowledge from opposite-language corpora, effectively multiplying available training resources without requiring additional annotation efforts.

The methodology combines supervised contrastive learning with retrieval-augmented classification, a pattern increasingly adopted across machine learning domains. By first creating a severity-focused embedding space, the system learns language-agnostic features that capture dysarthria patterns. During inference, it retrieves semantically similar references from a vector database built from speech in a different language, then fuses these retrieved examples through cross-attention mechanisms. This design elegantly transfers knowledge across linguistic boundaries while maintaining clinical relevance.

The performance improvements are substantial: 8.4 and 20.0 percentage point gains on Korean and Italian datasets respectively compared to monolingual baselines. The 20-point improvement on Italian particularly demonstrates CRAC's value when training data is extremely limited. This breakthrough has immediate implications for speech pathology applications, low-resource language medical AI, and telehealth platforms operating across multilingual populations.

Future development should explore CRAC's generalization to other speech disorders beyond dysarthria and its applicability to languages with minimal pathological speech resources. Integration into clinical assessment tools could accelerate screening for stroke and neurodegenerative disease populations worldwide.

• →CRAC achieves 87.3% and 86.7% balanced accuracy on Korean and Italian dysarthria datasets, substantially outperforming monolingual baselines
• →Cross-lingual retrieval-augmented learning effectively overcomes limited labeled pathological speech data through knowledge transfer across languages
• →The method combines contrastive learning and vector databases to create severity-focused embeddings that generalize across linguistic boundaries
• →20-point accuracy improvement on Italian demonstrates exceptional value for low-resource language medical AI applications
• →Approach enables practical deployment of dysarthria assessment tools in multilingual healthcare settings without extensive language-specific annotation