AINeutralarXiv – CS AI · 3h ago6/10
🧠
Cultural Benchmarking of LLMs in Standard and Dialectal Arabic Dialogues
Researchers introduce ArabCulture-Dialogue, a new dataset for evaluating large language models' cultural reasoning across 13 Arabic-speaking countries in both Modern Standard Arabic and regional dialects. Benchmarking reveals significant performance gaps, with LLMs consistently underperforming on dialectal Arabic compared to standardized variants, highlighting a critical blind spot in AI language model training.