Individual and Combined Effects of English as a Second Language and Typos on LLM Performance

Research reveals that Large Language Models (LLMs) experience greater performance degradation when facing English as a Second Language (ESL) inputs combined with typographical errors, compared to either factor alone. The study tested eight ESL variants with three levels of typos, finding that evaluations on clean English may overestimate real-world model performance.

• →LLMs trained primarily on English data perform worse on ESL inputs, which are common among global users.
• →Combining ESL variation with typographical errors causes more performance drops than either factor individually.
• →The combined negative effect is not simply additive, suggesting complex interactions between language variants and errors.
• →Closed-ended tasks show more consistent performance degradation patterns than open-ended tasks.
• →Standard English evaluations may significantly overestimate how AI models perform in real-world scenarios.