Culturally-Adapted Red-Teaming Across East and Southeast Asian Contexts: A Methodological and Comparative Analysis

Researchers demonstrate that direct translation of English LLM safety benchmarks into Asian languages significantly underestimates risks, with culturally-adapted prompts showing 9.3 percentage points higher attack success rates on average. The study reveals that translation-only approaches fail to capture cultural context, legal frameworks, and social norms critical for valid multilingual AI safety evaluation.

This research addresses a critical gap in AI safety evaluation that has grown alongside the rapid global deployment of large language models. While LLM safety testing has matured in English contexts, the multilingual evaluation landscape has relied on a problematic shortcut: translating English benchmarks directly into target languages. This approach treats language as a mere container for meaning while ignoring the cultural substrate that shapes threat models, acceptable behavior, and legal consequences across different societies.

The methodology reveals systematic underestimation of risk when using direct translation. Across 16 language-model combinations tested with Korean, Japanese, Thai, and Khmer datasets, culturally-adapted prompts consistently outperformed direct translations by over 9 percentage points in attack success rate. More strikingly, direct translation cultural realism scores averaged just 0.17 out of 3.0, while culturally-adapted versions reached 2.51, indicating that translation produces evaluations fundamentally misaligned with real-world scenarios users actually encounter.

For AI developers and safety researchers, this finding carries immediate practical consequences. Models currently approved as safe through English benchmarks may harbor serious vulnerabilities when deployed across Asian markets representing billions of potential users. The heterogeneous distribution of threat forms across languages suggests that one-size-fits-all safety protocols will fail. Organizations building multilingual AI systems cannot rely on translation pipelines for safety validation—they require localized threat modeling that accounts for regional legal frameworks, cultural sensitivities, and social norms.

Looking forward, this research establishes a methodological template for culturally-grounded safety evaluation that will likely become standard practice. The economic implication is significant: companies investing in proper localized safety evaluation gain competitive advantage in emerging markets while reducing deployment risks. Regulatory bodies may increasingly demand evidence of culturally-adapted safety testing before market approval.

• →Direct translation of English LLM safety benchmarks underestimates risk by an average of 9.3 percentage points across Asian language contexts
• →Cultural realism scores for translated prompts average just 0.17 out of 3.0, revealing systematic divergence from real-world multicultural scenarios
• →Threat distribution across languages is heterogeneous, requiring language-specific rather than universal safety evaluation approaches
• →Current multilingual LLM deployment may contain undetected vulnerabilities in non-English contexts due to inadequate safety testing methodology
• →Culturally-adapted safety benchmarks represent an emerging requirement for responsible AI deployment in global markets