When Surface Form Changes Moderation Decisions: A Paired Study of Code-Mixed Workflow Instability

Researchers found that content moderation systems trained on clean English perform significantly worse when processing code-mixed inputs (mixing English and Tamil), causing a 26.5% decision flip rate between allowing and flagging identical content. The study reveals workflow-level failures in moderation systems, including increased false positives on non-hateful content and higher review burdens, issues missed by standard classification metrics.

Content moderation systems deployed at scale face a critical blind spot: they are typically evaluated on clean, single-language inputs but must handle real-world linguistic diversity. This research exposes how surface-level language mixing destabilizes moderation workflows, with code-mixed variants of identical hateful content receiving different routing decisions than their English counterparts. The paired evaluation methodology is particularly valuable because it isolates the effect of linguistic form from content intent, revealing that systems trained on dominant languages systematically fail minority language variants.

The instability manifests across multiple failure modes. Review burden nearly doubles from 13.8% to 29.7%, indicating systems become overly cautious with unfamiliar linguistic patterns. More problematically, non-hateful content faces higher false-flag rates (6.9% to 10.4%), creating collateral damage for users speaking code-mixed languages. Tamil-only inputs show even worse degradation, suggesting the core issue extends beyond code-mixing to insufficient coverage of non-English languages in training data.

For AI systems developers and platform operators, this research highlights a gap between classification-focused evaluation and deployment reality. Moderation workflows involve sequential decision gates, and instability at any stage compounds downstream. The proposed disagreement-based deferral rule reduces errors but only by deferring decisions to human review—a costly mitigation that doesn't solve the underlying representation problem.

This work signals growing recognition that AI fairness requires evaluating systems at the workflow level, not just classification accuracy. As platforms expand globally, linguistic coverage becomes a core safety requirement, not an afterthought.

• →Code-mixed inputs cause 26.5% decision flip rates in hate speech moderation, revealing system instability across linguistic variants.
• →Review rates increase 115% and false-positive rates on non-hateful content jump 50% when processing Tamil-English code-mix.
• →Standard classification metrics miss workflow-level failures that emerge only under real-world linguistic diversity.
• →Language-specific training limitations affect not just code-mixed inputs but all non-English languages more broadly.
• →Simple deferral to human review mitigates errors but increases operational cost rather than improving model robustness.