"That's AI Slop, You Bot!" Studying Accusations, Evidence, and Credibility in Online Discourse Towards LLM-Generated Comments
A large-scale study of 25 million comments from Hacker News and Reddit reveals that accusations of AI-generated content have surged over 1000% since 2023, yet these accusations rarely correlate with actual linguistic markers of AI writing. The research shows that "AI slop" accusations function primarily as social gatekeeping rather than genuine detection, challenging assumptions about how AI impacts online discourse.
The study exposes a critical gap between perception and reality in how online communities respond to generative AI. Researchers analyzed millions of comments to track how users accuse others of posting AI-generated content, finding that pejorative labels mentioning AI grew tenfold while older inautenticity terms like "shill" remained flat. This suggests a genuine cultural shift, not merely vocabulary recycling. The accusatory tone evolved from mockery toward defensive gatekeeping, indicating communities are using AI accusations as boundary-setting mechanisms. The most striking finding undermines the premise of effective AI detection: prose features that statistically distinguish AI from human writing do not predict which human comments get accused as AI. This disconnect reveals that social signaling—not accurate detection—drives accusations. The research extends signaling theory by demonstrating that inaccurate signals can proliferate if the underlying problem remains unsolvable for non-experts. For the AI and content moderation industries, this creates a persistent challenge: technical solutions cannot resolve fundamentally social dynamics. Communities are not deploying accusations as fact-based identification tools but as expressions of cultural anxiety about authenticity and belonging. This means that even as AI detection improves technically, accusations may continue rising independently. The findings suggest that AI's impact on online discourse operates differently than production-side effects, requiring social rather than purely technical interventions.
- →AI slop accusations increased over 1000% on major platforms while actual linguistic markers of AI writing don't predict which texts get accused
- →Users employ AI accusations primarily for social gatekeeping and in-group signaling rather than accurate detection of machine-generated content
- →The phenomenon reflects cultural anxiety about authenticity that cannot be resolved through improved detection technology alone
- →Accusatory tone shifted from mockery toward structural protest, indicating communities are establishing boundaries around perceived authenticity
- →Substitute signals for inauthenticity can spread virally even when inaccurate if the underlying detection problem resists non-expert solutions