Beyond Surface Judgments: Human-Grounded Risk Evaluation of LLM-Generated Disinformation

A new study challenges the validity of using LLM judges as proxies for human evaluation of AI-generated disinformation, finding that eight frontier LLM judges systematically diverge from human reader responses in their scoring, ranking, and reliance on textual signals. The research demonstrates that while LLMs agree strongly with each other, this internal coherence masks fundamental misalignment with actual human perception, raising critical questions about the reliability of automated content moderation at scale.

The research exposes a significant blind spot in AI safety evaluation: the assumption that LLM judges can reliably substitute for human judgment when assessing disinformation risks. By analyzing 290 articles with 2,043 human ratings against outputs from eight frontier models, researchers discovered persistent gaps between machine and human evaluators. LLM judges prove systematically harsher than humans, fail to recover human-level item rankings, and weight different textual features—prioritizing logical rigor while over-penalizing emotional content.

This matters because the AI industry increasingly relies on LLM-based evaluation to audit content safety and disinformation risks, treating it as a cost-effective alternative to human studies. The assumption has been that high inter-LLM agreement validates this approach. The findings invert this logic: strong machine-to-machine consensus appears to reflect shared model biases rather than validity. When eight different frontier models align internally but diverge from humans, it suggests they're optimizing for similar but misaligned objectives.

For AI developers and safety teams, this implies that automated content moderation systems trained on LLM judge feedback may systematically mischaracterize risks as humans perceive them. An article flagged as dangerous by model consensus might resonate differently with actual audiences. Conversely, content machines deprioritize could carry persuasive weight humans recognize.

The research highlights the need for human-in-the-loop validation in critical applications. As LLM-generated disinformation becomes more sophisticated, evaluation methods must reflect genuine reader responses rather than machine preferences. Organizations deploying content moderation should validate LLM judge outputs against human panels, not treat internal agreement as evidence of accuracy.

• →LLM judges systematically diverge from human readers in scoring severity, item-level rankings, and textual signal weighting
• →High inter-LLM agreement masks fundamental misalignment with human perception, making consensus a poor validity indicator
• →AI judges penalize emotional intensity more strongly and reward logical rigor more than human evaluators do
• →Automated content moderation systems relying on LLM judge feedback may systematically mischaracterize disinformation risks
• →Human-in-the-loop validation is critical for assessing LLM-generated disinformation in safety-critical applications