Resources for Automated Evaluation of Assistive RAG Systems that Help Readers with News Trustworthiness Assessment

Researchers developed the TREC 2025 DRAGUN Track to evaluate AI systems that help readers assess news trustworthiness through automated report generation. The initiative created reusable evaluation resources including human-assessed rubrics and an AutoJudge system that correlates well with human evaluations for RAG-based news analysis tools.

• →TREC 2025 DRAGUN Track created standardized evaluation methods for AI systems that assess news trustworthiness.
• →The track included two main tasks: generating investigative questions and producing 250-word trustworthiness reports.
• →Human assessors created importance-weighted rubrics for 30 news articles to establish evaluation benchmarks.
• →The automated AutoJudge system achieved strong correlation with human evaluations (τ = 0.678 for Task 1, τ = 0.872 for Task 2).
• →These resources enable future research on improving RAG systems for news trustworthiness assessment and automated evaluation methods.