On the Limits of LLM-as-Judge for Scientific Novelty Assessment

Researchers demonstrate that Large Language Models systematically overestimate the novelty of AI-generated research questions compared to human expert assessment, revealing a critical gap in LLM-based scientific evaluation. The study introduces RQ-Bench, a benchmark showing that while LLMs rate model-generated questions as highly novel, domain experts prefer author-anchored reference questions and identify that many AI-generated questions lack depth or originality.

This research exposes a fundamental reliability problem with delegating scientific novelty assessment to language models. The study constructs a rigorous evaluation framework by extracting research questions from real arXiv papers, then comparing how LLMs and human experts rate AI-generated alternatives. The critical finding reveals a systematic bias: LLMs produce a "novelty mirage" where generated questions receive inflated novelty scores, while human scientists recognize these questions as derivative or narrow in scope.

The broader context reflects growing reliance on AI systems for knowledge work without adequate validation of their judgment capabilities. As LLMs become embedded in academic workflows and research ideation processes, unchecked overestimation of novelty could distort scientific direction and waste resources pursuing questions that appear novel but lack genuine substantive value. The research demonstrates that comparative evaluations amplify LLM bias even further, making standalone LLM judging particularly unreliable.

For the AI research community, this work highlights a critical validation gap. Organizations developing AI systems for research assistance face reputational and operational risks if their tools systematically mislead researchers about idea quality. The findings suggest that human expertise remains irreplaceable for evaluating scientific merit, and that AI-assisted ideation requires human expert oversight rather than autonomous assessment.

The implications extend to broader adoption of AI in knowledge domains where novelty and originality determine value. Future development should focus on hybrid human-AI evaluation frameworks rather than automation, and on understanding why LLMs fail at nuanced judgment tasks despite excelling at information synthesis.

• →LLMs consistently overrate novelty of AI-generated research questions compared to human expert assessment
• →The "novelty mirage" effect intensifies in comparative evaluations where LLMs show even stronger preference for generated content
• →Domain experts identify many AI-generated questions as narrow, source-bound, or lacking genuine originality that LLM judges miss
• →RQ-Bench benchmark demonstrates systematic reliability problems with autonomous LLM-based scientific novelty assessment
• →Human expert oversight remains essential for evaluating research quality; AI cannot reliably substitute for domain knowledge in ideation assessment