CiteCheck: Retrieval-Grounded Detection of LLM Citation Hallucinations in Scientific Text

Researchers introduce CiteCheck, a hybrid framework that detects when large language models fabricate or corrupt scientific citations by combining scholarly database retrieval with structured LLM verification. The system achieves 88.7% macro-F1 on a new 982-citation physics benchmark, outperforming GPT, Claude, and Gemini, addressing a critical reliability problem as LLMs become integrated into scientific research workflows.

The emergence of citation hallucinations in LLM-generated scientific content represents a fundamental credibility challenge for AI adoption in academic and research contexts. CiteCheck tackles this problem through a pragmatic three-layer approach: retrieving candidate publications from scholarly databases, using structured LLM comparison rather than unguided text generation, and applying calibrated decision rules that distinguish between exact matches, minor metadata errors, and fabricated references. This methodology acknowledges that LLMs excel at pattern matching but fail at verifiable fact-checking without external grounding.

The problem exists because LLMs are trained on historical academic literature and learn to generate plausible-sounding citations that statistically fit their training context—but without access to real-time publication databases or verification mechanisms. As institutions increasingly leverage LLMs for literature review, report generation, and research synthesis, undetected hallucinations can propagate false references and damage academic integrity. The physics benchmark with controlled corruptions is particularly valuable because it isolates both subtle drift (incorrect publication dates, author names) and complete fabrications, creating a reproducible evaluation framework.

For the research community and knowledge workers, CiteCheck signals that citation verification will likely become a standard preprocessing step for any LLM-generated scientific output. The framework's superior performance against commercial AI models suggests that specialized, task-specific verification systems outcompete general-purpose LLMs at citation validation. Organizations publishing or relying on AI-assisted research may need to implement citation verification tools to maintain credibility. The broader implication is that trustworthy AI deployment in high-stakes domains requires hybrid architectures combining retrieval, structured reasoning, and calibrated decision-making rather than end-to-end learning.

• →CiteCheck achieves 88.7% macro-F1 in detecting citation hallucinations, outperforming GPT, Claude, and Gemini models
• →The framework combines scholarly database retrieval with structured LLM verification to ground citations in reality
• →A new 982-citation physics benchmark with controlled corruptions enables reproducible evaluation of citation verification systems
• →Citation hallucinations remain a critical vulnerability in LLM-generated scientific content despite model improvements
• →Specialized verification systems outperform general-purpose LLMs at detecting subtle metadata drift and fabricated references