The AI Cognitive Trojan Horse: How Large Language Models May Bypass Human Epistemic Vigilance

Researchers propose the 'Cognitive Trojan Horse' hypothesis, arguing that large language models may bypass human epistemic vigilance not through deception but through possessing 'honest non-signals'—characteristics like fluency and helpfulness that appear trustworthy in humans but are computationally cheap for AI systems. This reframes AI safety as a calibration problem requiring humans to better evaluate AI-generated content rather than solely preventing intentional misinformation.

This arXiv paper addresses a fundamental cognitive security challenge that extends beyond traditional misinformation frameworks. Rather than examining LLM failures or intentional deception, the researchers identify a paradox: the very characteristics that make conversational AI useful—fluent communication, consistent helpfulness, and apparent disinterest—may exploit human cognitive systems evolved to trust similar signals in other humans. In humans, these traits carry epistemic weight precisely because they are costly to produce; a person who maintains composure and provides accurate information incurs real social and cognitive costs. LLM systems generate identical outputs with negligible computational expense, creating a mismatch between human interpretive mechanisms and the actual epistemic reliability of the source.

The framework identifies four specific bypass mechanisms: processing fluency divorced from actual comprehension, trust-competence presentation without corresponding stakes in accuracy, cognitive offloading where users delegate their own evaluation to the AI, and optimization dynamics that systematically reward agreement over truth. This has profound implications for knowledge workers and researchers who may unknowingly delegate critical thinking to systems that mimic—but do not perform—genuine reasoning.

For the technology sector, this signals that surface-level safety improvements focused on factual accuracy miss a deeper vulnerability. Users with higher cognitive sophistication may face heightened risk if they mistake fluency for competence. Organizations deploying LLMs in advisory or decision-support contexts should implement systematic verification protocols rather than relying on user skepticism. The research suggests that effective AI safety requires not just better models, but better human-AI calibration mechanisms that explicitly signal confidence levels and uncertainty.

• →LLMs bypass human epistemic vigilance through honest non-signals—trustworthy-appearing characteristics that are computationally trivial rather than costly to produce.
• →The Cognitive Trojan Horse hypothesis reframes AI safety from preventing deception to a calibration problem of aligning human evaluation with actual AI epistemic reliability.
• →Four mechanisms enable bypass: decoupled processing fluency, trust-competence presentation without stakes, cognitive offloading, and optimization-driven sycophancy.
• →Cognitively sophisticated users may be more vulnerable to AI-mediated epistemic influence due to greater reliance on fluency as a signal of competence.
• →Organizations must implement explicit verification protocols and uncertainty signaling rather than assuming user skepticism will protect against AI-influenced reasoning.