AINeutralarXiv – CS AI · 7h ago7/10
🧠
From Architecture to Output: Structural Origins of Hallucination in Large Language Models and the Amplifying Role of Data
Researchers identify three core architectural mechanisms in large language models that systematically produce hallucinations: self-attention's statistical confusion of entities, maximum likelihood training that rewards plausible-sounding falsehoods, and autoregressive decoding that cascades errors forward. Dataset quality issues amplify rather than originate these failures, suggesting that fixing hallucinations requires architectural redesign, not just better training data.