Transformers Remember First, Forget Last: Dual-Process Interference in LLMs

Research analyzing 39 large language models reveals they exhibit proactive interference (remembering early information over recent) unlike humans who typically show retroactive interference. The study found this pattern is universal across all tested LLMs, with larger models showing better resistance to retroactive interference but unchanged proactive interference patterns.

• →All 39 tested LLMs universally prioritize early information over recent information when encountering conflicts, opposite to human memory patterns.
• →Larger language models show better resistance to retroactive interference but maintain the same proactive interference levels regardless of size.
• →The research suggests transformer attention mechanisms create an inherent primacy bias in how LLMs process and retain information.
• →Different failure modes were identified: retroactive interference failures are passive retrieval issues while proactive interference shows active primacy intrusion.
• →The findings have direct implications for applications where LLMs must handle conflicting or updating information streams.