Human-like Working Memory Interference in Large Language Models

Researchers discovered that large language models exhibit working memory limitations similar to humans, encoding multiple memory items in entangled representations that require interference control rather than direct retrieval. This finding reveals a shared computational constraint between biological and artificial systems, suggesting that working memory capacity may be a fundamental bottleneck in intelligent systems rather than a limitation unique to biological brains.

This research challenges assumptions about transformer architecture's unlimited context capacity by demonstrating that pretrained LLMs struggle with working memory tasks despite theoretical advantages. The study reveals that models don't leverage their full attention mechanisms to directly retrieve relevant information; instead, they rely on interference suppression—a computationally expensive process that mirrors human cognitive limitations. This mechanism explains performance degradation under high memory loads and susceptibility to recency bias.

The findings connect to broader AI safety and capability research. Understanding that LLMs share fundamental cognitive constraints with humans has implications for designing more reliable systems. The correlation between working memory performance and general competence across benchmarks suggests this isn't a trivial quirk but reflects something fundamental about how these systems process information. Prior work on attention mechanisms and context windows assumed computational capacity was the limiting factor; this research identifies representation quality and interference management as the actual constraints.

For AI developers and practitioners, these insights matter because they explain why scaling context windows alone doesn't proportionally improve reasoning performance. Organizations building AI systems should expect fundamental limitations in simultaneous information manipulation regardless of model size. The successful intervention that suppresses irrelevant stimulus information offers a potential avenue for model improvement through targeted training techniques. Future developments may involve architectural or training modifications that improve interference control, potentially unlocking better performance on reasoning-intensive tasks that require maintaining multiple concurrent information streams.

• →Pretrained LLMs exhibit human-like working memory interference despite architectural access to full context through attention mechanisms
• →Models encode memory items in entangled representations requiring interference suppression rather than direct retrieval of relevant information
• →Working memory capacity correlates with general intelligence benchmarks in both biological and artificial systems, suggesting a fundamental computational constraint
• →Targeted interventions suppressing irrelevant stimulus content causally improve performance, offering potential paths for model enhancement
• →Working memory limitations appear to reflect shared computational challenges in selecting task-relevant information rather than architectural design differences