Do Language Models Track Entities Across State Changes?

Researchers investigated how transformer language models track entity states through multiple changes, finding that LMs use a non-incremental parallel aggregation strategy rather than sequential state tracking. The study reveals LMs implement state removal operations through a fragile global suppression mechanism, explaining various failure modes and suggesting mechanistic improvements for more robust entity tracking.

This research addresses a fundamental gap in understanding how large language models handle complex reasoning tasks involving sequential state changes. While prior work examined entity binding in simplified scenarios, this study investigates realistic entity tracking (ET) problems expressed in natural language, providing mechanistic insights into how transformers actually solve these problems at scale. The findings challenge intuitive assumptions about how LMs process sequential information, revealing they aggregate relevant information in parallel rather than tracking states incrementally across tokens or layers.

The research sits within a broader effort to interpret and improve language model reasoning capabilities. As LMs become increasingly deployed for complex reasoning tasks, understanding their underlying mechanisms is critical for predicting failures and building more reliable systems. This work contributes to mechanistic interpretability—a growing field focused on reverse-engineering neural networks to understand their decision-making processes.

For AI developers and researchers, these insights have practical implications. The discovery that LMs use fragile global suppression tags for removal operations explains why models fail in predictable ways when handling state changes. This mechanistic understanding enables targeted improvements. The proposed solution of nullifying suppression tags represents a concrete path toward more robust entity tracking without retraining entire models.

Looking forward, this research methodology—combining behavioral analysis with mechanistic investigation—establishes a template for auditing other reasoning capabilities in language models. Understanding where and how LMs fail systematically is essential as these models transition from language generation to serving as reasoning engines in more safety-critical applications. Future work should investigate whether these mechanisms generalize across different model architectures and whether the identified vulnerabilities persist in larger, newer model variants.

• →Language models aggregate state information in parallel at query time rather than tracking states incrementally across tokens
• →The REMOVE operation relies on a fragile global suppression mechanism that predicts specific failure modes confirmed through testing
• →Mechanistic analysis combined with behavioral evaluation reveals failure modes absent from conventional benchmarks
• →A mechanistic solution of nullifying suppression tags can partially improve entity tracking robustness without full retraining
• →LMs solve fundamentally sequential tasks using non-sequential strategies, explaining both their strengths and systematic weaknesses