AINeutralarXiv – CS AI · 14h ago6/10
🧠
Do Language Models Track Entities Across State Changes?
Researchers investigated how transformer language models track entity states through multiple changes, finding that LMs use a non-incremental parallel aggregation strategy rather than sequential state tracking. The study reveals LMs implement state removal operations through a fragile global suppression mechanism, explaining various failure modes and suggesting mechanistic improvements for more robust entity tracking.