SimpleToM: Exposing the Gap between Explicit ToM Inference and Implicit ToM Application in LLMs

Researchers introduced SimpleToM, a benchmark revealing that state-of-the-art language models can infer mental states but struggle to apply that knowledge for behavior prediction and judgment. The study exposes a critical gap between explicit Theory of Mind inference and implicit application in real-world scenarios.

• →SimpleToM benchmark tests LLMs across multiple levels of Theory of Mind reasoning in everyday scenarios like supermarkets and hospitals.
• →Current state-of-the-art models reliably infer mental states but fail when applying that knowledge to predict behaviors.
• →Performance drops sharply from mental state inference to behavior prediction and further to behavior judgment tasks.
• →The research reveals a fundamental fragility in LLMs' social reasoning capabilities.
• →The gap between explicit knowledge and implicit application represents a significant limitation in current AI systems.