A Neuropsychologically Grounded Evaluation of LLM Cognitive Abilities

Researchers introduced NeuroCognition, a new benchmark for evaluating LLMs based on neuropsychological tests, revealing that while models show unified capability across tasks, they struggle with foundational cognitive abilities. The study found LLMs perform well on text but degrade with images and complexity, suggesting current models lack core adaptive cognition compared to human intelligence.

• →Large language models exhibit a unified general factor of capability across 156 models and 10 benchmarks according to factor analysis.
• →Current benchmarks focus on task completion but fail to probe foundational cognitive abilities that drive intelligent behavior.
• →The NeuroCognition benchmark uses three neuropsychological tests to evaluate abstract reasoning, working memory, and cognitive flexibility.
• →LLM performance degrades significantly when moving from text to images and with increased task complexity.
• →Simple, human-like strategies yield better results than complex reasoning approaches for LLMs on cognitive tasks.