Think-Aloud Reshapes Automated Cognitive Model Discovery Beyond Behavior

Researchers demonstrate that incorporating think-aloud verbal protocols alongside behavioral data significantly improves automated cognitive model discovery using large language models. The approach shifts discovered models toward different structural classes, revealing decision-making mechanisms invisible to behavior-only analysis, particularly in risky decision-making contexts.

This research addresses a fundamental limitation in computational cognitive science: behavioral data alone produces under-determined models that fail to capture the actual cognitive processes driving decisions. By integrating think-aloud traces—verbal articulations of reasoning during tasks—researchers enhance model discovery accuracy and structural validity. The 69.4% shift from Explicit Comparator to Integrated Utility models suggests that process-level language data reveals cognitive mechanisms fundamentally different from what observable behavior implies.

The advancement emerges from growing sophistication in leveraging large language models for scientific discovery. Traditional cognitive modeling requires extensive manual hypothesis testing and parameter fitting. Automated discovery via LLMs reduces this friction, but previous iterations suffered from the classic inverse problem: multiple distinct models can produce identical behavioral outputs. Think-aloud protocols provide a richer data stream, constraining the solution space and enabling discovery of models with superior generalization to held-out test sets.

For AI development and cognitive science, this work demonstrates that multimodal data integration—combining behavioral metrics with language-based process traces—produces more robust and theoretically valid models. The methodology has implications for understanding human decision-making in high-stakes contexts like finance and healthcare, where understanding actual cognitive mechanisms matters beyond mere prediction. Researchers studying human-AI alignment and interpretability should note that process-level explanations appear essential for recovering true cognitive structures, not just surface-level behavioral patterns.

• →Think-aloud protocols significantly improve predictive performance of automatically discovered cognitive models compared to behavior-only approaches
• →Process-level language data reshapes model structure for 69% of participants, shifting toward Integrated Utility mechanisms
• →Behavioral data alone produces under-determined models that miss actual cognitive mechanisms driving risky decisions
• →Multimodal data integration enables discovery of cognitive models invisible to single-modality analysis
• →The approach demonstrates LLM-powered automated discovery of cognitive mechanisms with improved generalization and theoretical validity