When simulations look right but causal effects go wrong: Large language models as behavioral simulators

Research study reveals that Large Language Models can reproduce behavioral patterns but fail to accurately predict intervention effects. The study tested three LLMs on climate psychology interventions across 59,508 participants from 62 countries, finding that descriptive accuracy doesn't translate to causal prediction accuracy.

• →LLMs can simulate observed behavioral patterns reasonably well but struggle with accurate causal effect predictions.
• →Descriptive fit and causal accuracy follow different error structures and don't correlate reliably.
• →LLMs show larger errors for interventions requiring internal experience versus direct reasoning or social cues.
• →Models impose stronger attitude-behavior coupling than exists in actual human data.
• →Relying on descriptive fit alone may lead to overconfidence in AI simulation results.