Restoring Heterogeneity in LLM-based Social Simulation: An Audience Segmentation Approach

Researchers demonstrate that Large Language Models used for social simulation produce more accurate behavioral predictions when trained with audience segmentation strategies rather than averaged personas. The study finds that moderate identifier granularity and data-driven selection methods optimize structural and predictive fidelity, with no single configuration excelling across all evaluation dimensions.

This research addresses a fundamental limitation in using LLMs as social simulators: the tendency to flatten demographic and behavioral diversity into homogenized outputs. The study's significance lies in establishing systematic methodologies for preserving heterogeneity, which is essential for accurate social modeling in domains like climate opinion, consumer behavior, and policy analysis. Using U.S. climate data across Llama and Mixtral models, researchers tested multiple segmentation approaches and discovered a critical insight: more granular segmentation doesn't guarantee better results, contradicting the assumption that increased complexity improves accuracy.

The research builds on growing recognition that LLM-based simulations can supplement traditional survey research when properly calibrated. Current industry practice often treats LLMs as monolithic agents, missing subgroup variation crucial to realistic social dynamics. This work provides empirical guidance on configuration trade-offs, showing that parsimony sometimes outperforms comprehensive models—a counterintuitive finding that challenges optimization impulses in AI development.

For stakeholders leveraging LLMs in social research, market research, and policy analysis, these findings suggest substantial opportunities for improved fidelity. Organizations currently using LLM simulations may be capturing only partial behavioral signals and missing critical between-group differences that drive real-world outcomes. The three-dimensional evaluation framework (distributional, structural, predictive) provides practical standards for validation. The key implication: methodological choices in audience segmentation directly determine which aspects of social reality LLMs capture accurately, making this less a technical optimization problem and more a strategic research design decision that varies by use case and priority.

• →Audience segmentation with moderate granularity improves LLM social simulation accuracy better than either minimalist or maximally detailed approaches.
• →Data-driven identifier selection best recovers between-group structure and relationships, while instrument-based selection preserves distributional shape.
• →No single segmentation configuration optimizes all evaluation dimensions simultaneously; researchers must prioritize fidelity trade-offs based on use case.
• →Compact segmentation models often match or exceed more complex alternatives in structural and predictive fidelity while reducing computational overhead.
• →LLM-based social simulation requires heterogeneity-aware evaluation frameworks to avoid false confidence in averaged behavioral outputs.