Mapping how LLMs debate societal issues when shadowing human personality traits, sociodemographics and social media behavior

Researchers have created Cognitive Digital Shadows (CDS), a 190,000-record synthetic dataset of LLM-generated responses on controversial societal topics, designed to measure how language models shift their outputs based on persona prompting and sociodemographic attributes. The dataset enables systematic auditing of LLM bias, alignment, and social sensitivity across 19 different models.

The emergence of large language models as influential participants in social discourse has created an urgent need for empirical research into how these systems vary their outputs based on contextual and demographic prompting. The Cognitive Digital Shadows project addresses a critical gap by providing a structured, reproducible dataset that maps LLM behavior across controlled social variables. This work matters because it moves beyond anecdotal observations of AI bias toward systematic, quantifiable analysis.

The dataset's design is sophisticated: by encoding 17 sociodemographic and psychological attributes alongside LLM responses on four contentious topics—vaccines, disinformation, gender in STEM, and stereotypes—researchers create a foundation for understanding whether models amplify existing social divisions or generate outputs that vary meaningfully with persona. The inclusion of 19 different LLMs allows comparative analysis of architectural or training differences in social sensitivity.

For the AI industry, this research has significant implications. As regulators increasingly scrutinize AI systems for fairness and alignment, datasets like CDS provide evidence-based frameworks for demonstrating (or failing to demonstrate) responsible behavior. The interactive dashboard platform democratizes access to these insights, enabling developers, auditors, and policy researchers to conduct their own investigations without specialized technical infrastructure.

The work also highlights emerging vulnerabilities in LLM deployment. If models reliably adopt stances corresponding to demographic personas, this creates risks for social manipulation and polarization. Going forward, the field should monitor whether similar datasets reveal systematic biases that warrant architectural interventions or training modifications.

• →CDS dataset provides 190,000 LLM-generated records mapped to 17 sociodemographic variables for bias auditing.
• →Research demonstrates that LLM outputs measurably vary when prompted to shadow different human personas.
• →Framework enables comparative analysis across 19 LLM architectures on controversial societal topics.
• →Interactive dashboard enables non-technical stakeholders to conduct AI fairness audits and comparative analysis.
• →Dataset reveals potential vulnerabilities in LLM deployment related to social polarization and demographic-based output variation.