Does Persona Make LLMs K-pop Fans? A Pilot Study of LLM-Based Online Concert Audience Agents

Researchers tested whether large language models assigned distinct personas could simulate a live concert audience experience through real-time chat during K-pop video playback. While persona-conditioned LLM agents produced more natural and higher-quality chat messages than baseline models, the study found no measurable improvement in user engagement, social connectedness, or emotional response, suggesting that algorithmic personas alone cannot replicate the cultural and social depth of authentic fandom experiences.

This pilot study reveals a significant gap between technical performance and user experience in AI-generated social systems. Researchers deployed ten LLM agents with distinct fan identities to generate live-chat commentary during K-pop performances, comparing persona-based outputs against non-personalized baselines. The quantitative results showed clear improvements in chat naturalness and quality metrics when personas were applied, yet qualitative measures—user engagement, emotional response, and perceived social connection—remained unchanged. This disconnect highlights a fundamental limitation in current AI systems: they can convincingly mimic individual communication styles without capturing the underlying cultural and social dynamics that drive authentic collective experience.

The research contextualizes a broader challenge facing AI developers building social and entertainment platforms. As LLMs become increasingly sophisticated at generating human-like outputs, creators face growing pressure to deploy these systems in interactive contexts. However, this study demonstrates that surface-level naturalism does not translate into meaningful user experiences. Interview data suggested that concert chat functions as a "collective monologue" where participants broadcast reactions rather than engage in genuine dialogue, and that authentic participation requires deep alignment with specific artist fandom cultures—aspects no persona layer can algorithmically replicate.

For the AI industry, this research tempers expectations around using persona-conditioning as a solution for creating synthetic social experiences. While the approach shows promise for improving chatbot naturalness in general applications, the K-pop domain requires deeper cultural knowledge integration. Developers should expect that entertainment and social systems demand more than convincing individual personas; they require authentic community representation and fandom-specific mechanics that LLMs cannot generate from generic persona parameters.

• →Persona-conditioned LLMs improved chat quality and naturalness but did not enhance user engagement or emotional connection
• →Online concert chat operates as collective monologue rather than interpersonal dialogue, limiting what AI personas can replicate
• →Meaningful participation in fan communities requires shared identification with specific artists and fandom cultures
• →Technical improvements in individual agent outputs do not guarantee better overall user experience in multi-agent social systems
• →AI-generated social experiences may require deeper cultural alignment beyond persona-based prompting techniques