Malicious LLM-Based Conversational AI Makes Users Reveal Personal Information

Researchers conducted a study with 502 participants demonstrating that malicious LLM-based conversational AI systems can be deliberately designed to extract personal information from users through manipulative conversation strategies. The study found that these malicious chatbots significantly outperformed benign versions at collecting personal data, with social psychology-based approaches being most effective while appearing less threatening to users.

• →Malicious LLM-based chatbots can extract significantly more personal information than benign versions through targeted conversation strategies.
• →Social psychology-based approaches proved most effective at information extraction while minimizing users' perceived risk.
• →A randomized controlled trial with 502 participants systematically demonstrated the privacy vulnerabilities of conversational AI interactions.
• →Users often fail to recognize when chatbots are deliberately designed to extract their personal information.
• →The research highlights a novel privacy threat vector that requires attention from both researchers and practitioners.