Empathic Prompting: Non-Verbal Context Integration for Multimodal LLM Conversations

Researchers present Empathic Prompting, a framework that integrates facial expression recognition into multimodal LLM conversations to capture and embed users' emotional cues as contextual signals. The system operates unobtrusively through a locally deployed DeepSeek instance and demonstrates coherent integration of non-verbal input in a preliminary evaluation (N=5), with potential applications in healthcare and education.

Empathic Prompting represents a meaningful step toward more contextually aware AI interactions by automating the detection and incorporation of emotional signals that humans naturally convey through facial expressions. Rather than requiring explicit user input, the system passively monitors non-verbal cues and weaves this affective context into conversational prompts, creating a more nuanced dialogue between humans and LLMs. This addresses a fundamental gap in current multimodal AI: while vision models can process images and text models process language, few systems seamlessly integrate emotional context to inform conversational tone and relevance.

The research builds on growing recognition within AI development that emotion and context significantly influence communication effectiveness. Domains like healthcare and education have long relied on practitioners' ability to read emotional states to tailor their responses—empathic prompting attempts to democratize this capability in digital assistants. The modular architecture suggests the framework can accommodate additional non-verbal inputs (tone of voice, gesture, biometric signals), positioning it as a foundation for richer human-AI interaction paradigms.

For developers and AI product teams, this work signals increased investment in affective computing within conversational AI. The use of commercial facial recognition services highlights both opportunity and privacy considerations for production deployment. While the preliminary evaluation is small-scale, successful scaling could reshape expectations for AI assistants in sensitive domains where understanding user emotional state directly impacts outcome quality. The framework's implementation via locally deployed models also demonstrates feasibility for privacy-conscious deployments.

The path forward involves larger-scale validation, exploration of bias in emotion recognition across demographics, and clarification of consent and data handling protocols for emotion monitoring in production systems.

• →Empathic Prompting integrates facial expression recognition into LLM conversations to embed emotional context without explicit user control.
• →The modular architecture enables integration of additional non-verbal signals beyond facial expressions for richer AI interaction.
• →Preliminary evaluation (N=5) shows consistent non-verbal context integration and improved conversational fluidity in user feedback.
• →Applications target healthcare and education domains where users' emotional signals are critical yet often opaque in text-based interactions.
• →Privacy and bias considerations for emotion recognition require attention before large-scale deployment in production systems.