Self-EmoQ: Plutchik-Guided Value-based Planning to Drive Streaming Emotional TTS

Researchers propose Self-EmoQ, an emotion-planning framework that determines emotional context before text generation to improve streaming emotional text-to-speech synthesis. The system uses reinforcement learning with Plutchik's emotion theory and demonstrates superior performance on multiple dialogue datasets, with a functional real-time deployment pipeline.

Self-EmoQ addresses a significant gap in conversational AI by introducing a systematic approach to emotional consistency in speech synthesis. Current TTS systems often fail to maintain emotional coherence throughout responses, resulting in jarring or inappropriate tone shifts. This framework solves that problem by planning emotions upstream of text generation, ensuring downstream synthesis authentically reflects intended emotional states.

The technical approach leverages recent advances in large language models combined with reinforcement learning, using Plutchik's wheel of emotions as a theoretical foundation for reward signals. By grounding emotion selection in established psychological theory rather than pure data-driven patterns, the system achieves both empirical performance gains and theoretical coherence. The hybrid reward combining imitation learning with theory-driven scoring represents a pragmatic middle ground between pure learning and rigid rule-based systems.

For the conversational AI industry, this work signals movement toward more sophisticated emotional intelligence in deployed systems. Applications spanning customer service, mental health support, entertainment, and education depend on appropriate emotional delivery. The real-time deployment pipeline indicates the approach scales beyond academic proof-of-concept to practical implementation. The public release of code and demos accelerates adoption across research and commercial applications.

Developers building emotional AI experiences now have a validated framework to improve user engagement and satisfaction. As conversational AI becomes more prevalent in sensitive domains, the ability to maintain emotional coherence could differentiate quality implementations. Future work will likely explore emotion planning across longer conversations and multi-turn interactions where emotional consistency becomes increasingly challenging.

• →Self-EmoQ determines emotional context before text generation to ensure consistent emotional TTS synthesis
• →The framework combines reinforcement learning with Plutchik's psychological emotion theory for improved performance
• →System outperforms baseline methods on emotion determination and response quality across four major dialogue datasets
• →Real-time deployment pipeline confirms practical viability for production conversational AI applications
• →Open-source release with code, cases, and demos enables faster adoption by researchers and developers