UF-AMA: A unified framework for cross-domain emotion recognition via adaptive multimodal alignment

Researchers introduce UF-AMA, a unified framework for cross-domain emotion recognition using multimodal physiological signals like EEG and eye-tracking data. The model employs adaptive alignment mechanisms and multi-level domain adaptation to achieve state-of-the-art performance in cross-subject and cross-session emotion recognition tasks.

The UF-AMA framework addresses a significant challenge in affective computing: building emotion recognition systems that generalize across different subjects and recording sessions. Physiological signal-based emotion recognition offers advantages over behavioral approaches due to its objectivity, yet individual differences and variations in data quality create substantial distribution shifts that limit real-world deployment. This research tackles that problem through three key innovations: deep multimodal fusion combining EEG and eye-tracking via Transformer architectures, dynamic confidence-aware sample screening that identifies reliable predictions per modality, and a multi-level domain adaptation strategy targeting both marginal and conditional distributions across granularities.

The broader context reflects growing momentum in affective computing and biomedical signal processing. As emotion AI applications expand into mental health monitoring, user experience research, and human-computer interaction, robust cross-domain models become increasingly critical infrastructure. Current systems typically fail when deployed across different subjects or sessions, limiting commercial viability. The SOTA results on SEED and SEED-IV benchmarks validate that confidence-aware mechanisms and adaptive alignment successfully mitigate these generalization failures.

For the AI research community, this work establishes replicable techniques for multimodal domain adaptation applicable beyond emotion recognition. Open-source code availability accelerates adoption and extension. However, the advancement remains academically focused with limited immediate commercial impact. Potential applications in healthcare, wearable technology, and human-centered AI development create longer-term industry relevance, particularly as emotion recognition transitions from research prototypes to production systems.

• →UF-AMA combines Transformer encoders with confidence-aware screening to handle cross-domain emotion recognition from physiological signals
• →Multi-level domain adaptation framework reduces distribution shifts across both subject-specific and session-specific variations
• →State-of-the-art performance on SEED and SEED-IV datasets demonstrates superior generalization compared to existing approaches
• →Dynamic modality assessment enables adaptive reliance on EEG versus eye-tracking based on target domain sample characteristics
• →Open-source release accelerates research adoption and potential development of production-grade emotion recognition systems