MS-rPPG: Multi-spectral State Space Model for Remote Photoplethysmography in Driver Monitoring Systems
Researchers introduce MS-rPPG, a multi-spectral framework combining RGB and near-infrared video for remote heart rate estimation in driver monitoring systems. The method uses a novel state space model (MS-Mamba) to improve accuracy under challenging driving conditions with varying lighting and head movements, validated on real-world datasets.
MS-rPPG addresses a critical gap in automotive health monitoring by tackling the technical challenges of remote photoplethysmography in uncontrolled environments. Traditional rPPG systems struggle with illumination variations and motion artifacts typical in vehicles, making real-time driver health assessment unreliable. This work leverages multi-spectral imaging—combining visible light with near-infrared—to capture complementary physiological signals that remain robust across different lighting conditions. The cross-spectral linear modulation strategy enables intelligent fusion of data from two spectral domains, while the MS-Mamba architecture applies advanced temporal modeling to extract meaningful patterns from noisy, continuous video streams. The introduction of MS-Drive, a dataset from 50 real-world driving scenarios, provides the research community with practical validation infrastructure beyond controlled laboratory settings. For the automotive and health-tech industries, this development opens pathways for non-intrusive driver monitoring systems that could detect fatigue, stress, or cardiac irregularities during operation. Insurance companies and autonomous vehicle manufacturers could leverage such systems for safety enhancement and risk assessment. The work demonstrates that specialized neural architectures and multi-modal sensor fusion can overcome environmental constraints that previously limited camera-based physiological monitoring. As driver assistance systems become increasingly sophisticated, accurate biometric monitoring without wearables or additional hardware represents a significant competitive advantage. The open-source code release accelerates industry adoption and encourages further refinement of the technology across different driving conditions and vehicle types.
- →Multi-spectral fusion of RGB and NIR video improves remote heart rate estimation reliability in challenging driving environments
- →MS-Mamba state space model effectively captures temporal dependencies and cross-channel interactions in physiological signal processing
- →Real-world MS-Drive dataset provides validation infrastructure for driver monitoring systems beyond laboratory conditions
- →Non-intrusive health monitoring technology enables new applications in vehicle safety and driver wellbeing assessment
- →Open-source release accelerates adoption and development of camera-based physiological monitoring systems across industries