EmambaIR: Efficient Visual State Space Model for Event-guided Image Reconstruction

EmambaIR introduces a novel State Space Model architecture for event-based image reconstruction that achieves superior performance over CNNs and Vision Transformers while maintaining linear computational complexity. The framework combines sparse attention mechanisms with gated state-space modules to process event camera data efficiently across motion deblurring, deraining, and HDR enhancement tasks.

EmambaIR represents a significant architectural advancement in computational imaging, addressing fundamental efficiency challenges that have constrained event camera applications. Event cameras capture asynchronous pixel-level changes rather than full frames, generating sparse temporal data that existing deep learning architectures struggle to process effectively. CNNs lack the receptive fields to capture global dependencies in this sparse data, while Vision Transformers' quadratic complexity makes them impractical for high-resolution reconstruction tasks.

The research emerges within a broader trend of replacing traditional attention mechanisms with linear-complexity alternatives, particularly State Space Models (SSMs) that have gained prominence following successes in language and image domains. EmambaIR advances this paradigm by introducing domain-specific innovations: the Top-k Sparse Attention Module exploits event data's inherent sparsity for efficient cross-modal fusion, while the Gated State-Space Module adds nonlinearity to capture temporal dynamics that vanilla SSMs miss.

For the computational imaging and robotics sectors, this work unlocks practical applications previously hindered by computational constraints. Event cameras enable low-latency, high-dynamic-range capture essential for autonomous systems and surveillance, yet reconstruction quality has lagged traditional cameras. EmambaIR's demonstrated improvements across three diverse restoration tasks suggest the approach generalizes well. The method's reduced memory consumption and computational cost enable deployment on edge devices, a critical requirement for real-world applications.

The open-source release positions this work to accelerate adoption among researchers and practitioners developing event-based vision systems. Future developments may extend these principles to real-time video reconstruction or multi-task learning scenarios, further expanding the practical utility of event cameras in consumer and industrial applications.

• →EmambaIR achieves linear computational complexity O(n) compared to Vision Transformers' O(n²), enabling efficient high-resolution event image reconstruction
• →The framework outperforms state-of-the-art methods across motion deblurring, deraining, and HDR enhancement tasks with significantly lower memory consumption
• →Top-k Sparse Attention Module leverages event data's natural sparsity for efficient cross-modal feature fusion between event and intensity streams
• →Gated State-Space Module combines linear-complexity SSMs with nonlinear gating to capture temporal dependencies in asynchronous event streams
• →Open-source availability accelerates adoption in robotics, autonomous systems, and computational imaging applications requiring low-latency processing