E2Former-V2: On-the-Fly Equivariant Attention with Linear Activation Memory

Researchers introduce E2Former-V2, a more scalable architecture for Equivariant Graph Neural Networks that models 3D molecular systems. By combining algebraic sparsity with hardware-optimized execution, the model achieves 20× computational improvements while maintaining competitive accuracy on molecular datasets.

E2Former-V2 addresses a fundamental computational bottleneck in equivariant neural networks for molecular modeling. Traditional Equivariant Graph Neural Networks require explicit geometric feature construction and dense tensor operations on every edge, creating severe scalability limitations for larger molecules and systems. The researchers circumvent this through Equivariant Axis-Aligned Sparsification (EAAS), which leverages group theory—specifically an SO(3) to SO(2) change of basis—to convert expensive tensor contractions into sparse parity re-indexing operations.

The innovation extends beyond mathematical reformulation into practical hardware implementation. A custom Triton kernel performs on-the-fly equivariant attention computation, eliminating the need to materialize intermediate edge tensors. This kernel-level optimization maximizes SRAM utilization, delivering the reported 20× TFLOPS improvement. Validation on SPICE and OMol25 datasets confirms that computational gains don't sacrifice predictive performance.

This work has substantial implications for computational chemistry and drug discovery pipelines. Faster, more efficient equivariant models make sophisticated molecular property predictions feasible on standard GPU infrastructure, reducing computational costs and democratizing access to advanced modeling tools. The released code enables broader adoption across research institutions and industry applications.

The convergence of mathematical elegance with hardware pragmatism represents emerging best practices in deep learning infrastructure. As molecular ML increasingly drives drug discovery and materials science, efficient architectures become competitive advantages. Future developments likely focus on extending these sparse methods to larger model architectures and exploring additional symmetry groups beyond SO(3).

• →E2Former-V2 achieves 20× performance improvement through sparse tensor operations and custom GPU kernels
• →Equivariant Axis-Aligned Sparsification (EAAS) converts expensive tensor contractions into efficient parity re-indexing
• →Model maintains competitive accuracy on molecular datasets while dramatically accelerating inference
• →Hardware-aware implementation demonstrates feasibility of large equivariant transformers on accessible GPU platforms
• →Open-source release enables broader adoption in molecular modeling and drug discovery applications