EquiformerV3: Scaling Efficient, Expressive, and General SE(3)-Equivariant Graph Attention Transformers

EquiformerV3, an advanced SE(3)-equivariant graph neural network, achieves significant improvements in efficiency, expressivity, and generality for 3D atomistic modeling. The new version delivers 1.75x speedup, introduces architectural innovations like SwiGLU-S² activations and smooth-cutoff attention, and achieves state-of-the-art results on major molecular modeling benchmarks including OC20 and OMat24.

EquiformerV3 represents a meaningful advancement in machine learning infrastructure for molecular simulation, addressing practical deployment challenges that have limited broader adoption of equivariant neural networks. The 1.75x performance improvement directly translates to reduced computational costs, making these models accessible to organizations with constrained compute budgets. The architectural refinements—particularly SwiGLU-S² activations and smooth-cutoff attention mechanisms—enhance both theoretical expressivity and practical accuracy for modeling potential energy surfaces, critical for materials discovery and drug development pipelines.

This work builds on years of incremental progress in equivariant graph neural networks. SE(3)-equivariance ensures that predictions remain physically consistent under rotations and translations, a fundamental requirement for atomistic systems. Previous generations established the transformer architecture's viability for this domain; EquiformerV3 optimizes implementation and theoretical expressivity simultaneously. The inclusion of many-body interactions through activation functions strengthens the model's capacity to capture complex molecular interactions while maintaining strict equivariance.

The industry impact extends across pharmaceutical development, materials science, and computational chemistry sectors. Organizations relying on quantum mechanical simulations for molecular property prediction now have access to faster, more accurate alternatives. The denoising non-equilibrium structures auxiliary task demonstrates the model's robustness across diverse scenarios. For enterprises evaluating AI-accelerated discovery platforms, EquiformerV3's performance benchmarks become relevant comparison points. The work establishes practical improvements beyond theoretical contributions, making deployment-ready solutions increasingly viable for commercial applications requiring energy-conserving simulations and high-order derivative calculations.

• →EquiformerV3 achieves 1.75x speedup while maintaining SE(3)-equivariance for molecular modeling
• →SwiGLU-S² activations incorporate many-body interactions while preserving strict physical equivariance
• →Smooth-cutoff attention enables accurate modeling of potential energy surfaces with higher-order derivatives
• →State-of-the-art results on OC20, OMat24, and Matbench Discovery benchmarks validate the approach
• →Auxiliary denoising task demonstrates robustness across non-equilibrium structure modeling