Communication Dynamics Neural Networks: FFT-Diagonalized Layers for Improved Hessian Conditioning at Reduced Parameter Count

Researchers introduce CDLinear, a neural network layer based on the Communication Dynamics framework that achieves 3.8× parameter reduction compared to dense layers while maintaining comparable accuracy. The layer uses block-circulant matrices with FFT-diagonalization to dramatically improve Hessian conditioning, reducing the condition number by 310× in empirical tests.

This paper presents a theoretically-grounded approach to neural network design that bridges physics-inspired spectral methods with deep learning optimization. The CDLinear layer leverages circulant matrix structure to achieve simultaneous gains in parameter efficiency and training stability—two orthogonal goals that typically require trade-offs. The Hessian diagonalization property (Theorem 1) is particularly significant because it provides explicit control over optimization landscape geometry, a problem that has plagued second-order methods in deep learning for years.

The work builds on the Communication Dynamics framework previously applied to atomic energy prediction and superconductivity modeling, demonstrating that physics-inspired architectural choices can transfer to general machine learning. The theoretical guarantees are strong: under input pre-whitening, the population Hessian achieves perfect conditioning (κ=1), with empirical degradation bounded by sample-dependent terms. This contrasts sharply with dense networks where condition numbers exceed 10^6.

The empirical results show a 0.65% accuracy penalty when reducing parameters 3.8×, a favorable trade-off for deployment scenarios where model size matters. The transferable dropout rate calibrated from atomic spectroscopy (α=0.0118) reveals an unexpected connection between physical noise characteristics and neural network regularization. The 310× condition number improvement validates the theoretical predictions and suggests potential benefits for second-order optimization methods.

Limitations include evaluation only on relatively simple tasks (implied by parameter counts) and unclear generalization to modern architectures (transformers, convolutional networks). The approach's applicability to larger-scale problems remains unexplored, though the parameter reduction could prove valuable for edge deployment or efficient fine-tuning.

• →CDLinear achieves 3.8× parameter reduction with only 0.65% accuracy loss through spectral circulant matrix design
• →FFT-diagonalization of the Hessian provides 310× better conditioning than dense layers, enabling improved optimization
• →Theoretical guarantees prove perfect population Hessian conditioning under pre-whitening, validated empirically
• →Physics-inspired architectural choices transfer from atomic modeling to general neural network design
• →Transferable dropout rate derived from physical spectroscopy suggests deep connections between domains