#branching-processes News & Analysis

Towards Critical Branching Mechanism in Recurrent Neural Networks

Researchers demonstrate that small LSTM neural networks exhibit critical dynamics near optimal training, displaying scale-free avalanche statistics and branching parameters close to unity, while larger models remain subcritical. The study introduces a mixture branching process framework to explain how subcritical dynamics can coexist with long-range temporal correlations, suggesting criticality emerges as a capacity-dependent property in artificial neural networks.