Topological derivative approach for deep neural network architecture adaptation

Researchers developed a novel algorithm using topological derivatives to automatically determine where and how to add new layers to neural networks during training. The approach uses mathematical principles from optimal control theory and topology optimization to adaptively grow network architecture, showing superior performance compared to baseline networks and other adaptation strategies.

• →New algorithm uses topological derivatives to mathematically determine optimal placement of new layers in neural networks during training.
• →The approach connects topology optimization framework with Hamiltonian optimal control theory for the first time.
• →Method provides both optimal layer insertion location and proper initialization parameters for new layers.
• →Algorithm can be derived from optimal transport theory as a solution maximizing topological derivative in p-Wasserstein space.
• →Testing on fully connected networks, CNNs, and vision transformers shows outperformance of baseline and other adaptation strategies.