From Shallow Bayesian Neural Networks to Gaussian Processes: General Convergence, Identifiability and Scalable Inference

Researchers established a new theoretical framework connecting Bayesian neural networks to Gaussian processes, developing improved convergence results and identifiability properties. They introduced a scalable computational method using Nyström approximation for training and prediction, demonstrating competitive performance on real-world datasets.

• →General convergence result from shallow Bayesian neural networks to Gaussian processes established with relaxed assumptions.
• →New covariance function proposed as convex mixture of components from four activation functions with proven positive definiteness.
• →Scalable maximum a posterior training procedure developed using Nyström approximation for computational efficiency.
• →Theoretical characterization of identifiability properties under different input designs provided.
• →Experiments show stable hyperparameter estimates and competitive predictive performance at realistic computational cost.