AINeutralarXiv – CS AI · 14h ago6/10
🧠
Teaching the Teacher: The Role of Teacher-Student Smoothness Alignment in Genetic Programming-based Symbolic Distillation
Researchers propose a novel framework for improving symbolic distillation of neural networks by regularizing teacher models for functional smoothness using Jacobian and Lipschitz penalties. This approach addresses the core challenge that standard neural networks learn complex, irregular functions while symbolic regression models prioritize simplicity, resulting in poor knowledge transfer. Results across 20 datasets demonstrate statistically significant improvements in predictive accuracy for distilled symbolic models.