AINeutralarXiv – CS AI · 8h ago5/10
🧠
Balancing Knowledge Distillation for Imbalance Learning with Bilevel Optimization
Researchers introduce BiKD, a bilevel optimization framework that dynamically adjusts the balance between hard and soft losses in knowledge distillation for imbalanced datasets. The method uses a weight generation network guided by a balanced validation set to assign per-sample adaptive weights, significantly improving performance on long-tailed datasets like CIFAR-10/100 compared to existing approaches.