AIBullisharXiv – CS AI · Feb 277/106
🧠
ViT-Linearizer: Distilling Quadratic Knowledge into Linear-Time Vision Models
Researchers developed ViT-Linearizer, a distillation framework that transfers Vision Transformer knowledge into linear-time models, addressing quadratic complexity issues for high-resolution inputs. The method achieves 84.3% ImageNet accuracy while providing significant speedups, bridging the gap between efficient RNN-based architectures and transformer performance.