AIBullisharXiv – CS AI · Apr 76/10
🧠
Training Transformers in Cosine Coefficient Space
Researchers developed a new method to train transformer neural networks using discrete cosine transform (DCT) coefficients, achieving the same performance while using only 52% of the parameters. The technique requires no architectural changes and simply replaces standard linear layers with spectral layers that store DCT coefficients instead of full weight matrices.
🏢 Perplexity