AINeutralarXiv – CS AI · Mar 37/104
🧠
Not-Just-Scaling Laws: Towards a Better Understanding of the Downstream Impact of Language Model Design Decisions
New research analyzing 92 open-source language models reveals that factors beyond model size and training data significantly impact performance. The study shows that incorporating design features like data composition and architectural choices can improve performance prediction by 3-28% compared to using scale alone.