AIBearisharXiv – CS AI · Jun 257/10
🧠
Internal Data Repetition Destroys Language Models
Researchers demonstrate that data repetition in language model training systematically degrades performance, with peak damage occurring at moderate repetition levels rather than following linear degradation. Using modern scaling laws, they quantify that repeated data consuming just 10% of training compute can waste up to 67% of computational resources, revealing a critical inefficiency in how AI models are currently trained.