y0news
AnalyticsDigestsSourcesTopicsRSSAICrypto

#data-composition News & Analysis

2 articles tagged with #data-composition. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

2 articles
AINeutralarXiv – CS AI · Jun 196/10
🧠

Towards Engineering Scaling Laws with Pretraining Data Composition

Researchers demonstrate that neural scaling laws in particle physics can be engineered by optimizing pretraining data composition, shifting computational requirements toward larger datasets rather than bigger models. By using more diverse and task-aligned synthetic data from physics simulators, the study shows improved scaling efficiency for hadronic jet classification, offering a template for other domains with access to high-fidelity generative systems.

AIBearisharXiv – CS AI · Apr 206/10
🧠

Where does output diversity collapse in post-training?

Researchers discover that post-trained language models experience systematic output diversity collapse, where fine-tuning methods reduce the variety of generated responses compared to base models. This collapse is determined during training by data composition choices and cannot be fixed through inference-time adjustments, with implications for scaling methods and creative AI applications.