#experimental-methodology News & Analysis

2 articles tagged with #experimental-methodology. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

2 articles

AINeutralarXiv – CS AI · May 126/10

🧠

Probing Routing-Conditional Calibration in Attention-Residual Transformers

Researchers question whether routing traces in Attention-Residual transformers provide genuine evidence of improved post-hoc calibration beyond standard confidence metrics. Through rigorous statistical testing with matched controls, the study finds that routing-specific features offer minimal stable evidence of better calibration, suggesting previous claims of calibration improvements may reflect methodological artifacts rather than true model improvements.

AINeutralarXiv – CS AI · Apr 146/10

🧠

Can Small Training Runs Reliably Guide Data Curation? Rethinking Proxy-Model Practice

Researchers demonstrate that small-scale proxy models commonly used by AI companies to evaluate data curation strategies produce unreliable conclusions because optimal training configurations are data-dependent. They propose using reduced learning rates in proxy model training as a simple, cost-effective solution that better predicts full-scale model performance across diverse data recipes.

🏢 Meta