AINeutralarXiv – CS AI · 7h ago6/10
🧠
Preregistration for Experiments with AI Agents
Researchers propose extending preregistration practices from human subjects research to AI agent experiments, addressing methodological vulnerabilities introduced by the ease of iterating on model selection, prompts, and experimental settings. The paper catalogs researcher degrees of freedom that make p-hacking and selective reporting easier to exploit in AI experiments while remaining difficult to detect, and calls for journals and conferences to adopt standardized preregistration templates.