#reproducibility News & Analysis

79 articles tagged with #reproducibility. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

79 articles

AINeutralarXiv – CS AI · Apr 76/10

🧠

Reproducibility study on how to find Spurious Correlations, Shortcut Learning, Clever Hans or Group-Distributional non-robustness and how to fix them

A reproducibility study unifies research on spurious correlations in deep neural networks across different domains, comparing correction methods including XAI-based approaches. The research finds that Counterfactual Knowledge Distillation (CFKD) most effectively improves model generalization, though practical deployment remains challenging due to group labeling dependencies and data scarcity issues.

AINeutralarXiv – CS AI · Mar 26/1014

🧠

Jailbreak Foundry: From Papers to Runnable Attacks for Reproducible Benchmarking

Researchers introduce Jailbreak Foundry (JBF), a system that automatically converts AI jailbreak research papers into executable code modules for standardized testing. The system successfully reproduced 30 attacks with high accuracy and reduces implementation code by nearly half while enabling consistent evaluation across multiple AI models.

AIBullisharXiv – CS AI · Mar 35/104

🧠

PaperRepro: Automated Computational Reproducibility Assessment for Social Science Papers

Researchers introduced PaperRepro, a two-stage AI agent system that automates the assessment of computational reproducibility in social science research papers. The system achieved a 21.9% improvement over existing baselines on the REPRO-Bench benchmark by separating code execution from evaluation phases.

AINeutralarXiv – CS AI · Mar 34/105

🧠

Agentic Scientific Simulation: Execution-Grounded Model Construction and Reconstruction

Researchers introduce JutulGPT, an AI agent system for physics-based simulation that addresses the problem of underspecified natural language descriptions in scientific modeling. The system uses an execution-grounded approach where the simulator validates physical accuracy, but reveals limitations in tracking tacit assumptions made through simulator defaults.

← PrevPage 4 of 4