#scientific-reproducibility News & Analysis

2 articles tagged with #scientific-reproducibility. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

2 articles

AIBullisharXiv – CS AI · Apr 157/10

🧠

Towards grounded autonomous research: an end-to-end LLM mini research loop on published computational physics

Researchers demonstrate an autonomous LLM agent capable of executing a complete research loop—reading, reproducing, critiquing, and extending computational physics papers. Testing across 111 papers reveals the agent identifies substantive flaws in 42% of cases, with 97.7% of issues requiring actual computation to detect, and produces a publishable peer-review comment on a Nature Communications paper without human direction.

AINeutralarXiv – CS AI · May 16/10

🧠

Can Large Language Models Implement Agent-Based Models? An ODD-based Replication Study

Researchers evaluated 17 large language models on their ability to implement agent-based models from standardized specifications, finding that while GPT-4.1 and Claude 3.7 Sonnet produce statistically valid implementations, executability alone doesn't guarantee scientific reliability. The study reveals both significant promise and critical limitations in using LLMs as automated tools for scientific model engineering and replication.

🧠 GPT-4🧠 Claude