y0news
AnalyticsDigestsSourcesTopicsRSSAICrypto

#ai-reproducibility News & Analysis

2 articles tagged with #ai-reproducibility. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

2 articles
AIBearisharXiv – CS AI · Jun 57/10
🧠

Search-Time Contamination in Deep Research Agents: Measuring Performance Inflation in Public Benchmark Evaluation

Researchers identify Search-Time Contamination (STC) in deep research agents, where web search during inference allows models to access benchmark answers and metadata, artificially inflating performance by up to 4%. The study reveals widespread contamination across six public benchmarks and calls for contamination-aware evaluation practices including sandboxed environments and transparent search tracking.

🏢 Meta
AIBullisharXiv – CS AI · Jun 17/10
🧠

Fully Open Meditron: An Auditable Pipeline for Clinical LLMs

Researchers introduce Fully Open Meditron, the first completely transparent pipeline for building clinical AI systems that exposes training data, curation procedures, and generation methods. The framework achieves state-of-the-art performance on medical benchmarks while maintaining full auditability and reproducibility, addressing a critical gap in transparent healthcare AI.