AIBearisharXiv – CS AI · 6h ago6/10
🧠
A Controlled Audit of Pretraining Contamination in Public Medical Vision-Language Benchmarks
Researchers audited major medical vision-language models for pretraining data contamination across public benchmarks like SLAKE-En and PathVQA, finding measurable image-side overlap (up to 19.8%) and text-side signals suggesting potential training data leakage. However, manual verification revealed distributional rather than pixel-level duplication, and several detection methods proved unreliable when tested against external baselines, raising questions about contamination assessment methodology.