AINeutralarXiv – CS AI · Mar 44/102
🧠
No Memorization, No Detection: Output Distribution-Based Contamination Detection in Small Language Models
Researchers developed CDD (Contamination Detection via output Distribution) to identify data contamination in small language models by measuring output peakedness. The study found that CDD only works when fine-tuning produces verbatim memorization, failing at chance level with parameter-efficient methods like low-rank adaptation that avoid memorization.