On The Fragility of Benchmark Contamination Detection in Reasoning Models

New research reveals that benchmark contamination in language reasoning models (LRMs) is extremely difficult to detect, allowing developers to easily inflate performance scores on public leaderboards. The study shows that reinforcement learning methods like GRPO and PPO can effectively conceal contamination signals, undermining the integrity of AI model evaluations.

• →Contamination detection in language reasoning models is alarmingly easy to evade using standard training methods.
• →GRPO and PPO-style reinforcement learning training can effectively conceal benchmark contamination signals.
• →Chain-of-thought contamination in advanced models makes detection methods perform near random accuracy.
• →Model developers can achieve inflated leaderboard performance while leaving minimal contamination traces.
• →Current evaluation protocols for language reasoning models are fundamentally vulnerable to manipulation.