AIBearisharXiv – CS AI · 15h ago7/10
🧠
Seeing vs. Believing: Evaluating the Language Bias of Open-Source MLLMs in Counter-Intuitive Scenes
Researchers introduced CAIT, a benchmark testing multimodal large language models' ability to understand counter-intuitive visual scenes that contradict common sense. The study reveals that open-source MLLMs fail dramatically at these tasks due to language bias, automatically overriding visual evidence with statistically common text patterns, while proprietary models like Claude and Gemini demonstrate robust performance.
🧠 Claude🧠 Gemini