AIBearisharXiv – CS AI · Jun 27/10
🧠
Prototypicality Bias Reveals Blindspots in Multimodal Evaluation Metrics
Researchers identify prototypicality bias as a systematic flaw in automated text-to-image evaluation metrics, where models prefer visually plausible but semantically incorrect images over accurate ones. The study introduces PROTOBIAS, a diagnostic benchmark revealing that widely-used metrics fail to prioritize semantic faithfulness to prompts, while proposing PROTOSCORE as a mitigation approach.