When to Call an Apple Red: Humans Follow Introspective Rules, VLMs Don't

Researchers introduce the Graded Color Attribution dataset to test whether Vision-Language Models faithfully follow their own stated reasoning rules. The study reveals that VLMs systematically violate their introspective rules in up to 60% of cases, while humans remain consistent, suggesting VLM self-knowledge is fundamentally miscalibrated with serious implications for high-stakes deployment.

This research exposes a critical flaw in how Vision-Language Models operate: they can articulate decision rules but fail to follow them consistently. The Graded Color Attribution benchmark cleverly isolates this behavior by having VLMs establish color-attribution thresholds, then measuring whether their subsequent decisions honor those thresholds. The disparity is striking—GPT-4-mini violates its own rules nearly 60% of the time when objects have strong color associations, despite accurately estimating pixel coverage. This reveals that VLM failures stem not from inability to perceive or calculate, but from a fundamental disconnect between reasoning and action.

The research carries significant implications for AI trustworthiness. Previous work attributed VLM errors to task difficulty or knowledge gaps; this study demonstrates the problem runs deeper into the architecture itself. World-knowledge priors—the model's learned associations between objects and colors—systematically override stated rules, suggesting VLMs lack genuine introspective self-knowledge. Human participants, by contrast, exhibit rule-faithful behavior with violations explained by a well-understood cognitive bias about color perception.

For deployment in high-stakes domains like medical imaging, autonomous systems, or legal document analysis, this matters profoundly. Users relying on VLM explanations for critical decisions may trust reasoning that the model subsequently violates. The miscalibration of self-knowledge means users cannot rely on model introspection as a safety mechanism. This finding challenges the narrative that scaling or fine-tuning will resolve VLM failures, suggesting instead that fundamental architectural changes may be necessary. Organizations deploying VLMs must account for this introspective faithfulness gap through external verification rather than trusting internal reasoning chains.

• →VLMs establish clear decision rules but violate them systematically, contradicting assumptions about reasoning reliability in AI systems.
• →World-knowledge priors override stated rules in VLMs, causing faithfulness rates to drop to 40%, unlike human cognition patterns.
• →VLM perception and calculation accuracy mask deeper reasoning failures, making errors harder to detect through conventional testing.
• →Current VLM introspective self-knowledge is miscalibrated, making internal explanations unreliable safety mechanisms.
• →High-stakes VLM deployment requires external verification protocols rather than reliance on model-provided reasoning transparency.