VisioMath: Benchmarking Figure-based Mathematical Reasoning in LMMs

Researchers introduced VisioMath, a new benchmark with 1,800 K-12 math problems designed to test Large Multimodal Models' ability to distinguish between visually similar diagrams. The study reveals that current state-of-the-art models struggle with fine-grained visual reasoning, often relying on shallow positional heuristics rather than proper image-text alignment.

• →VisioMath benchmark exposes significant weaknesses in current LMMs when comparing visually similar mathematical diagrams
• →Model accuracy consistently declines as inter-image similarity increases across both closed-source and open-source systems
• →The primary failure mode is image-text misalignment, with models using positional shortcuts instead of proper reasoning
• →Three alignment-oriented strategies showed substantial accuracy improvements in testing
• →The research highlights the need for better multi-image comparative reasoning capabilities in AI systems