Almieyar-Oryx-BloomBench: A Bilingual Multimodal Benchmark for Cognitively Informed Evaluation of Vision-Language Models

Researchers introduce BloomBench, a bilingual English-Arabic benchmark grounded in Bloom's Taxonomy to rigorously evaluate Vision-Language Models across six cognitive levels. The study reveals that state-of-the-art VLMs excel at semantic understanding but struggle with factual recall and creative synthesis, while exposing significant performance gaps between Arabic and English reasoning tasks.

BloomBench addresses a critical gap in AI evaluation methodology by moving beyond task-specific benchmarks toward cognitively structured assessment frameworks. Rather than measuring isolated capabilities, the benchmark systematically diagnoses weaknesses across cognitive layers, revealing that current VLMs possess uneven cognitive profiles masked by strong average performance metrics. This approach mirrors how educational psychology has long understood learning—progress isn't linear, and strengths in one domain can conceal deficiencies in others.

The bilingual dimension adds crucial nuance often overlooked in AI research. The documented Arabic-English performance gap suggests that multimodal reasoning capabilities don't transfer uniformly across languages, likely reflecting training data disparities and architectural biases toward English-centric datasets. This finding has implications beyond academic rigor; it indicates that widely-deployed VLMs may provide degraded service to non-English speakers in practical applications, from medical diagnosis systems to legal document analysis.

For AI developers and organizations building on VLM infrastructure, BloomBench provides diagnostic value similar to how clinical assessments guide medical treatment. Rather than celebrating benchmark improvements, teams can now identify whether gains come from genuine reasoning advancement or superficial pattern matching. The framework's open availability democratizes this diagnostic capability across research institutions. Looking forward, this work likely catalyzes methodological shifts in how AI progress is measured, potentially influencing funding decisions and research priorities toward addressing specific cognitive weaknesses rather than pursuing marginal overall improvements.

• →State-of-the-art VLMs demonstrate strong semantic understanding but fail substantially at factual recall and creative tasks, indicating uneven cognitive development.
• →The bilingual benchmark reveals significant performance degradation in Arabic compared to English, exposing critical cross-lingual reasoning limitations.
• →Bloom's Taxonomy-based evaluation framework provides more diagnostic value than existing benchmarks by mapping performance to specific cognitive layers.
• →Current VLM proficiency metrics mask deeper limitations in particular cognitive domains, necessitating more granular evaluation approaches.
• →The open-source framework enables broader industry adoption of cognitive-aligned evaluation methodologies for future model development.