MOSAIC: Unveiling the Moral, Social and Individual Dimensions of Large Language Models

Researchers introduce MOSAIC, the first comprehensive benchmark to evaluate moral, social, and individual characteristics of Large Language Models beyond traditional Moral Foundation Theory. The benchmark includes over 600 curated questions and scenarios from nine validated questionnaires and four platform-based games, providing empirical evidence that current evaluation methods are insufficient for assessing AI ethics comprehensively.

• →MOSAIC is the first large-scale benchmark designed to jointly assess moral, social, and individual characteristics of LLMs beyond Moral Foundation Theory.
• →The benchmark comprises over 600 curated questions and scenarios drawn from moral philosophy, psychology, and social theory.
• →Testing across three different model families demonstrated that MFT alone is insufficient for comprehensive ethical evaluation of AI systems.
• →The dataset and Python library are publicly released as an extensible resource for researchers.
• →LLMs are increasingly deployed in sensitive applications like healthcare and psychological support, making ethical evaluation critical.