Common Sense vs. Morality: The Curious Case of Narrative Focus Bias in LLMs

Researchers have identified a critical flaw in Large Language Models (LLMs) where they prioritize moral reasoning over commonsense understanding, struggling to detect logical contradictions within moral dilemmas. The study introduces the CoMoral benchmark and reveals a 'narrative focus bias' where LLMs better identify contradictions attributed to secondary characters rather than primary narrators.

• →LLMs consistently prioritize moral reasoning over basic commonsense understanding when evaluating scenarios.
• →A new benchmark called CoMoral was created to test LLMs' ability to identify commonsense contradictions within moral dilemmas.
• →Ten different LLMs across various sizes showed consistent struggles in detecting logical contradictions without explicit signals.
• →LLMs exhibit narrative focus bias, performing better at identifying contradictions from secondary characters versus primary narrators.
• →The research highlights the need for improved reasoning-aware training to enhance LLMs' commonsense robustness.