MuPHI: Learning Implicit Multimodal Harm Reasoning via Semantically Grounded Reward Optimization

Researchers introduce MuPHI, a dataset and training framework for detecting implicit multimodal harm in image-text pairs where danger emerges from context-dependent reasoning rather than surface features. The proposed MuPHIRM framework uses reward optimization to improve vision-language models' ability to reason about compositional harm while demonstrating stronger generalization to out-of-distribution scenarios.

This research addresses a critical gap in AI safety by tackling the challenge of detecting harm that emerges from multimodal context rather than explicit visual or textual cues alone. Current vision-language models struggle with compositional reasoning where benign images combined with benign text create harmful semantics—a vulnerability that poses real risks as these systems increasingly mediate human interactions. The MuPHI dataset provides a structured benchmark for this underexplored problem, containing annotated harm rationales that enable systematic evaluation of VLM reasoning chains.

The field has largely focused on detecting surface-level harmful content while overlooking the pragmatic dimension where meaning depends on implicit context and intent. This mirrors broader challenges in AI safety where systems must understand not just what is said, but what is meant. MuPHIRM's reward optimization approach represents progress toward interpretability and robustness by forcing models to develop reasoning capabilities aligned with human judgment about harm.

For AI developers and safety practitioners, this work carries immediate implications. Systems deployed in content moderation, educational contexts, or sensitive applications need to handle compositional harm detection—a capability that current approaches largely lack. The demonstrated out-of-distribution robustness is particularly valuable, suggesting the framework resists adversarial evasion tactics. As multimodal AI becomes ubiquitous, building systems that generalize beyond benchmark shortcuts rather than exploiting surface patterns becomes essential for trustworthy deployment.

• →MuPHI dataset enables evaluation of vision-language models on implicit, context-dependent harm reasoning across diverse categories
• →MuPHIRM framework uses multi-perspective reward optimization to improve both harm detection accuracy and reasoning quality
• →The approach demonstrates superior out-of-distribution robustness compared to standard training and inference-time baselines
• →Compositional harm detection—where benign components create harmful meaning—represents an underexplored safety challenge for multimodal AI
• →Reasoning-augmented training shows promise for building AI systems that generalize beyond benchmark-specific shortcuts