3D-VCD: Hallucination Mitigation in 3D-LLM Embodied Agents through Visual Contrastive Decoding

Researchers introduce 3D-VCD, an inference-time framework that reduces hallucinations in 3D-LLM embodied agents by contrasting predictions against distorted scene graphs. The method addresses failures specific to 3D spatial reasoning without requiring model retraining, advancing reliability in embodied AI systems.

This research tackles a critical challenge in embodied AI: hallucinations in large multimodal models operating within 3D environments. Unlike existing hallucination mitigation techniques designed for 2D vision-language tasks, 3D-VCD specifically addresses failures arising from incorrect object presence detection, spatial layout misunderstanding, and geometric grounding errors—issues fundamentally different from pixel-level inconsistencies in 2D systems.

The approach leverages contrastive decoding by deliberately corrupting 3D scene graphs through semantic perturbations (category substitutions) and geometric distortions (coordinate or extent corruption). By comparing model predictions under both original and distorted contexts, the framework identifies and suppresses tokens that rely on language priors rather than grounded scene evidence. This represents a meaningful shift toward interpretable reasoning in embodied agents.

For the AI development community, this work signals progress in making language models safer for physical-world deployment—a prerequisite for autonomous systems in robotics and spatial computing. The inference-time approach avoids expensive retraining, making adoption practical. Evaluation on 3D-POPE and HEAL benchmarks demonstrates consistent improvements in grounded reasoning, validating the method's effectiveness.

Looking ahead, researchers should monitor whether this technique generalizes across different 3D environments and whether similar contrastive frameworks emerge for other structured reasoning domains. The success of inference-time solutions may reshape how the field approaches reliability without retraining, particularly relevant as embodied AI systems move closer to real-world deployment where hallucinations pose safety and liability risks.

• →3D-VCD mitigates hallucinations in embodied agents through inference-time visual contrastive decoding over structured 3D scene graphs
• →The method addresses 3D-specific failures like spatial layout errors and object presence hallucinations, not just pixel-level inconsistencies
• →Semantic and geometric perturbations help distinguish language-prior-driven tokens from grounded reasoning signals
• →Inference-time approach requires no model retraining, enabling practical deployment with existing models
• →Consistent improvements on 3D-POPE and HEAL benchmarks validate effectiveness for more reliable embodied AI systems