Probing Collision Grounding in Vision-Language Models for Safe Human-Robot Collaboration

Researchers introduce TouchSafeBench, a physics-grounded benchmark for evaluating how well vision-language models can detect robot collisions with humans and objects. Testing three frontier VLMs reveals critical safety gaps, with best performance below 50% accuracy, exposing that visual fluency in AI models does not guarantee physical safety accountability in real-world human-robot collaboration scenarios.

This research addresses a critical vulnerability in deploying vision-language models for physical robotics applications. While VLMs have achieved impressive capabilities in visual understanding and description, the study demonstrates they fundamentally lack grounding in spatial geometry, robot morphology, and temporal prediction—essential requirements for safe human-robot interaction. TouchSafeBench's evaluation across 2,940 simulated episodes with calibrated sensor data and physics-derived labels reveals that current models struggle particularly with human-proximity risks and robot-scene contact prediction, despite explicit depth information being available.

The gap between visual fluency and physical accountability represents a systemic problem in embodied AI. Models trained primarily on internet-scale image-text data develop sophisticated pattern recognition without understanding metric geometry or collision dynamics. This limitation becomes acute in safety-critical applications where failure modes have real consequences. The benchmark's finding that robot-scene contact detection is harder than human-contact classification suggests models may be biased toward human-centric visual features while neglecting rigid body dynamics.

For the robotics industry, these results indicate that vision-language models cannot yet serve as reliable autonomous safety monitors without substantial architectural improvements. Developers implementing human-robot collaboration systems must supplement VLM-based perception with explicit physics simulation, depth integration, and motion planning constraints. The research establishes a rigorous evaluation framework that can drive progress toward physically-grounded models, though it simultaneously validates concerns about deploying current-generation VLMs in production environments where safety guarantees matter.

• →Leading vision-language models achieve below 50% accuracy on collision detection tasks despite strong visual understanding capabilities
• →Current VLMs fail to automatically translate explicit depth information into collision risk assessment
• →Visual fluency in AI models does not correlate with physical safety accountability in robotic systems
• →Human-robot contact risk assessment proves easier than robot-environment collision detection across all tested models
• →Future embodied AI systems require explicit grounding in viewpoint geometry, robot morphology, and temporal motion prediction