Blind Dexterous Grasping via Real2Sim2Real Tactile Policy Learning

Researchers developed a framework for teaching dexterous robotic hands to grasp objects using only touch sensation, without visual input or real-world demonstrations. The approach combines tactile sensor calibration, geometry-aware learning, and diffusion-based policy aggregation to achieve 27% grasp success on both seen and unseen objects.

This research addresses a fundamental challenge in robotics: enabling dexterous manipulation through tactile feedback alone. The work tackles the sim-to-real gap—the persistent mismatch between simulated and real-world sensor behavior—through a contact-calibrated digital-twin approach that ensures simulated tactile signals accurately reflect physical hardware. This is particularly significant because tactile sensors generate sparse, high-dimensional data that traditional learning methods struggle to interpret effectively.

The framework's three-component architecture reflects broader trends in robotics research. Real2Sim calibration pipelines have gained prominence as researchers recognize that perfect simulation is unnecessary if the simulation accurately captures the specific phenomena relevant to the task. The layout-aware tactile encoder using self-supervised pretraining aligns with recent advances in representation learning across modalities. The diffusion-based policy aggregation represents an emerging pattern of leveraging generative models as tools for combining multiple learned behaviors, moving beyond traditional reinforcement learning paradigms.

For the robotics and AI industry, this work demonstrates viability of contact-driven manipulation without vision—critical for applications in dark environments, occluded scenarios, or when visual processing is infeasible. The 27% success rate on unseen objects suggests the approach generalizes meaningfully despite the inherent limitations of tactile-only perception. This has implications for robot designers considering sensor placement and for developers building manipulation systems requiring robust blind-grasping capabilities.

Future research should focus on improving absolute success rates and extending the framework to more complex, dynamic manipulation tasks. The integration of multiple tactile calibration techniques and exploration of how diffusion policies compare to other aggregation methods will likely drive the next generation of tactile-driven robotic systems.

• →Real2Sim tactile calibration successfully bridges simulation-to-reality gaps by reproducing accurate contact signals on physical hardware
• →Layout-aware tactile encoding with self-supervised pretraining significantly improves sparse tactile signal expressiveness for learning
• →Diffusion-based policy aggregation effectively combines object-specific grasp trajectories for generalization to unseen objects
• →Blind dexterous grasping achieves 27% success on real robotic hands without visual input or human demonstrations
• →Contact-event consistency between simulation and hardware is critical for deploying purely tactile manipulation policies