GarmentPile++: Affordance-Driven Cluttered Garments Retrieval with Vision-Language Reasoning

Researchers developed GarmentPile++, an AI pipeline that uses vision-language models to retrieve individual garments from cluttered piles following natural language instructions. The system integrates visual affordance perception with dual-arm robotics to handle complex garment manipulation tasks in real-world home assistant applications.

• →GarmentPile++ addresses the real-world challenge of retrieving garments from cluttered piles rather than single-item scenarios.
• →The system combines vision-language models with visual affordance perception for high-level reasoning and low-level action execution.
• →A dual-arm cooperation framework handles large garments and incorrect grasping scenarios that single-arm systems cannot manage.
• →The pipeline uses SAM2 visual segmentation to enhance VLM awareness of individual garment states within piles.
• →Testing demonstrates effectiveness across diverse tasks in both simulation and real-world environments.