Difference-Aware Retrieval Policies for Imitation Learning

Researchers present DARP, a semi-parametric retrieval-based approach to imitation learning that improves upon standard behavior cloning by predicting actions based on k-nearest neighbors from training data rather than learning a global policy. The method achieves 15-46% performance improvements across continuous control and robotic manipulation tasks without requiring additional data collection or expert feedback.

DARP addresses a fundamental limitation in parametric imitation learning: the compounding error problem that occurs when policies encounter states outside their training distribution. Rather than training a single global policy model, DARP leverages the training dataset itself during inference as a retrieval mechanism, combining nearest-neighbor similarity with learned action predictions. This hybrid approach represents a pragmatic solution to the distribution shift problem that has plagued behavior cloning in robotics and control applications.

The technical innovation reparameterizes imitation learning around local neighborhood structure, where the model learns to predict actions based on relative distances between query states and retrieved expert demonstrations. This design choice exploits the intuition that similar states should have similar actions, grounding predictions in concrete training examples. The method maintains simplicity—requiring no additional assumptions beyond standard behavior cloning—while achieving substantial empirical gains across diverse domains.

For the robotics and AI research community, DARP offers immediate practical value. The 15-46% performance improvements translate to more reliable autonomous systems without expensive data collection or interactive learning loops. High-dimensional visual representations support real-world deployment scenarios. The open-sourced implementation and demo materials accelerate adoption.

Looking forward, semi-parametric approaches like DARP may become standard components in deployed imitation learning systems. Research should explore scaling these methods to larger datasets, investigating theoretical generalization bounds, and adapting retrieval strategies to continuous high-dimensional state spaces where nearest-neighbor assumptions weaken. Integration with transformer-based architectures could further enhance performance.

• →DARP uses k-nearest neighbor retrieval during inference to reduce distribution shift errors in imitation learning
• →Empirical results show 15-46% performance improvements over behavior cloning across multiple domains
• →Semi-parametric approach requires no additional data collection or expert feedback beyond standard behavior cloning
• →Method demonstrates robustness across continuous control, robotic manipulation, and visual feature representations
• →Open-source implementation enables rapid adoption by robotics and AI research communities