MVR: Multi-view Video Reward Shaping for Reinforcement Learning

Researchers introduce Multi-View Video Reward Shaping (MVR), a new reinforcement learning framework that uses multi-viewpoint video analysis and vision-language models to improve reward design for complex AI tasks. The system addresses limitations of single-image approaches by analyzing dynamic motions across multiple camera angles, showing improved performance on humanoid locomotion and manipulation tasks.

• →MVR framework uses multi-viewpoint videos instead of single static images for better reinforcement learning reward shaping.
• →The system leverages frozen pre-trained vision-language models to learn state relevance functions for complex dynamic tasks.
• →State-dependent reward formulation automatically reduces VLM guidance influence once desired motion patterns are achieved.
• →Testing on HumanoidBench and MetaWorld tasks demonstrates superior performance over existing image-based methods.
• →The approach mitigates bias towards specific static poses that plague single-viewpoint reward systems.