A DVDrive Approach for doScenes Instructed Driving Challenge

Researchers submitted a vision-language-action driving agent called OmniDrive to the doScenes Instructed Driving Challenge, which predicts autonomous vehicle trajectories based on visual context, motion history, and natural language instructions. The team introduced a divided-view perception module that improves multi-camera visual grounding by reducing cross-view interference, enabling better alignment between language instructions and driving-relevant visual evidence.

This research addresses a critical frontier in autonomous driving: instruction-conditioned trajectory prediction. Traditional prediction systems rely on visual perception and historical motion patterns, but this challenge adds a layer of complexity by incorporating natural language instructions that guide the vehicle's future behavior. OmniDrive represents an evolution in vision-language-action architectures, bridging perception, reasoning, and planning into a unified framework capable of understanding both what the environment contains and what the driver intends to do.

The divided-view perception enhancement (DVPE-style module) marks a technical advancement in multi-camera fusion. Rather than processing all camera inputs globally, the architecture segments visual data into local view spaces with visibility-aware attention mechanisms. This localized approach fundamentally improves how the model grounds language instructions to relevant visual evidence, reducing computational interference and improving interpretability. Such efficiency gains matter significantly for real-time autonomous driving applications where latency directly impacts safety.

For the autonomous vehicle industry, this work demonstrates measurable progress toward instruction-following systems that could enable more intuitive human-vehicle interaction. As autonomous systems become more sophisticated, the ability to accept and execute natural language commands opens deployment possibilities in ride-sharing, delivery, and enterprise logistics. The public code release accelerates collaborative development across the research community.

Looking forward, instruction-conditioned prediction could become standard in next-generation autonomous stacks. Key metrics to watch include real-world safety validation, generalization to unseen driving scenarios, and whether instruction-following capabilities reduce the reliance on pre-mapped routes.

• →OmniDrive combines vision-language-action understanding for instruction-conditioned trajectory prediction in autonomous vehicles.
• →A novel divided-view perception module improves multi-camera grounding by reducing cross-view interference through localized attention.
• →The approach generates 12-waypoint 6-second future trajectories aligned with natural language driving instructions.
• →Public code release enables broader research adoption and community-driven improvements in instruction-following autonomous systems.
• →Technical advancement in language-grounding for autonomous driving may accelerate deployment of command-responsive vehicle systems.