MAGNIFIED: RL Fine-tuning of Multimodal Large Language Models for Motion Planning

Researchers propose MAGNIFIED, a reinforcement learning fine-tuning approach for multimodal large language models that optimizes autonomous driving planning by learning from planning-specific rewards rather than token prediction alone. Testing on the Waymo Open Motion Dataset shows substantial improvements including 10.5% reduction in trajectory overlap and 38.9% reduction in off-road violations compared to supervised fine-tuning baselines.

MAGNIFIED addresses a fundamental misalignment between how large language models are traditionally trained and the actual requirements of autonomous vehicle planning. While pre-training and supervised fine-tuning focus on next-token prediction accuracy, real-world driving demands multi-step reasoning that considers safety margins, trajectory efficiency, and compliance with traffic rules. The research demonstrates that token-level prediction objectives inadequately capture planning constraints.

This work builds on the growing recognition that general-purpose large language models require task-specific alignment beyond standard imitation learning. Recent advances in reinforcement learning from human feedback (RLHF) have shown promise in constraining model behavior, and this paper applies similar principles to the autonomous driving domain with planning-oriented reward signals rather than human preference rankings.

The technical approach maps predicted token sequences to vehicle trajectories, enabling direct optimization against planning metrics like collision avoidance and staying within road boundaries. Validation on the Waymo dataset—a large-scale real-world driving dataset—provides credible evidence of practical effectiveness. The 38.9% reduction in off-road rate particularly suggests the method learns genuine safety constraints rather than memorizing training patterns.

This advancement matters for the autonomous vehicle industry as it bridges the gap between language model capabilities and domain-specific requirements. For AI developers, it illustrates how reward function design can dramatically improve task performance beyond standard fine-tuning. The methodology could extend beyond driving to other sequential decision-making domains like robotics or logistics planning.

• →Reinforcement learning fine-tuning substantially outperforms supervised fine-tuning for autonomous driving planning tasks.
• →Planning-specific reward signals enable language models to optimize for safety and compliance beyond token prediction accuracy.
• →MAGNIFIED demonstrates 10.5% overlap reduction and 38.9% off-road rate improvement on Waymo dataset benchmarks.
• →Token-level rewards map predicted text sequences to vehicle trajectories for direct planning metric optimization.
• →The approach indicates broader potential for task-aligned reinforcement learning across sequential decision-making domains.