MASER: Modality-Adaptive Specialist Routing for Embodied 3D Spatial Intelligence

Researchers introduce MASER, a framework that dynamically routes questions to specialized adapters of a vision-language model based on modality relevance, achieving 51.3% oracle agreement on the Open3D-VQA benchmark. The approach demonstrates that no single modality optimally answers all spatial reasoning questions, with point clouds proving superior in over half of test cases.

MASER addresses a fundamental limitation in embodied AI systems: the assumption that a single modality-optimized model can effectively handle diverse spatial reasoning tasks. Traditional vision-language models are fine-tuned on one modality, forcing all questions through the same inference pathway regardless of whether the query would benefit from different sensory inputs like point clouds, depth maps, or RGB images. This research reveals that question semantics naturally align with specific modalities—point clouds excel in 51.5% of cases—yet existing architectures ignore this heterogeneity.

The technical elegance of MASER lies in its efficiency. Rather than maintaining five independent models, the framework shares a VLM backbone across five lightweight adapters and trains a routing MLP on question embeddings from a frozen sentence transformer. This design minimizes computational overhead while achieving specialized performance. The routing network reaches 51.3% oracle agreement, substantially outperforming a random-forest baseline at 43.5%, demonstrating that learned routing policies capture meaningful patterns in question-modality alignment.

For the embodied AI and 3D vision communities, MASER's findings carry practical implications. Developers deploying spatial reasoning systems should expect heterogeneous performance across modalities rather than seeking universal solutions. The work validates adaptive routing as a scalable approach to multi-modal reasoning without proportional increases in model parameters or inference latency. Moving forward, researchers should investigate whether this modality selection pattern generalizes across different 3D environments, datasets, and question distributions, and whether more sophisticated routing mechanisms could approach oracle performance more closely.

• →Point clouds are optimal for 51.5% of spatial reasoning tasks, challenging single-modality design assumptions
• →MASER's neural routing achieves 51.3% oracle agreement using lightweight adapters on a shared VLM backbone
• →The framework requires only one adapter call per question, maintaining computational efficiency despite multi-modal capabilities
• →No universal modality dominates 3D spatial reasoning, indicating task-dependent modality selection is necessary
• →Random-forest routing underperforms learned neural routing by 7.8 percentage points, validating deep learning approaches