Distilling 3D Spatial Reasoning into a Lightweight Vision-Language Model with CoT

Researchers have developed a knowledge distillation framework that compresses a 7B 3D vision-language model into a 2.29B student model, achieving 8.7x faster inference while retaining 54-72% performance. The approach introduces "Hidden CoT," learnable latent tokens that enable spatial reasoning without explicit chain-of-thought training data, making 3D scene understanding feasible on resource-constrained devices.

This research addresses a critical bottleneck in deploying advanced 3D vision-language models: computational efficiency. Large-scale VLMs like LLaVA-3D demonstrate strong spatial reasoning capabilities but require substantial hardware resources, limiting real-world deployment in edge computing, robotics, and mobile applications. The distillation framework tackled this by systematically transferring knowledge from a teacher to a significantly smaller student model while preserving core functionality.

The innovation centers on two key technical contributions. First, uncertainty-aware loss weighting in multi-task distillation optimizes which knowledge matters most during compression. Second, the "Hidden CoT" mechanism represents a novel approach to reasoning: learnable latent tokens function as an internal scratchpad, enabling the model to perform complex spatial reasoning without explicit chain-of-thought examples during training. This is particularly valuable since CoT data is expensive to annotate, especially for 3D understanding tasks.

The results demonstrate practical viability. The 3x model size reduction and 8.7x latency improvement enable deployment scenarios previously infeasible with larger models, while maintaining 54-72% accuracy on proximity and contact tasks across ScanNet and 3D-FRONT benchmarks. This performance-efficiency tradeoff suggests the framework could unlock applications in autonomous systems, augmented reality, and spatial computing on constrained hardware.

Looking forward, the latent scratchpad reasoning concept may inspire similar approaches across other distillation tasks. The research validates that spatial understanding doesn't require massive models, potentially democratizing 3D AI capabilities and reducing computational overhead in production systems.

• →Knowledge distillation reduced 3D VLM size by 3x and inference latency by 8.7x while preserving 54-72% of teacher performance.
• →Hidden CoT introduces learnable latent tokens as internal reasoning mechanisms without requiring explicit chain-of-thought training data.
• →Student model jointly performs spatial description, depth estimation, and object detection on resource-constrained platforms.
• →Framework achieves 68-72% accuracy on proximity and contact tasks, demonstrating strong spatial understanding despite compression.
• →Latent scratchpad reasoning represents the first application of this technique in distilled 3D vision-language models.