HilDA: Hierarchical Distillation with Diffusion for Advancing Self-Supervised LiDAR Pre-trainin

HilDA introduces a self-supervised pretraining framework for LiDAR systems in autonomous driving by combining hierarchical knowledge distillation from Vision Foundation Models with diffusion-based temporal consistency. The approach achieves state-of-the-art results on cross-modal distillation benchmarks and improves performance across 3D object detection, scene flow, and semantic occupancy prediction tasks.

HilDA addresses a critical bottleneck in autonomous driving development: the scarcity of annotated LiDAR data needed to capture the geometric and kinematic diversity of real-world driving scenarios. Rather than treating Vision Foundation Models as static black-box teachers, the framework extracts multi-layer semantic information and global scene context, enabling more sophisticated knowledge transfer than traditional frame-wise feature matching approaches.

The technical innovation combines three complementary mechanisms: hierarchical distillation across multiple model layers ensures progressive semantic alignment, global context distillation captures scene-level semantics beyond individual frames, and temporal occupancy diffusion enforces spatiotemporal consistency across LiDAR sequences. This layered approach directly addresses limitations in existing cross-modal distillation methods that ignore the rich structural information available in pre-trained vision models.

For the autonomous driving industry, HilDA's performance gains across multiple downstream tasks—3D detection, scene flow estimation, and occupancy prediction—demonstrate practical value for production systems. The framework enables more efficient model training without massive labeled datasets, reducing development costs and accelerating the deployment timeline for AD systems. The released code democratizes access to this advancement, allowing researchers and practitioners to build on the approach.

The broader implications extend to any robotics or perception system requiring robust 3D understanding from multi-modal sensor inputs. As autonomous systems become increasingly critical infrastructure, improvements in pre-training efficiency and cross-modal knowledge transfer directly impact safety and deployment feasibility.

• →HilDA improves LiDAR pre-training by extracting multi-layer semantic structure and global context from Vision Foundation Models rather than treating them as black boxes.
• →The framework combines hierarchical distillation with temporal occupancy diffusion to capture both semantic and spatiotemporal consistency in driving scenarios.
• →State-of-the-art results on cross-modal distillation benchmarks with measurable improvements in 3D object detection, scene flow, and semantic occupancy tasks.
• →Reduces annotation burden for autonomous driving development by leveraging existing vision models, accelerating AD system deployment timelines.
• →Open-sourced implementation enables broader adoption and refinement within robotics and autonomous systems research communities.