GeoSAM-3D: Geodesic Prompt Propagation for Open-Vocabulary 3D Scene Segmentation from Monocular Video

GeoSAM-3D introduces a novel approach to 3D scene segmentation from monocular video by combining foundation models with Gaussian Splatting and geodesic propagation, enabling users to segment objects with simple clicks or text prompts without requiring RGB-D cameras or pre-reconstructed meshes.

GeoSAM-3D represents a meaningful advancement in democratizing 3D scene understanding by lowering hardware requirements for practical segmentation tasks. Traditional approaches demand calibrated multi-view imagery or RGB-D sensors, creating barriers for broader adoption. This work eliminates those constraints by operating on standard monocular video, making the technology accessible to users with commodity cameras and smartphones.

The technical innovation centers on geodesic propagation rather than Euclidean distance metrics. This distinction proves critical because heat-kernel distances respect surface topology, preventing segmentation leakage across geometrically close but semantically separate objects—a common failure mode in 3D nearest-neighbor approaches. By leveraging frozen foundation models (image and video), the system avoids expensive retraining while maintaining flexibility through prompt-based interaction.

For computer vision and robotics developers, this work enables faster iteration on 3D understanding tasks without reconstructing full scene geometry upfront. The approach sits at an interesting intersection: it's powerful enough for practical applications yet lightweight enough for rapid prototyping. The separation of evaluation into implementation validation, propagation quality, leakage control, and latency testing provides a rigorous framework others can adopt.

The broader impact depends on open-source availability and real-world performance on diverse scenes. If the codebase proves stable and latency acceptable for interactive use cases, this could accelerate adoption of 3D scene understanding in AR/VR applications, robotics, and content creation tools. The lightweight monocular requirement particularly benefits mobile and edge computing scenarios where sensors are constrained.

• →GeoSAM-3D enables 3D object segmentation from monocular video with user prompts, eliminating RGB-D or multi-view camera requirements
• →Geodesic heat-kernel propagation on scene graphs preserves surface continuity and prevents leakage across disconnected objects better than Euclidean distance
• →The system combines frozen foundation models with differentiable Gaussian Splatting, avoiding expensive retraining while maintaining interactivity
• →Evaluation framework separates validation concerns including propagation quality, leakage control, and interactive latency
• →Lower hardware requirements and simple click-or-name interface could expand 3D segmentation accessibility for AR/VR and robotics applications