QueryGaussian: Scalable and Training-Free Open-Vocabulary 3D Instance Retrieval

QueryGaussian introduces a training-free framework for retrieving 3D instances from massive scenes using natural language prompts, achieving 70% GPU memory reduction and 180x faster inference compared to existing methods. The approach decouples semantic understanding from geometric representation through instance-level queries rather than scene-level embeddings, enabling practical deployment on consumer hardware for city-scale environments with millions of 3D primitives.

QueryGaussian addresses a critical scalability bottleneck in 3D instance retrieval—a task fundamental to autonomous systems, robotics, AR/VR applications, and smart city infrastructure. Existing methods embed semantic information directly into every 3D primitive, creating quadratic resource demands as scenes grow. This architectural constraint has practically limited deployment to small-scale environments, making the technology unsuitable for real-world applications requiring analysis of entire urban landscapes.

The research represents a significant methodological shift in computer vision. Rather than attempting holistic semantic distillation across entire scenes, QueryGaussian uses a two-stage process: leveraging pre-trained 2D vision models to interpret natural language queries, then lifting 2D segmentation results into 3D space through weighted association strategies and temporal fusion. This decoupling allows the system to process arbitrarily complex scenes without the memory overhead that plagued prior approaches.

The practical implications are substantial. By reducing GPU memory usage by over 70% and accelerating inference 180-fold, QueryGaussian democratizes advanced 3D retrieval capabilities. Organizations can now perform sophisticated scene analysis on consumer-grade hardware rather than requiring expensive server infrastructure. This efficiency gain particularly benefits robotics companies, autonomous vehicle developers, and spatial computing platforms that currently face prohibitive computational costs.

Looking forward, this work establishes a template for scaling other complex 3D vision tasks. The success of decoupled semantic-geometric processing may influence how researchers approach 3D scene understanding more broadly, potentially spawning similar optimizations across the computer vision field.

• →QueryGaussian reduces GPU memory consumption by over 70% and accelerates inference 180x compared to state-of-the-art methods
• →The framework operates training-free using pre-trained 2D vision models, avoiding fine-tuning costs and data requirements
• →Instance-level query mechanism decouples semantic understanding from 3D geometry, solving the architectural bottleneck of prior scene-level embedding approaches
• →System handles city-scale 3D scenes with tens of millions of Gaussians on consumer-grade hardware, enabling practical real-world deployment
• →Temporal fusion with adaptive density clustering mitigates projection ambiguity when lifting 2D segmentation masks to 3D space