From Open Vocabulary to Open World: Teaching Vision Language Models to Detect Novel Objects

Researchers have developed a framework that enables open vocabulary object detection models to operate in real-world settings by identifying and learning previously unseen objects. The method introduces techniques called Open World Embedding Learning (OWEL) and Multi-Scale Contrastive Anchor Learning (MSCAL) to detect unknown objects and reduce misclassification errors.

• →Traditional object detection models are limited to detecting only predefined objects from their training sets.
• →Open vocabulary detection models currently rely on accurate prompts and struggle with misclassifying similar unknown objects.
• →The new framework introduces OWEL to detect far-out-of-distribution objects using pseudo unknown embeddings in semantic space.
• →MSCAL technique helps identify misclassified unknown objects by improving consistency of object embeddings across different scales.
• →The method achieves state-of-the-art performance on autonomous driving benchmarks while maintaining open vocabulary capabilities.