FOCUS: Forcing In-Context Object Localization through Visual Support Constraints and Policy Optimization

Researchers introduce a two-stage training framework for in-context object localization that eliminates the need for category supervision, using visual support constraints and reinforcement learning to achieve robust instance-level localization. A 7B-parameter model trained with this approach outperforms significantly larger models up to 72B parameters, demonstrating that specialized training objectives can surpass pure model scaling.

This research addresses a fundamental limitation in current vision-language models: their inability to localize objects without explicit category labels. Traditional ICL methods rely on semantic priors learned during training, introducing bias that prioritizes categorical knowledge over actual visual evidence from support examples. This fragility becomes problematic for real-world applications involving unlabeled or instance-specific objects. The proposed framework tackles this by training models to focus on visual correspondence between support examples and query images rather than semantic categories.

The two-stage approach first optimizes in-context attention mechanisms without category supervision, then refines predictions through Group Relative Policy Optimization, a reinforcement learning technique that directly minimizes localization error. This design philosophy represents a shift in how researchers approach vision model training—moving away from category-based supervision toward visual grounding as the primary objective.

The efficiency gains demonstrated by the 7B model outperforming 72B models carry significant implications for both research and deployment. Smaller models require less computational resources, lower inference costs, and greater accessibility for developers. This challenges the prevailing assumption that scaling alone solves vision tasks. For applications like image editing, personalized visual search, and retrieval systems, improved category-agnostic localization opens new possibilities without requiring object labels.

Future developments should focus on real-world deployment scenarios, particularly how these techniques perform on novel object categories and edge cases. The research validates comprehensive ablations, suggesting reproducibility. Industry adoption depends on whether these improvements translate to practical advantages in production environments where inference speed and accuracy both matter.

• →A 7B-parameter model outperforms models up to 72B parameters using specialized training objectives, challenging the scaling-alone paradigm
• →The framework eliminates category supervision requirements, enabling robust localization of unnamed and instance-specific objects
• →Two-stage training combining visual attention optimization and reinforcement learning enforces visual correspondence over semantic priors
• →Significant efficiency gains reduce computational requirements while improving performance on in-context localization tasks
• →Results suggest that training objective design matters more than raw model size for certain vision-language tasks