PInVerify: An Offline Embodied Benchmark for Active Instance Verification

Researchers introduce PInVerify, an offline benchmark for training embodied AI agents to verify whether objects match fine-grained descriptions through active viewpoint selection. The benchmark includes 3,000 episodes across 18 object categories and evaluates multimodal language models at on-device scale, with best results reaching 85.6% accuracy using fine-tuned approaches.

PInVerify addresses a critical limitation in embodied AI: while navigation to target objects has advanced significantly, agents struggle with fine-grained instance verification that requires close inspection and multi-angle assessment. This benchmark bridges the gap between reaching an object's vicinity and confirming it matches specific attributes like color or pattern variations. The research demonstrates that subtle semantic distinctions demand more than single-viewpoint analysis, forcing agents to make strategic navigation decisions.

The benchmark's design reflects real-world constraints through its 6-sector navigation topology, which includes trap views—navigable but uninformative positions—that realistically model environmental challenges. This structure prevents agents from achieving artificially high accuracy through brute-force multi-view capture. The evaluation across multiple open-source models (Qwen3-VL, SenseNova-SI, CLIP, SigLIP2) provides a comprehensive baseline landscape for the AI research community.

Key findings reveal important limitations in current approaches: the 4.9 percentage-point gap between MLLM-based and embedding-based baselines indicates that vision-language understanding substantially outperforms pure embedding strategies. However, the absence of reliable gains from active viewpoint selection strategies suggests next-best-view algorithms require fundamental improvements. The 3.1 percentage-point detection gap from ground-truth bounding boxes highlights that object localization remains a bottleneck.

For the embodied AI and robotics communities, PInVerify establishes concrete benchmarks for real-world deployment scenarios where agents must distinguish between visually similar objects. The open-source code release enables broader adoption and iterative improvements. Success on this task directly translates to practical applications in warehouse automation, household robotics, and inventory management where precise instance identification matters.

• →PInVerify introduces a 3,000-episode benchmark requiring agents to verify object attributes through active multi-viewpoint inspection.
• →Multimodal language models significantly outperform embedding baselines by 4.9 percentage points in fine-grained verification tasks.
• →Current next-best-view selection strategies fail to deliver reliable gains, indicating algorithmic improvements are needed.
• →Ground-truth bounding box analysis reveals a 3.1 percentage-point performance gap, highlighting object localization as a remaining bottleneck.
• →The benchmark's trap-view design realistically models environmental constraints that prevent artificially inflated accuracy metrics.