ELVA: Exploring Ranking-Driven Universal Multimodal Retrieval

Researchers introduce ELVA, a reinforcement learning framework that improves multimodal retrieval by addressing 'grain blindness'—where models fail to capture fine-grained query details. The approach treats negative samples with varying importance based on similarity and achieves 13.1% improvement on a new MRBench benchmark designed for multi-grain queries.

ELVA represents a meaningful advancement in multimodal AI systems, tackling a specific but consequential limitation in how current models process complex queries. Traditional contrastive learning treats negative examples uniformly, missing opportunities to extract nuanced information from samples at different similarity distances. This oversight becomes critical when users submit queries with multiple granular requirements—ELVA's ranking-driven approach differentiates negative samples by their proximity to positive ones, enabling the model to learn distinct semantic patterns at each level.

The research emerges from growing recognition that contrastive paradigms, while effective for general retrieval, struggle with query complexity. Prior work either ignored this limitation or lacked systematic frameworks to address it. By extending Reinforcement Learning with Verifiable Rewards (RLVR) to retrieval contexts, ELVA eliminates dependency on explicit ranking labels—a practical advantage that reduces annotation overhead. Rule-based rewards jointly optimize negative ranking while maximizing the similarity gap between relevant and irrelevant results.

The introduction of MRBench signals an important methodological contribution beyond the algorithm itself. Existing benchmarks may underweight multi-grain scenarios, potentially masking retrieval weaknesses in real-world applications. The 13.1% improvement on MRBench suggests ELVA captures performance gains where traditional metrics might miss them. For developers building multimodal search systems, this work offers both architectural insights and empirical validation that differential treatment of negatives yields measurable benefits.

The framework's effectiveness across standard benchmarks while excelling on multi-grain tasks indicates broad applicability without sacrificing baseline performance. Future research should explore how grain blindness manifests in different domains and whether similar ranking-driven strategies improve other retrieval variants.

• →ELVA addresses grain blindness in multimodal retrieval by treating negative samples with differentiated importance based on similarity distance
• →Rule-based RL rewards jointly optimize negative ranking and positive-negative similarity gaps without requiring explicit ranking labels
• →MRBench benchmark validates performance gains on multi-grain query scenarios where standard metrics may underperform
• →Framework achieves state-of-the-art results on standard benchmarks while showing 13.1% improvement specifically on complex queries
• →Approach eliminates reliance on reward models, reducing annotation overhead and improving scalability for practical applications