Ranking vs. Assignment: The Metric Mismatch in Multi-View Object Association

Researchers identify a fundamental mismatch between pairwise ranking metrics (AP and FPR-95) commonly used to evaluate multi-view object association models and the actual one-to-one assignment objective these systems aim to solve. The study demonstrates that optimal ranking performance does not guarantee correct assignments, and proposes Sinkhorn-based normalization as a solution to better align evaluation metrics with real-world performance goals.

Multi-view object association represents a critical problem in computer vision where systems must match objects across multiple camera feeds—a task naturally suited to constrained one-to-one matching formulations. However, the field has drifted toward using pairwise ranking metrics like Average Precision (AP) and False Positive Rate at 95% recall (FPR-95) for model evaluation, despite these metrics measuring something fundamentally different from the assignment task itself.

The research reveals a significant theoretical gap: models can achieve perfect AP and FPR-95 scores while still producing incorrect assignments, and conversely, optimal ranking does not guarantee optimal assignments. This mismatch stems from how ranking metrics evaluate individual prediction pairs in isolation rather than considering the global constraints of one-to-one matching. The authors propose Sinkhorn-based normalization to bridge this gap, a post-processing technique that reframes the problem through optimal transport theory.

The practical implications are substantial for the computer vision community. Current benchmark comparisons may mislead researchers about true model performance, as reported metrics can improve dramatically through parameter tuning without actual assignment accuracy gains. This affects downstream multi-camera perception systems used in autonomous vehicles, surveillance, and robotics, where assignment correctness directly impacts system reliability.

The findings suggest that future multi-view association research should adopt assignment-level metrics like ACC and IPAA as primary evaluation criteria, with ranking metrics serving only supplementary roles. This metric realignment could reshape how researchers design and validate object association models, potentially leading to more robust real-world systems.

• →Pairwise ranking metrics (AP, FPR-95) do not reliably reflect assignment accuracy in multi-view object association tasks.
• →Sinkhorn-based normalization can reconcile the mismatch between ranking metrics and actual one-to-one matching objectives.
• →Current benchmark evaluations may overstate model performance by optimizing metrics that don't correlate with assignment correctness.
• →Assignment-level metrics (ACC, IPAA) should become primary evaluation criteria for multi-camera perception systems.
• →Post-processing parameter tuning can artificially inflate ranking metrics without improving real-world assignment performance.