Unsupervised Effectiveness Estimation Measure Based on Rank Correlation for Image Retrieval

Abstract

In recent years, the amount of image data has increased exponentially, driven by advancements in digital technologies. As the volume of data expands, the efforts required for labeling also escalate, which is costly and time-consuming. This scenario highlights the critical need for methods capable of delivering effective results in scenarios with few or no labels at all. In unsupervised retrieval, the task of Query Performance Prediction (QPP) is crucial and challenging, as it involves estimating the effectiveness of a query without labeled data. Besides promising, the QPP approaches are still largely unexplored for image retrieval. Additionally, recent approaches require training and do not exploit rank correlation to model the data. To address this gap, we propose a novel QPP measure named Accumulated JaccardMax, which considers contextual similarity information and innovates by exploiting a recent rank correlation measure to assess the effectiveness of ranked lists. It provides a robust estimation by analyzing the ranked lists in different neighborhood depths and does not require any training or labeled data. Extensive experiments were conducted across 5 datasets and over 20 different features including hand-crafted (e.g., color, shape, texture) and deep learning (e.g., Convolutional Networks and Vision Transformers) models. The results reveal that the proposed unsupervised measure exhibits a high correlation with the Mean Average Precision (MAP) in most cases, achieving results that are better or comparable to the baseline approaches in the literature.