Unsupervised Effectiveness Estimation Through Intersection of Ranking References

Abstract

Estimating the effectiveness of retrieval systems in unsupervised scenarios consists in a task of crucial relevance. By exploiting estimations which dot not require supervision, the retrieval results of many applications as rank aggregation and relevance feedback can be improved. In this paper, a novel approach for unsupervised effectiveness estimation is proposed based the intersection of ranking references at top-k positions of ranked lists. An experimental evaluation was conducted considering public datasets and different image features. The linear correlation between the proposed measure and the effectiveness evaluation measures was assessed, achieving high scores. In addition, the proposed measure was also evaluated jointly with rank aggregation methods, by assigning weights to ranked lists according to the effectiveness estimation of each feature.