Transduction to induction: Unsupervised representation learning based on rank information

Abstract

The use of deep learning in supervised scenarios has become well-established. However, there is growing interest in exploring unsupervised learning methods. Transductive approaches are promising for learning rich contextual relationships in unsupervised scenarios but face challenges when dealing with large amounts of data. The main motivation of this study is to investigate the feasibility of an inductive model, based on a neural network, learning representations from ranked lists generated by transductive methods in unsupervised scenarios. We propose an unsupervised approach called Inductive Ranking Learning (IRL), which leverages techniques to learn similarities and dissimilarities from pairs derived from ranked lists produced by transductive methods. This technique involves weighting the most relevant and irrelevant elements when calculating the error of likely positive and negative pairs, based on the position of the element in the ranked list relative to its pair. This allows learning without the need for labels. The proposed approach enables the use of transductive techniques to train inductive models, promoting generalization to unseen data, which is particularly important in scenarios where new data is constantly being introduced. Experimental results show promising performance, although the method may face challenges when dealing with ranked lists derived from large datasets. Overall, the proposed approach offers significant potential for both unsupervised learning and the exploration of transductive approaches in inductive models.