Manifold Learning for Brain Tumor MRI Image Retrieval and Classification

The evolution of image acquisition and storage technologies has been fundamental in numerous medical fields, supporting doctors to deliver more precise diagnoses and, consequently, recommend more effective treatments for their patients. Recently, deep learning techniques have played a key role in more accurate medical image analysis, mainly due to the capacity to effectively represent the image visual content. However, in spite of tremendous advances, deep-learning techniques commonly require huge quantities of data for training, that are not available in many scenarios, especially in the medical domain. Conversely, manifold learning techniques have been successfully applied in unsupervised and semi-supervised scenarios for more effective encoding of similarity relationships between multimedia data in the absence or restriction of labeled data. In this work, we propose to exploit jointly the representation power of deep-learning strategies with the ability of unsupervised manifold learning in delivering more effective similarity measurement. Convolutional Neural Networks (CNNs) and Transformer-based models trained through transfer learning are combined by unsupervised manifold learning methods, which define a more effective similarity among images. The output can be used for unsupervised retrieval and semi-supervised classification based on a k-NN strategy. An experimental evaluation was conducted on different datasets of MRI brain tumor images, considering different features. Effective results were obtained on both retrieval and classification tasks, with significant gains obtained by manifold learning approaches. In scenarios with limited training data, our approach achieves results that are competitive or superior to state-of-the-art deep learning approaches.