Feature Fusion for Graph Convolutional Networks in Semi-Supervised Image Classification

Abstract

In recent years, the volume of multimedia data has been rapidly increasing across various applications. Consequently, classification methods capable of handling scenarios with limited labeled data (e.g., semi-supervised, weakly supervised) have become critically important, particularly because acquiring labeled data is often expensive and time-consuming. Regarding image data, feature extraction approaches are commonly employed in many tasks. Feature extraction involves identifying and extracting key characteristics or patterns, such as edges, textures, shapes, and colors. Nowadays, most extractors consider deep learning strategies, such as Convolutional Neural Networks (CNNs) and Vision Transformers (ViT). With various feature extractors available in the literature, there is a wide diversity of features that can be considered. The features extracted from an image depend on the application, the extractor used, and its configuration. Therefore, combining different extractors can be a promising strategy to exploit complementary information. Graph Convolutional Networks (GCNs) are fundamental and promising strategies in the scenario of semi-supervised image classification, being able to leverage labeled and unlabeled data, and exploiting the graph structures that offer valuable information. This work proposes an approach for GCNs in scenarios where labeled data is scarce, combining sets of features and graphs considering different extraction approaches. Among the main contributions, the experimental results reveal that these combinations and the use of manifold learning to process these graphs improve the classification results in most cases.