GRAPH CONVOLUTIONAL NETWORKS AND MANIFOLD RANKING FOR MULTIMODAL VIDEO RETRIEVAL
Nenhuma Miniatura disponível
Data
2022-01-01
Autores
de Almeida, Lucas Barbosa [UNESP]
Valem, Lucas Pascotti [UNESP]
Pedronette, Daniel Carlos Guimarães [UNESP]
Título da Revista
ISSN da Revista
Título de Volume
Editor
Resumo
Despite the impressive advances obtained by supervised deep learning approaches on retrieval and classification tasks, how to acquire labeled data for training remains a challenging bottleneck. In this scenario, the need for developing more effective content-based retrieval approaches capable of taking advantage of multimodal information and advances in unsupervised learning becomes imperative. Based on such observations, we propose two novel approaches that combine Graph Convolutional Networks (GCNs) with rank-based manifold learning methods. The GCN models were trained in an unsupervised way, using the Deep Graph Infomax algorithm, and the proposed approaches employ recent rank-based manifold learning methods. Multimodal information is exploited through pre-trained CNNs via transfer learning for extracting audio, image, and video features. The proposed approaches were evaluated on three public action recognition datasets. High-effective results were obtained, reaching relative gains up to +29.44% of MAP compared to baseline approaches without GCNs. The experimental evaluation also considered classical and recent baselines in the literature.
Descrição
Palavras-chave
graph convolutional networks, manifold learning, rank aggregation, video multimodal retrieval
Como citar
Proceedings - International Conference on Image Processing, ICIP, p. 2811-2815.