Entropy-Based Filter Selection in CNNs Applied to Text Classification

Abstract

Filter selection in convolutional neural networks aims at finding the most important filters in a convolutional layer, with the goal of reducing computational costs and needed storage, as well as understanding the networks’ inner workings. In this paper we propose an entropy-based filter selection method that ranks filters based on the mutual information between their activations and the output classes using validation data. Our proposed method outperforms using filters’ absolute weights sum by a large margin, allowing to regain better performance with fewer filters.